Early on, our scanner returned a number. The number was defensible, derived from severity-weighted static rules, sandboxed runtime behaviour, and a reasoning pass over both. But a number on its own asks a security engineer to take it on faith, and security engineers correctly decline to do that.

The problem with a bare score

The failure mode was consistent. An engineer would see a high score on a skill they recognised as harmless, have no way to check the reasoning, and lose confidence in every subsequent score. One unexplained false positive is enough to discount a tool entirely, because from the outside there's no way to distinguish a tool that's wrong once from a tool that's wrong often.

The reverse was equally unhelpful. A low score on something genuinely suspicious gave no thread to pull on. Either way the verdict was a conversation ender rather than a starting point, which is the opposite of what a triage signal should be.

What a verdict carries now

Every verdict now travels with the evidence that produced it: which static rules matched and at what severity, the specific code or instruction snippet that triggered each one, what the sandbox actually observed at runtime, which dangerous actions were attempted and safely skipped, and the filesystem and network activity recorded during execution.

Crucially, it also carries how the evidence was obtained. A pattern that was merely mentioned in a skill's instructions is weighted differently from one the sandbox watched execute, which is weighted differently again from one corroborated by two independent runtime artifacts. That provenance is now visible rather than folded silently into a single figure.

The change wasn't to the detection. It was to what the detection hands back. The scores barely moved; the arguments about them largely stopped.

What it changed

Two things, both practical. Disputes got shorter, because disagreeing with a verdict now means disagreeing with a specific piece of evidence rather than with the tool as a whole. And the ambiguous middle became workable: a skill whose documentation legitimately discusses credential handling in a defensive context now shows exactly that reasoning, instead of surfacing as an unexplained elevated score a reviewer has to either trust or dismiss.

It also made the false positives useful. A wrong verdict with visible evidence tells us precisely which rule over-fired and why, which is a considerably faster path to fixing it than a wrong number with nothing attached.

FAQ

Does this make the output harder to consume at scale? No, because the evidence is structured rather than prose. Automated consumers can act on the verdict and confidence alone; the evidence is there when a human needs to adjudicate a specific case.

Did surfacing the evidence change the underlying scores? Very little. The scoring model was already severity-weighted and provenance-aware. What changed is that the reasoning is now inspectable rather than implicit.

What if the evidence itself is ambiguous? Then the verdict says so. Ambiguous cases route to human review rather than being resolved silently in either direction, which we'd rather do explicitly than hide behind a confident-looking number.

Related reading: "Building a Malicious Score: Severity-Weighted Risk Scoring for Agent Skills," "The Real Cost of False Positives in Agent Skill Security"