We built our detection pipeline around a simple belief: no single analysis layer is sufficient to judge whether an agent skill is malicious. Static rules read code without running it. Sandboxes observe behavior but only within the run they were given. Reasoning over both catches the intent a rule or a log alone can miss. When we tested this against a balanced set of 100 malicious and 100 benign skills, the numbers backed up exactly why all three layers need to work together.

What each layer caught on its own

Our static layer, using severity-weighted rule matching, reached an ROC AUC of 0.78 on its own, useful as an early, cheap filter, but with real overlap between benign and malicious scores. Our dynamic sandbox layer improved on that slightly, an AUC of 0.80, by directly observing runtime behavior like credential access attempts and unauthorized network connections. Both layers still assigned meaningfully high risk scores to a share of completely benign packages, a mean of roughly 34 to 35 out of 100 on packages that were, in fact, safe.

Reasoning over the combined evidence from both layers is where the real separation appeared. Our context-aware evaluation layer reached an AUC of 0.985, pushing benign packages down to a median score of just 2.5 while malicious packages sat at a median of 92.5. Put together, the full hybrid pipeline reached an F1 score of 0.98 on our 200-skill benchmark, with zero false positives.

On a balanced 200-skill benchmark, our hybrid pipeline reached 98% precision with zero false positives, while the best individually-tested standalone approach in our evaluation topped out around 83.5% accuracy, roughly one wrong verdict in six.

Why stacking layers isn't automatically better

It's worth being honest about a counterintuitive finding here: combining detectors doesn't automatically improve results. When one layer already dominates the discriminative signal, adding more detectors on top can introduce noise instead of value. The gain from a hybrid approach doesn't come from simply running more scanners at once. It comes from giving each layer a distinct job it's actually suited for, static rules for cheap pre-deployment triage, sandboxing for behavioral validation, and reasoning for resolving ambiguity, and letting later layers correlate and adjudicate the evidence earlier layers produce, rather than just voting alongside them.

What this means operationally

A skill that trips a static rule isn't automatically guilty, and a skill that passes static review isn't automatically safe. The static layer's job is to triage cheaply and flag for further review. The dynamic layer's job is to validate that flag against actual observed behavior. The reasoning layer's job is to decide what the combined evidence actually means, particularly for the ambiguous cases where a skill mentions "credentials" or "command injection" in a legitimate, defensive context rather than a malicious one.

FAQ

Doesn't adding more analysis layers just slow everything down? It adds latency at the layers that require it, which is why static analysis runs on every pull request while sandboxed execution is better suited to a pre-deployment or periodic re-scan schedule, and reasoning-based evaluation runs selectively when the earlier evidence is genuinely ambiguous.

How much of the improvement comes from the reasoning layer specifically? In our evaluation, it was the largest single jump, from an AUC in the high 0.70s and 0.80s for static and dynamic analysis independently to 0.985 once evidence from both was correlated and reasoned over in context.

Is a hybrid approach overkill for a small engineering team? The static layer alone is inexpensive enough to run everywhere. The judgment call is how far down the pipeline you invest in dynamic and reasoning-based validation, and that should scale with how much of your environment already depends on unreviewed third-party agent skills.

Related reading: "Why Static Analysis Alone Can't Catch Malicious Agent Skills," "Building a Malicious Score: Severity-Weighted Risk Scoring for Agent Skills"