Dynamic analysis observes what an agent skill actually does by running it inside an isolated sandbox and recording the resulting behavior, rather than inferring intent from source code alone. It's the layer that catches what static analysis structurally cannot: behavior that only appears once the skill is actually invoked.

Safe execution, not full execution

The obvious concern with running a potentially malicious skill is that you might actually get harmed by it. A well-designed sandbox handles this by executing only safe, local, non-destructive workflows, setup checks, dry runs, mock executions, help or version commands, using harmless dummy inputs, while explicitly refusing to carry out anything genuinely dangerous. If a skill attempts to fetch and execute an unknown remote script, access real credentials, or perform an irreversible action, the sandbox skips it and records exactly what was attempted and why. That skipped attempt is still valuable evidence. It just doesn't require letting the harmful action actually happen.

What gets collected

A useful sandbox run produces four kinds of runtime evidence: what the agent itself reported doing or skipping, low-level system call traces showing file access and process creation, network captures showing DNS queries and outbound connections, and a record of every file created, modified, or deleted during the run. Correlating these four sources is what turns raw activity into a verdict. A successful open() call against a sensitive file, on its own, could be an innocent check. That same successful read followed immediately by a connect() call to an unfamiliar external domain is a very different signal.

A malicious sample with confirmed shadow features contained repeated file access to sensitive paths alongside an unauthorized outbound connection, evidence a runtime engine calculated at a risk score high enough to confidently flag the behavior as malicious, independent of how the underlying source code was written.

Filtering out the noise

Not everything that looks suspicious in a sandbox trace is meaningful. Commands like checking whether a package manager exists, a syntax check, or a simple import probe are common, benign operational noise, not malicious evidence. Files created inside a designated sandbox working directory are typically expected workflow artifacts, not payloads, unless they're later executed or correlated with stronger evidence. And a failed attempt to open a credential file, one that returns "file not found" or "permission denied", is not the same as successfully reading it. Filtering these cases out is what keeps dynamic analysis from simply trading static analysis's false positive problem for a different one.

FAQ

Does dynamic analysis replace the need for static analysis? No, the two are complementary. Static analysis is cheap and can run before a skill ever executes. Dynamic analysis is more expensive computationally and catches what only shows up at runtime. Most effective pipelines use static analysis as an early filter and dynamic analysis as a validation layer.

What happens if a skill's malicious behavior only triggers under a specific condition the sandbox didn't hit? This is a genuine limitation. A benign result from a single sandbox run doesn't prove the absence of malicious behavior if that behavior depends on a specific credential, input, or delayed condition the run never exercised.

Is sandboxed execution safe to run at scale, given the cost of spinning up isolated containers? It's more expensive than static analysis, which is why most teams position it as a validation layer for skills that have already passed an initial static filter, rather than the very first check every artifact goes through.

Related reading: "Why Static Analysis Alone Can't Catch Malicious Agent Skills," "One Verdict Isn't Enough: The Case for Hybrid Agent Skill Scanning"