Threat Crystallization: How a New Agent Attack Becomes a 3ms Rule in About an Hour

Threat crystallization is this: an AI reads a new agent attack once, spends a few seconds understanding why it works, and that understanding is written down as a deterministic regex rule. From then on the rule runs in milliseconds for every install, with no model in the hot path. New attack to shipped rule takes about an hour. A committee rulebook takes weeks to months.

I build ATR (Agent Threat Rules), an open, MIT-licensed detection standard, and PanGuard, the free product built on it. This post explains the mechanism behind a term I keep using, threat crystallization, because it is the actual reason the corpus grew to nearly 690 rules (688 as of July 2026, npm agent-threat-rules v3.5.6) across 10 categories without me hand-writing each one alone.

What does threat crystallization actually mean?

It means separating two jobs that are usually mashed together: understanding an attack, and detecting it.

Understanding is slow and expensive. A large model reads a malicious skill or a poisoned tool response, reasons about the intent, and decides whether it is an attack. That reasoning costs seconds and real tokens, and you do not want to pay it on every tool call for every user forever.

Detection should be fast and free. Once the model has understood the attack, the durable part of that understanding is a pattern: a specific instruction shape, an exfiltration primitive, a suspicious argument. That pattern gets written as a regex rule. The rule is a frozen artifact of the reasoning. It runs in milliseconds, deterministically, the same way for everyone, and it never needs the model again.

That is the crystallization: liquid reasoning cooled into a solid rule. The AI judged it once. The rule executes forever.

Why regex instead of running the model every time?

Because determinism and speed are the whole point of a Layer 1 defense. A rule that runs in milliseconds can sit inline before a skill installs or before a tool call executes, and it behaves identically on every machine. You can read it, audit it, version it, and reproduce exactly why it fired. A model call in the hot path is slow, costs money per invocation, and gives a slightly different answer each time.

The measured behavior of these deterministic Layer 1 rules, always cited with their corpus, holds up. On NVIDIA garak's in-the-wild jailbreak corpus (650 samples), the rules hit 97.2% recall. On a self-built PINT-format corpus (850 samples, drawn from deepset/prompt-injections and Lakera Gandalf, not Lakera's official private PINT), 99.7% precision at 63.6% recall. On HackAPrompt (EMNLP 2023, 4,780 samples), 69.6% recall against a 28.6% baseline at 100% precision. On a 498-sample real-world SKILL.md corpus, 100% recall and 97% precision at 0.2% false positives.

One honest caveat: there is no single engine-wide false-positive number. False positives are lane-based. Against a 65,000-sample benign gate, the enforce lane runs about 0.24% FP and the hunt lane, which is the default, runs about 9%. Anyone who quotes you one blended precision figure for a detection engine is selling something.

How does the flywheel work?

Every scan that finds something new can become a rule that protects everyone. That is the loop.

I ran a wild scan across six public registries: 96,096 AI agent skills, 751 confirmed malicious. Each novel malicious pattern in that pile is a candidate for crystallization. The model understands it once, the pattern becomes a rule, the rule ships to the standard, and the next person who runs pga scan is protected before they ever install the thing. The cost of understanding an attack is paid one time by one machine. The benefit is shared across every install.

This is also why the corpus can grow daily instead of quarterly. A new attack does not wait for a standards committee to convene. It waits for one model to understand it and one gate to pass.

What stops a bad rule from shipping?

A gate, because the failure mode of any fast-moving rule pipeline is shipping a rule that flags legitimate behavior. I have made that mistake. An auto-merge pipeline once merged eleven false-positive rules before I caught it, which is exactly why the gate exists now.

Before a crystallized rule is allowed into the standard it has to survive the 65,000-sample benign gate and prove it still catches its own true positives. A rule that fires on clean skills does not ship. A rule that no longer detects the attack it was written for does not ship. Crystallization without a gate is just a faster way to generate noise.

Threat crystallization vs a committee rulebook

The point is not that committees are useless. ATR maps to real committee-driven frameworks: 866 rule-to-category mappings across all 10 OWASP Agentic Top 10 (2026) categories, and 34 of 101 top-level MITRE ATLAS techniques. The point is that the gap between discovering a live attack and defending against it should be an hour, not a quarter. The threats do not wait. postmark-mcp, a trusted MCP server, silently BCC'd 3,000 to 15,000 emails a day for fifteen versions before anyone noticed. MCPJam Inspector (CVE-2026-23744) shipped a default 0.0.0.0 binding that turned one HTTP request into RCE for every version before v1.4.3. Speed of response is not a nicety here.

FAQ

Is threat crystallization the same as an LLM firewall?

No. An LLM firewall runs a model on every request. Crystallization runs the model once to understand an attack, then ships a deterministic regex rule so the model is never in the hot path. You get the model's judgment at regex speed and cost.

Does putting the model in front make it too slow to use inline?

That is the reason for the design. Because detection is regex, a Layer 1 rule runs in milliseconds and can sit inline before a skill installs. The expensive reasoning happened earlier, once, on someone else's machine.

Why is there no single false-positive number?

Because false positives are lane-based. Against a 65,000-sample benign gate the enforce lane runs about 0.24% FP and the default hunt lane about 9%. A single blended engine-wide FP number would hide that tradeoff.

How do I try it?

Run npm install -g @panguard-ai/panguard && pga up, then pga scan <target> to check a skill or MCP server before you install it. It is free and MIT-licensed, and it auto-detects Claude Code, Cursor, Windsurf, Gemini CLI, Cline, Codex CLI, and more.

The next attack against your agents already exists somewhere. Scan a skill before you install it with pga scan <target>, and if you find something new, that finding can become a rule that protects everyone.