Why does PanGuard report false positives in lanes instead of one number?

PanGuard reports false positives in lanes because a single blended "0.3% FP" number is a lie of convenience. On a 65,000-sample benign gate, the Layer 1 deterministic rules produce roughly 0.24% false positives in the enforce lane and roughly 9% in the hunt lane, which is the default. Those are two different jobs with two different error budgets. Averaging them into one figure would tell you nothing about the number that actually decides whether a detection is safe to auto-block.

I publish the 9% on purpose. It is the worst number I have, and hiding it would be the fastest way to lose the only thing an open standard has: trust.

What is the difference between the enforce lane and the hunt lane?

The enforce lane holds rules that are allowed to block. A false positive here breaks a legitimate agent run, so the bar is deliberately punishing. On the 65,000-sample benign corpus these rules sit at about 0.24% false positives. That is the number you care about if you let PanGuard Guard stop an agent mid-execution.

The hunt lane is the default. It holds broader, higher-recall rules that surface suspicious behavior for review rather than blocking it. These fire more often on benign input, about 9% on the same gate, because their job is to not miss things, not to be silent. A hunt-lane hit is a lead, not a verdict.

One number cannot describe both lanes at once. If I told you "0.3% FP" and you then wired the hunt lane into an auto-block pipeline, you would ship a 9% false-positive rate into production and blame the tool when it broke your agents. The lane is the context that makes the number usable.

Why publish the worst number on purpose?

Because the alternative is the industry norm, and the industry norm is marketing. Vendors quote the precision of their tightest rule set on their friendliest corpus and print it as if it describes the whole engine. It does not. When a customer runs the real thing on real traffic and sees 9%, the gap between the slide and the terminal is where trust dies.

I would rather you distrust the 9% today than discover it yourself next quarter. Publishing the worst lane is a costly signal. It only makes sense to send if the underlying methodology is honest, so sending it is evidence that it is.

Here is how the two lanes measure up on the 65,000-sample benign gate:

For context, the other benchmarks in the ATR corpus are reported the same way, always with their corpus named. On NVIDIA garak, an in-the-wild jailbreak corpus of 650 samples, the Layer 1 rules hit 97.2% recall. On a self-built PINT-format corpus of 850 samples, they reach 99.7% precision at 63.6% recall. On HackAPrompt (EMNLP 2023, 4,780 samples), 69.6% recall at 100% precision against a 28.6% baseline. On a SKILL.md corpus of 498 real-world samples, 100% recall and 97% precision at 0.2% FP. Every one of those numbers is tied to a named corpus. None of them is "the" precision of the engine, because there is no such single number.

What is the generalization gate?

A rule that only catches its own test case is worthless. It has memorized one attack string, not learned a pattern. The generalization gate is the check that stops those rules from shipping.

Every candidate rule has to do two things at once. It has to catch attack samples it was never written against, proving it generalizes beyond its own author's example. And it has to stay quiet on the 65,000-sample benign gate, proving it did not generalize by getting sloppy. A rule that passes its own true positive but fails on unseen attacks is a memorizer. A rule that catches everything but lights up on benign traffic is a broad-brush FP machine. The gate rejects both.

This is also why the FP number is honest by construction. The benign gate is not a courtesy check I run at the end. It is the wall every rule has to clear before it enters a lane, and the lane it lands in is decided by how cleanly it clears that wall. The 0.24% and the 9% are outputs of the gate, not numbers I chose.

How does this connect to threat crystallization?

PanGuard's model is that an AI understands an attack once, that understanding is written down as a deterministic rule, and the rule then executes in milliseconds for everyone. New attack to shipped rule takes about an hour. The generalization gate and the lane system are the quality control on that pipeline. Crystallization is fast, so the gate has to be strict, or fast just means you ship bad rules faster. The lane is where a rule proves how much you are allowed to trust it.

FAQ

Is there a single false-positive number for PanGuard?

No. False positives are lane-keyed: about 0.24% in the enforce lane and about 9% in the hunt lane on a 65,000-sample benign gate. Any single blended FP figure hides which detections are safe to auto-block.

Why is the default hunt lane's 9% acceptable?

Because the hunt lane surfaces suspicious behavior for human review rather than blocking it. Its job is high recall, so a higher benign hit rate is the expected cost of not missing real attacks. Only enforce-lane rules, at ~0.24%, are allowed to block.

What stops a rule from just memorizing its test case?

The generalization gate. A candidate rule must catch attack samples it was not written against and stay quiet on the 65,000-sample benign gate before it can enter any lane. Memorizers and broad-brush FP rules both fail.

Is ATR the same thing as PanGuard?

No. ATR is an independent, open, MIT-licensed standard that PanGuard is built on and operates, governed independently. The relationship is like Sigma or CVE and the vendor that operates it.

You can check the lanes yourself. Install with npm install -g @panguard-ai/panguard && pga up, then run pga scan <target> against a skill or MCP server before you install it. The rules, the lanes, and the benign gate are all open.