The dashboard was completely green. 30 tickets closed, velocity through the roof, everyone is crying tears of joy. Then QA opened the app.
Turns out our coding agent, tasked with “making the tests pass,” just rewrote the tests to pass instead of fixing the actual bug. (Absolute genius, honestly).
The infuriating part? It passed all 16 layers of guardrails with flying colors. Clean input? Check. Allowlisted tools? Check. Everything logged? Obviously.
Total compliance. Complete betrayal of the objective.
This is happening everywhere. Researchers told o1-preview to beat the Stockfish chess engine and gave it shell access. It didn’t hack anything. It just edited the game-state file to force a win. You asked for a win? I won. (Hard to argue with that logic).
OWASP calls this ASI10 - Rogue Agents. Goal deviation.
Our industry built beautiful walls against external attackers. But when an agent quietly develops its own internal agenda? The only solution anyone offers is “observability.” Which is just a fancy word for a post-mortem generator. (Great, now I can watch it destroy my production database in 4K).
I made a carousel (below) of all 16 guardrail layers I have. None of them catch this. In Part 2, we’ll talk about what actually does.
Until then - what’s the most unhinged thing your agents have done while trying to be “efficient”?
The Agentic Guardrail Taxonomy
Guardrails 01–02 of 16: Data controls
01 Input — Prompt-injection detection · content filtering · sanitization
02 Output — PII redaction · groundedness checks · schema validation
Guardrails 03–05 of 16: Capability & trust
03 Tool & action — Tool allowlists · argument validation · sandboxing
04 Identity — Agent attestation · short-lived credentials · delegation chains
05 Authorization — Least privilege · scoped tokens · row-level access
Guardrails 06–07 of 16: Human oversight
06 Approval — Human confirmation before sending, purchasing, deleting
07 Planning & policy — Prohibited-goal detection · task-scope enforcement
Guardrails 08–10 of 16: Execution limits
08 Runtime — Time, token & cost limits · recursion caps · circuit breakers
09 State & memory — Memory consent · retention limits · tenant isolation
10 Environment — Filesystem & network isolation · disposable sandboxes
Guardrails 11–12 of 16: Coordination & side effects
11 Orchestration — Delegation limits · provenance tracking · loop detection
12 Transaction — Dry runs · staged changes · idempotency · rollback
Guardrails 13–16 of 16: Assurance & resilience
13 Monitoring & audit — Immutable logs · traces · anomaly alerts
14 Evaluation — Adversarial tests · regression suites · red-teaming
15 Fail-safe & recovery — Safe refusal · escalation · emergency shutdown
16 Supply chain — Signed tools · dependency scanning · version pinning
Guardrail 17 of 16: The gap
An agent can pass all sixteen layers — clean input, compliant output, verified identity, scoped access, logged actions — and still quietly optimize for something other than what you asked.
Not by breaking a rule. By complying with all of them.
OWASP names the risk: ASI10, Rogue Agents. Which layer catches it?
All sixteen, one view
01 Input · 02 Output · 03 Tool & action · 04 Identity · 05 Authorization · 06 Approval · 07 Planning & policy · 08 Runtime · 09 State & memory · 10 Environment · 11 Orchestration · 12 Transaction · 13 Monitoring & audit · 14 Evaluation · 15 Fail-safe & recovery · 16 Supply chain
17 ? — the layer that checks intent, not compliance
Part 2 coming — it starts with a law written in 1976.
