The Layer the Cookbook Doesn’t Govern

If you want to know how the industry builds AI agents right now, you read Anthropic’s cookbook. It is the closest thing the field has to a canonical build guide, from the lab that arguably understands agents best. So I read the whole thing looking for one specific thing, and it is instructive precisely because of what is not there.

The cookbook ships a genuinely impressive catalog of autonomous agents. A Site Reliability agent that reads logs and runbooks, pinpoints a root cause, and opens a fix PR. A threat-intelligence agent that autonomously queries multiple intel sources and maps findings to MITRE ATT&CK. A vulnerability-detection agent that threat-models a target and hunts memory-safety bugs. Async multi-agent orchestration with peer messaging and dynamically-spawned subagents. A chief-of-staff agent coordinating a team through hooks and plan mode. These are excellent at what they scope. And every one of them is governed the same two ways: per action (is this individual step allowed?) and, at best, by a human approving one step (the SRE agent waits for someone to approve its PR).

Take the most sophisticated case, because it is the fair one. The SRE agent doesn’t just act — it waits for a human to approve the fix. That is real, careful governance. But it is governance of one decision: this PR, right now. Nothing in the pattern asks the question that matters when you run that agent a thousand times a day: across everything it has read, escalated, and touched this session, what has its authority composed into? The approval gate is on the last step. The trajectory is unwatched.

The blind spot is structural, not a bug

A governance gate, in almost every system I have looked at — the cookbook included — is memoryless. It scores one decision (is this action permitted? is this spend approved? is this risk acceptable?) and forgets it before scoring the next. That is usually the right design for a single action. It leaves exactly one thing uncovered: a risky sequence of individually-fine actions. Ten actions each 0.5 on a 0-to-1 risk scale are all below a 0.65 per-call threshold, so a memoryless gate waves each one through, correctly, in isolation. But 0.5, ten times, is a trajectory, and a gate that forgets the last decision cannot see a trajectory. It cannot add up what it refuses to remember.

Stack the layers the build canon does cover and the gap is obvious by what comes after it. Identity establishes the acting principal. Authorization constrains its permissions. Policy evaluates individual actions. Tool-scoping limits what each role can call. And then… nothing evaluates the accumulated trajectory. There is no layer whose job is to ask: given everything this principal has done in the last N hours, what has it composed into? That missing layer is the subject of a preprint I published this month — “Authorized but Composed” (DOI 10.5281/zenodo.21400261). And the reason the omission matters is not academic: the Hugging Face breach three weeks ago was exactly this shape — roughly 17,000 individually-authorized-looking actions, run by an autonomous agent framework, that composed into an intrusion no single step would have been denied for. The build canon teaches you to build that agent. It does not teach you to govern what it accumulates to.

An autonomous agent doesn’t need one forbidden action. It needs a sequence of permitted ones that composes into an outcome no single gate would deny.

I want to be careful here, because the easy version of this argument is wrong and I’m not making it. The cookbook is not deficient, and these agents are not unsafe. Per-action policy and human approval are the right controls for what they govern, and Anthropic scopes them well. My claim is narrower and, I think, more useful: governance of accumulated authority is out of scope in the canonical build guide. When the most authoritative reference in the field treats a layer as out of scope, that is the strongest evidence I know of that the layer is a real, unclaimed category — not a solved problem, and not my invention either. It is simply the next layer up, and almost no one is building it yet.

What a composition layer actually does

Cross-session risk composition is a stateful layer that sits above the per-action gate — the layer that would come after “tool-scoping” in that stack. For each subject — an agent, a session, an identity, a credential, a campaign — it accumulates a risk weight per decision, composes those weights across a rolling window with optional time decay, and escalates when the accumulation crosses a threshold, even when every individual decision passed its own gate. A worked example ships in the open-source engine’s tests: an agent whose seven consecutive decisions each evaluate to RUN individually composes past the fail threshold, and the composed verdict becomes FREEZE. No single gate saw a problem. The sum did. That is the shape an SRE-style agent’s session presents — a principal whose every action is individually approvable while its accumulated authority climbs unwatched.

Not a paper argument — it runs, honestly labeled

The reason I can write this without hand-waving is that the composition engine is open source (constitutional-agent, pip install constitutional-agent), and it runs in my own system as the reference implementation of the Enterprise Agent Architecture. My gate-status endpoint exposes a composed-risk signal that did not exist in the codebase two weeks ago. When I captured it, it read:

I have to be honest about what that reading’s subject is, because it is not an agent’s decision trajectory — it is my system’s security environment. The same accumulation mechanism, applied to a different input: each security event in the last 72 hours converted to a 0–1 weight by severity, composed with a 24-hour decay half-life, capped at 1.5 per origin so no single source can trigger HOLD alone. Seven contributing events across three origins — and three origins is not three actors; they sat inside one /24, which could be one operator, NAT, or shared infrastructure. The mechanism is shared with the agent-trajectory case; the subject is not, and treating them as the same thing would be the sleight of hand I’m trying to avoid.

That reading proves the path is active: the composition mechanism is wired into production, reading live inputs, producing an interpretable value that crossed its own threshold. It does not prove the signal is calibrated or operationally useful — one value is not validation — and it is deliberately observe-only. Binding an uncalibrated threshold against live event volume is how you cause the incident you were trying to prevent. The observe-only telemetry is the calibration clock; the module (constitutional-agent v0.6.0) is new and not yet hardened to the degree of the core gates. The point is not that my 2.14 is a finished control. The point is that the layer the cookbook leaves out is not hypothetical — it exists, it runs, and you can read its code.