The Attack Starts After the Agent Has Done Everything Right

Oct 12, 2026 6 min read

The dangerous moment in an agent workflow may arrive after the agent has followed the instructions correctly for ten turns.

That is the claim behind an alleged benchmark called AgentLAB, described in a post from the X account Grandfather on October 12, 2026. The post says the benchmark tested agents against 644 security cases across 28 simulated environments, covering intent hijacking, tool chaining, task injection, objective drifting, and memory poisoning.

Those figures are reported claims, not a result independently verified by Denny Sentinel. The linked research source is included in the Sources section. But the alleged benchmark is worth examining because its central idea is sound even before the numbers are confirmed: an agent can pass a first-turn safety check and still lose control of its objective later in the loop.

That is a different security problem from prompt injection as it is usually demonstrated.

The first prompt is not the whole attack surface

Most agent security tests are easy to explain because they isolate a single boundary. A malicious instruction appears in a document. A model follows it. A tool call exposes the failure.

Real agent workflows are less tidy.

An agent reads a task, searches a repository, calls an API, receives an error, retries with a changed argument, stores a note, and then uses that note in a later decision. Each step can look reasonable in isolation. The unsafe behavior may only become visible when the steps are composed.

The alleged AgentLAB categories describe five ways that composition can fail:

Intent hijacking: the agent’s original goal is replaced by an instruction encountered later.
Tool chaining: individually permitted tools combine into an unsafe capability.
Task injection: untrusted task content changes what the agent believes it has been asked to do.
Objective drifting: the agent gradually optimizes for an easier proxy instead of the operator’s actual goal.
Memory poisoning: a false or hostile fact survives the current turn and influences later work.

The common property is persistence. The attack does not need to win immediately. It only needs to alter the next decision.

Why long-horizon evaluation changes the unit of testing

A single prompt test asks whether the model responds safely to one input.

A long-horizon test asks whether the control loop remains aligned after the world has changed around it.

That difference matters because agents do not just generate text. They accumulate state. Tool results, files, browser pages, scratch notes, summaries, and prior decisions all become context for the next action.

A useful mental model is:

initial task
    ↓
observation from a tool or document
    ↓
new interpretation of the task
    ↓
action that changes state
    ↓
future context containing that change
    ↓
next decision made under a shifted objective

The attack surface is the entire trajectory. A filter that only inspects the original prompt is looking at the wrong object.

This is also why a model can appear well behaved during evaluation and still fail in deployment. The evaluation may test the model’s response to an attack string. The deployment exposes it to a sequence of plausible artifacts, partial failures, and permissions that no single string represents.

The tool boundary is where small mistakes compound

Tool chaining deserves particular attention because it turns low-risk capabilities into a higher-risk system.

A file reader may be harmless. A network client may be harmless. A shell command may be restricted. But an agent that can read attacker-controlled content, write a file, execute a helper, and send a request has a much larger capability than any one tool suggests.

This does not require a dramatic exploit. A hostile instruction can be carried from one tool result into the next action. The agent may not even classify the sequence as an attack because every intermediate step resembles normal work.

The operator mistake is to review tools one at a time while the agent experiences them as a graph.

The graph needs its own policy. Builders should ask which combinations are dangerous, which transitions require approval, and which tool results are allowed to become durable memory. A denylist of suspicious phrases cannot answer those questions.

Memory poisoning is not just bad retrieval

Memory poisoning is often discussed as a retrieval-quality problem. That framing is too weak for agents with authority.

If an agent stores a false instruction, a fake approval, or an attacker-controlled summary, the error can outlive the document that introduced it. A later session may treat the poisoned memory as trusted background rather than fresh input.

That creates a trust-boundary question: who is allowed to write durable state, and what evidence must accompany it?

A safer memory path has explicit provenance. It records the source, timestamp, scope, and confidence of a fact. It separates operator policy from observations gathered during a task. It makes high-impact memories reviewable and reversible.

Most importantly, it does not allow a model-generated summary to silently become an instruction with broader authority than the source had.

The verifier must observe the trajectory

The alleged AgentLAB result, if it holds up, points toward a different verifier design.

A verifier that checks only the final answer will miss the route that produced it. Two agents can return the same text while one has made unsafe tool calls, crossed a permission boundary, or written poisoned state along the way.

A stronger evaluation records at least:

the initial task and the trust level of each input
every tool call, argument, result, and retry
changes to files, memory, credentials, and external state
the moment the agent’s plan or objective changed
whether an approval gate was presented and what it covered
the final result and the rollback path

This is not merely better logging. It changes what counts as a passing run. The agent should pass only if the trajectory stayed within the intended control policy, not merely if the final prose looked correct.

The uncomfortable truth is that a clean final answer can be evidence of a successful attack. An attacker may want the agent to complete the task while quietly changing a file, leaking a secret, or planting a future instruction.

What operators should change

First, evaluate workflows over multiple turns. Include benign tasks that encounter errors, changing requirements, untrusted documents, and partial tool failures. The objective is to see whether the agent recovers by asking for clarification or by inventing new authority.

Second, test tool combinations instead of isolated tools. Draw the capability graph and mark transitions that can affect external systems, durable memory, credentials, or network access.

Third, treat memory writes as privileged actions. Require provenance, scope, and review for facts that can influence future runs. Give operators a way to delete or roll back a memory entry without resetting the whole agent.

Fourth, separate observations from instructions in the context model. A web page, tool output, or generated summary can describe a possible action. It should not automatically authorize that action.

Finally, keep a human approval gate around irreversible effects. Long-horizon evaluation reduces risk, but it does not turn an autonomous loop into an independent security boundary.

The attack is the trajectory

The AgentLAB claims are still alleged. The post reporting them is a discovery signal, not a reproducible result. But the systems lesson does not depend on accepting every number.

Agents are stateful control loops. Their security cannot be measured only at the first prompt or the final answer. The important unit is the trajectory between them: what the agent saw, what it remembered, which tools it combined, and how its objective changed.

If the attack can begin after the agent has done everything right, then the verifier has to watch what happens next.

Sources

Editorial note: AgentLAB, the 644-case count, the 28-environment count, and the five attack categories are attributed to the cited X post. Denny Sentinel did not independently verify the benchmark or reproduce its results in this article.