Signal Quality Degradation in Long-Running Feedback Loops

A software bug stops a program. An agent bug lets the program keep going, on the wrong information, while everything about its behavior looks normal. That distinction is the entire subject of this piece. The Thinking Company's production evaluation guide states that agent bugs finish the conversation with wrong state, and that alone is enough to make operators trust a system longer than they should. A crashed process demands attention because it announces itself. A long-running agent loop that has quietly drifted onto the wrong task, the wrong user record, or the wrong set of constraints announces nothing. It produces a fluent answer. The loop moves to the next turn as if nothing happened, because as far as its own internal accounting goes, nothing did.

How a Production Agent Loop Is Structured

To see why that gap exists, it helps to look at what actually gets assembled every time an agent takes a turn. A single step in a production agent loop is not one call to one model. It draws on conversation history, retrieved documents, persistent memory, the outputs of earlier tool calls, workflow state carried over from prior steps, system instructions, and in many deployments the outputs of other agents working the same task. All of that gets combined into the context the model reasons over, and the response that comes back gets parsed, routed, stored, or handed to a tool before the next turn even starts. The behavior of the loop, in other words, is a product of how these stages interact with each other, not a property of the model sitting at the center of it.

That matters because each of those stages can drop information, compress it past the point of usefulness, replace it with something stale, or route it to the wrong place, all without raising anything that looks like an error. The failure surface sits in the handoffs between stages, not inside any single component you could point to and blame. And because each turn's output becomes part of the next turn's input, whatever gets lost or corrupted at one stage doesn't stay contained. It becomes part of the material the next turn treats as fact.

Most harness designs don't even track a deeper layer: the model itself. Anthropic, OpenAI, Google, and the other major providers continuously update the models sitting behind their API endpoints, and none of them push a changelog notification to every caller when that happens, even though all three do publish official release notes. The alias a team pins in production is not a frozen artifact. It can shift under a team's feet between one turn and the next, and the harness calling it has no built-in way of knowing. That's distinct from a deliberate model upgrade a team schedules and tests. A silent weight change, a tuning pass, or a safety-filter adjustment can alter output formatting or refusal behavior in ways that quietly break whatever downstream parsing was built to expect the old shape of response.

The three structural mechanisms by which signal degrades across turns

Given that architecture, signal degrades across turns through three specific mechanisms, and each one is invisible to standard error monitoring because none of them produce an error.

The first is context compression and constraint loss. As a conversation grows, most production systems compact or summarize the history to keep it inside a token budget, and that process routinely drops or dilutes constraints that were set several turns earlier. The model keeps generating as though those constraints still apply, because nothing in its prompt tells it otherwise. Its actual behavior has already diverged from them. Retrieval and memory systems compound this: content that is stale or irrelevant to the current step can surface, and that content displaces the accurate material that should have been in context, with no flag raised anywhere. The most recent input dominates the model's attention far more than material from several turns back, a basic property of how autoregressive generation works, so the original goal a task started with loses influence with every turn that passes, even when nobody has explicitly changed it.

The second mechanism is tool schema drift. Tool calling is a probabilistic output format the model produces, and most tool-calling failures trace back to schema drift and missing validation rather than to the model being incapable of the task. When a field like user_id gets renamed to member_id somewhere in a system's tooling, the model often keeps generating the old key, because the traces, few-shot examples, and stored memory it's conditioning on still encode the old schema. The call can appear to succeed at one layer and fail several steps later, quietly. The same thing happens when a parameter moves from an optional, server-side default to a required field: the model omits it out of habit, and that omission propagates through the system before anything catches it. Old examples sitting in retrieval or memory make this worse over time, not better, because they keep teaching the model to reproduce the schema that no longer exists.

The third is prompt ambiguity that compounds at each decision point. Agents can't pause mid-execution to ask a clarifying question the way a human collaborator would. Every ambiguity in a task becomes a fork where the agent has to pick an interpretation and commit to it, and that committed interpretation becomes part of the context shaping every subsequent turn. A weak early choice narrows the space of paths the agent can recover along, and there is often no bounded stopping condition, no step limit, token budget, or wall-clock timeout, so a single ambiguous task can consume unbounded compute while a downstream system waits indefinitely for a response that never comes. Standard retry logic makes this worse rather than better: if the model is generating malformed JSON because it misread the task, retrying the identical prompt reliably produces the identical malformed output.

Why the Degradation Is Self-Concealing

What makes all three of these mechanisms dangerous is the architecture built to make agent loops powerful. It's that the same architecture built to make agent loops powerful, accumulated context, persistent memory, multi-step reasoning, is what lets the corruption hide and grow across turns. Every turn conditions on the output of the one before it. A corrupted state doesn't get corrected by the next turn's reasoning. It gets treated as settled fact and built on top of, the same way a foundation error in a building gets buried under the next floors rather than fixed.

Nothing about the model's confidence changes as this happens. Autoregressive generation produces fluent, well-formed, internally consistent output regardless of whether the state underneath it is accurate, so there is no drop in tone, no hedging, and no signal in the text itself that anything has gone wrong. And when something has gone wrong badly enough that a team turns to the model itself to diagnose it, the model runs into a documented limitation of its own reasoning: research on LLM-based failure attribution names a specific pattern called premature commitment, where a model settles on a plausible-sounding explanation for a failure before it has actually explored the available evidence. That means the model cannot reliably diagnose its own mid-loop corruption even when directly asked to.

Multi-agent systems make the concealment worse still. When one agent hands its output to another, the receiving agent has no mechanism for verifying that the state it just inherited was accurate. A corrupted handoff looks exactly like a valid one from the receiving agent's point of view, so the corruption doesn't just persist, it spreads to a second component that had no way of catching it. And the instrument teams often rely on to catch quality problems is itself not immune: LLM-as-judge evaluators drift and miscalibrate over time, static evaluation sets get gamed as prompts get optimized against them, and the measurement layer designed to catch degradation can stop catching it without anyone noticing that it has stopped.

What standard observability stacks catch and what they structurally cannot

Standard error monitoring and the dashboards most teams already have catch none of this. Conventional observability, latency, token counts, error rates, uptime, tells a team whether a request succeeded and how long it took, and infrastructure can look perfectly healthy while the actual quality of what the agent produces collapses. Confident AI's 2026 comparison of LLM observability tools makes this point directly: a response can be fast, on-brand, and still wrong, hallucinated, unsafe, or faithful to context that was never the right context to begin with. Rate limit errors are the single leading cause of LLM call failures in production, and catching them tells a team nothing about whether the calls that did succeed produced output that was accurate, policy-compliant, or coherent with everything that came before it.

A CHI 2025 study on observability design for LLM systems, cited in Confident AI's analysis, names four pillars developers actually need: awareness, monitoring, intervention, and operability. All four assume a system is scoring output for quality, beyond logging that a call happened. A pipeline that records prompts, tokens, latency, and cost without ever scoring what came out the other end is an infrastructure dashboard borrowing the vocabulary of an LLM product. It shows what ran. It does not show whether what ran was any good. Catching that requires tracking drift at the level of individual prompts and use cases, categorizing behavior, and alerting on quality scores directly, alongside the standard infrastructure signals like 500s and latency spikes. Worsening answers, unsafe outputs, retrieval that has gone stale, and tools being called on the wrong arguments all belong to a category of failure that is structurally invisible to infrastructure monitoring. Surfacing them requires evaluating the actual production traces, not watching the pipes those traces flow through.

Root cause attribution assigned to a specific harness layer, not just flagged as "a failure

Knowing that a run went wrong somewhere is close to useless on its own. What matters is which layer went wrong, the prompt, the tool schema, the memory system, the workflow logic, the model itself, or the product logic wrapping all of it, because each of those requires an entirely different fix. A prompt rewrite does nothing for a tool schema that has silently changed underneath it. A tool schema patch does nothing for a memory compaction step that dropped a constraint several turns earlier. Applying the wrong fix doesn't just waste engineering time. It leaves the actual failure mode fully active while the team believes they've resolved it.

Agent failures are also harder to localize than traditional software bugs by nature: they're non-deterministic, and they live inside long execution trajectories full of natural-language reasoning rather than a clean call stack. There's no line number to point to. Attribution has to be actively reconstructed from the interaction between components. Research by Raj et al., described in the State of Agent Engineering 2026 report, formalizes this problem by framing failure attribution around the interaction between two components and which side of that interaction carries the fault. In multi-agent settings the same benchmark work has produced dedicated tools for the job: Who&When, from Zhang et al. in 2025, identifies which agent was responsible and at which step the decisive error occurred, and Who&When Pro, from Liu et al. in 2026, substantially expands that evaluation across more agent frameworks, domains, and modalities.

The compounding version of this problem appears constantly in multi-agent systems: a downstream agent gets blamed for a failure that actually originated in a corrupted handoff from an upstream agent, and the downstream agent bears no fault. Confident AI's comparison of agent observability platforms makes the underlying architectural point directly: a single small prompt change can break an entire flow, and the only way to catch it is to see which configuration changed, what behavior changed alongside it, and which specific component in the chain caused the regression.

Trace-level attribution across the full loop as the diagnostic method that closes the gap

Closing that gap requires visibility into the full trace of a run, covering a sampled slice of it and just the final prompt and response. Useful agent observability means capturing every tool call, every retrieval, every sub-agent handoff, every LLM call and retry, along with the inputs, outputs, latency, cost, and version metadata attached to each one. Version metadata matters because model providers like Anthropic, OpenAI, and Google continuously update the models behind their API endpoints without proactively notifying API callers, so even a pinned model alias can shift between turns without the harness knowing.

Confident AI's 2026 comparison of agent observability platforms identifies the capability that separates a tool that merely displays traces from one that can actually diagnose them: agent-step evaluation, meaning fine-grained scoring of tool selection, tool arguments, planning quality, retrieval quality, step-level faithfulness, and reasoning coherence at each individual step, not just at the end. The diagnostic signal that matters lives in how output quality changes across turns rather than in any single-turn score. A metric that checks whether a full trace stayed on task, maintained its context across every turn, and followed policy throughout is answering a fundamentally different question than a metric that scores one response in isolation. In multi-agent systems this extends further still: visibility into which agent handled which step, where coordination between agents broke down, and how a downstream agent's decisions changed as a result, is something single-agent tracing simply cannot provide.

MLflow's 2026 guide to production-ready agents makes a related argument for where this evaluation should actually live: probes need to be embedded inside the agentic workflow itself for real-time auditability, rather than run as an offline batch process after the fact, because a probe only catches a problem at the moment it happens if it sits right next to the decision it's evaluating. And distinguishing a genuine harness regression from a silent model update requires tracking exactly which prompt version, which model, which hyperparameters, which tool schema, which retrieval index, and which agent version produced a given run. Complete trace visibility is the baseline requirement. What a team does with that visibility, how it turns a diagnosed failure into a verified fix, is the harder and more consequential half of the problem.

Replay-based validation as the mechanism for verifying that a harness fix reverses the degradation

A harness fix that hasn't been replayed against the production traces that exposed the original failure is a fix nobody has actually tested. The only real evidence that a repair works is that it produces the correct output on the exact inputs that broke the system the first time. Shipping a change and hoping it holds leaves the failure undiagnosed.

The loop that closes this gap runs in a consistent order: detect a failure in production, convert the representative traces that captured it into concrete test cases, build or tune an evaluator against those cases, replay the test set offline to confirm the fix actually resolves what it was meant to resolve, then deploy that evaluator back into live monitoring so the same failure mode gets caught automatically if it recurs. DeepEval 4.0's local evaluation harness shows this loop in a concrete setting: a coding agent can generate its own dataset from prior failures, run the full evaluation suite against it, and surface the failed cases for direct inspection. That is what turns a one-time diagnosis into a permanent regression check, and it's the only way a team can say with confidence that a harness change didn't just patch the symptom it happened to notice, but actually reversed the mechanism that produced it.