Compounding Returns and Flywheel Stall Conditions

Production agents earn compounding returns only when failures lead to precise, validated changes. This article examines the conditions that stall that improvement loop, especially the inability to determine which layer caused a failure. The central design requirement is not more execution or more telemetry, but a reliable path from production evidence to a targeted fix that the next cycle can test.

Why the compounding improvement loop matters for production agents

The improvement loop has a concrete sequence, namely execute, diagnose, fix, redeploy, and repeat. Each cycle brings more tasks, edge cases, and failure modes, which give the team evidence to improve the next version. Teams can build on earlier corrections with every deploy.

This mechanism operates on the harness, including prompts, tools, workflows, and memory, leaving model weights unchanged. It therefore does not depend on a team being able to retrain the underlying model. Its value comes from converting varied production experience into changes to the system around that model. The diagnostic step is the hinge, because execution alone cannot determine what should change.

What a stalled flywheel actually looks like in production

A failed run gives the team an outcome. Scale AI's RCA paper states that “while outcome-level failure signals indicate that an execution failed, they offer minimal insight into the underlying failure mechanisms and where to intervene.” A wrong answer or incomplete task establishes that the system failed its objective. It does not establish whether the prompt, a tool, the workflow, or the model was responsible. The familiar gap between a working demo and a broken production agent is therefore also a gap between detecting failure and explaining it.

Without a diagnostic layer, a team can change the prompt, redeploy, and hope the error rate falls. Activity continues, but the evidence does not narrow the choice of fix. An unattended improvement loop with a weak verifier can be worse because it continues producing bad outputs at token cost until someone notices. In that case, the system compounds cost rather than capability. These stalls are not random interruptions to an otherwise sound process, but the effects of specific, recurring weaknesses in the loop.

Why long-horizon traces make failure attribution hard enough to stall the loop

Long-horizon execution separates the evidence of a mistake from its eventual consequence. Scale AI's Continual Search paper describes execution logs that can reach millions of tokens, where the evidence that matters can appear long before the final outcome or across actions far apart in the log. Attribution then calls for a search through the log as well as a reading of it. A one-shot LLM judge can settle on a plausible explanation while leaving decisive evidence unexamined. An approach that works on a short trajectory therefore becomes unreliable as the history grows.

LongRCA Bench makes that distance concrete. Its reference root-cause steps precede completion by a median of 48 recorded steps, while the strongest evaluated baseline reaches only 13.2% exact root-step accuracy. Even purpose-built methods leave most reference roots unrecovered. A visible error near the end of the trace is not a reliable substitute for that origin.

TRAJDEBUG adds the problem of error co-occurrence. A long trajectory can contain mistakes that were repaired, mistakes that were harmless, and mistakes that caused the terminal failure. The responsible event is therefore neither necessarily the first error nor the last. Fixing the most conspicuous deviation can leave the originating event untouched, allowing the same failure to recur.

How telemetry format determines whether attribution is even possible

Search can recover evidence only if the telemetry preserved it. TelemetrySuffBench separates failure detection from fault-origin localization and shows that they require different information. Its views that meet telemetry and inference instrumentation standards are still good at failure detection, but their origin-step accuracy is at most 0.5%. Terminal status can reveal that a run failed without explaining where the failure began. More records in that format do not by themselves close the diagnostic gap.

TelemetrySuffBench also finds that when decision content is removed, origin-step accuracy falls to zero for every model tested. Removing provenance causes large, model-dependent losses. Attributable telemetry must therefore connect each action to its causal antecedents, including model invocations, tool calls, memory reads and writes, and decision branches. A sequence of discrete events is not equivalent to a record of those dependencies.

Even with structured telemetry, some inputs do not support a unique origin. TelemetrySuffBench finds that some models nevertheless answer confidently under that ambiguity, making abstention part of the diagnostic requirement. The system needs to distinguish missing evidence from evidence for a particular cause. Instrumenting for monitoring is not the same as instrumenting for diagnosis. Teams that conflate the two can have data without having answers.

The three harness failure categories that most commonly cause the loop to stall

Once the evidence supports attribution, the next task is assigning responsibility. Scale AI's RCA paper distinguishes faults in the model, harness, environment, and grader because each calls for a different intervention. Harness failures show how to change the harness design, model failures show what to work on in post-training, and environment or grader issues call for repairs to the benchmark. Within the harness, teams can look for prompt ambiguity, schema drift, and loop failures when they diagnose what went wrong.

Prompt ambiguity can leave an agent with every fact it needs but an incorrect reading of a constraint. TRAJDEBUG documents erroneous decisions that conflict with constraints in the original instruction. Identifying that conflict requires comparing the decision with context much earlier in the trajectory. Looking only at the final output misses the instruction that makes the decision wrong.

Schema drift occurs when a tool's input/output schema changes but the harness remains built against the earlier version. Calls can silently fail or return malformed data while the harness continues running. Because calls are made and responses arrive, a broken tool can appear to have produced a wrong answer. Loop failures then create a different compounding risk when repeated iterations generate bad traces that feed back into improvement without being flagged. These categories can all look like incorrect output from outside, but treating them all as prompt problems leaves their distinct causes unresolved.

Why misattributing a failure to the wrong layer costs more than finding it late

An incorrect diagnosis does more than postpone a repair. It can consume engineering time, introduce new failure modes, and corrupt the trace-based signal used in later iterations. Scale AI's RCA paper states that “At production scale, reliable attribution becomes critical, since misattributions incur substantial cost and can even trigger unnecessary interventions”. A harness fault misclassified as a model problem can redirect work toward post-training without changing the production behavior that caused the failure.

The same mismatch appears at a smaller scale when a prompt change is used to address schema drift. The instruction becomes clearer, but the tool call still fails, and the next cycle produces the same evidence. A confident false attribution is especially costly because it supplies a reason to make that ineffective intervention. Abstention leaves the uncertainty visible and prevents it from driving a change.

Scale AI's RCA paper describes the Hugging Face incident, in which investigators recovered and analyzed more than 70,000 agent messages and files to reconstruct the intrusion timeline. Repeated passes surfaced evidence that earlier readings had missed. That forensic work makes attribution all the more important as agents act more on their own in safety-critical contexts. Because a wrong diagnosis is costly, a diagnostic method must test its first explanation even if it can produce one quickly.

What better attribution methods do differently to keep the loop moving

Better attribution methods keep the diagnosis provisional while searching for evidence that could overturn it. Scale AI's Continual Search asks the judge to challenge its standing answer at successive turns and look beyond the evidence already supporting it. Within the same model family, lower-tier models using Continual Search can surpass higher-tier models using one-shot approaches.

LongRCA Bench's RCTA method separates finding failure-relevant activity from identifying where the decisive error originated. It retrieves original candidate evidence from long logs and compares it with upstream instructions. That comparison helps tell whether an error was inherited or first arose during execution.

TRAJDEBUG decomposes diagnosis into error trigger detection, error state classification, and candidate-set-guided causal attribution. It tracks resolution status and terminal impact separately so that mistakes that were repaired and downstream symptoms are not taken to be root causes by default. These approaches share a process of elimination that tests plausible matches. TelemetrySuffBench supplies the necessary limit to that process through its findings on ambiguity. When evidence cannot support a unique origin, the attribution system should abstain instead of producing a diagnosis that drives an unsupported intervention.

How trace-based attribution feeds back into harness improvements that the loop can validate

Attribution becomes useful when it produces a concrete, reviewable change to the responsible layer. That change must then face the historical traces that exposed the failure before it ships. Synthetic evaluations alone can give a good score even as the system behaves worse for real users. TRAJDEBUG's application studies show that actionable diagnostic feedback improves downstream agent success. The diagnosis is therefore an input to a testable repair.

A user-reported error, internal validation rejection, or post-hoc audit flags a case for a triage queue with an expected-behavior label. On the next deploy, that case runs as a regression test. Without that path, production evidence can be collected and diagnosed yet never constrain what ships.

Validation also has to extend beyond the single case that motivated the change. Prompts, tools, workflows, and memory are co-dependent, so repairing one component can shift failures to another. Replay against a broader historical set can catch failures that shift in this way. Shadow evaluation brings in live traffic by sending the same inputs through a separate candidate path and saving its outputs without showing them to users. This can reveal regressions that synthetic golden sets miss, including changes that improve aggregate scores while degrading performance on the actual distribution of requests. The next production cycle then provides evidence of whether the intervention worked.

Designing a flywheel that does not stall — the conditions it needs to sustain

A durable flywheel is built around evidence that can support a decision about what to change. The team preserves decision content and provenance in every span, so a failed outcome can be traced through the actions that produced it. Its attribution process searches beyond the first plausible answer and can reject an explanation when contradictory evidence appears. When the record leaves the cause unclear, abstention keeps that uncertainty from driving a change based on a false diagnosis.

That discipline continues into the intervention itself. The team distinguishes prompt, tool, workflow, memory, and model failures before choosing a fix. A schema drift failure receives a harness update. Every candidate change is replayed against the production cases that showed the problem and a broader historical set, while shadow evaluation tests how it behaves on live inputs. A successful local repair is not treated as proof that the whole harness improved.

Search cannot find evidence that was left out, precise attribution cannot save a fix made to the wrong layer, and a targeted fix needs validation to show its worth.