Context rot

What it is

More context helps until it hurts. Past a point, extra material dilutes the instruction, buries the key fact or introduces contradictions, and answers get worse.

How it works

• Lost in the Middle (Liu et al., 2023) found that performance was often highest when the relevant information was at the beginning or end of the input, and dropped significantly when it was in the middle. That held even for long-context models.
• Chroma’s July 2025 technical report tested 18 models, including GPT-4.1, Claude 4, Gemini 2.5 and Qwen3. It found performance grew increasingly unreliable as input length increased, even on simple tasks, and that how information is presented matters as much as whether it’s there. Chroma sells a vector database, so weigh that interest.
• Practitioners. Christopher Penn describes the same pattern in long-running agents, where early instructions fade as the conversation grows. Dan Martell teaches both sides: give the model far more background than a one-line request, but don’t flood it with irrelevant data.

Benefits

• Better answers from the same model, with no upgrade needed.
• Lower cost. Less irrelevant input means fewer billed tokens.

Limits and risks

• Relevant and non-redundant helps. Irrelevant and contradictory hurts. Curate the context and put the key instruction and facts at the start or end.
• Long-running agents need resets. Reset or summarize periodically, and keep durable facts in an indexed store (see agent memory).
• Evidence date. The 2025 report covers newer long-context models, but neither study tested every current model, so test yours on your own material.