Context Rot
The phenomenon where a model uses key information worse as the context grows longer.
In plain language
A long context is not equally effective. When key information sits in the middle of a long document or is surrounded by irrelevant text, recall and reasoning accuracy drop—a phenomenon called context rot. Causes include attention dilution, weak positional extrapolation and insufficient long-document training data. In practice, use retrieval, section summaries and put key information early instead of stuffing everything into the prompt.
Example
Asking a model to find a penalty clause in a 200-page contract may be less accurate than retrieving the relevant clause first.
Recommended tools
Related terms
Chunking
Splitting long documents into smaller passages for retrieval and embedding, a key step in RAG quality.
Context Window
The maximum number of tokens a model can read and remember at once, including input and output.
Retrieval-Augmented Generation
Retrieve relevant passages from a knowledge base, then have the model answer from them.
Agent Memory
A mechanism for an agent to store and retrieve user preferences, past tasks and key facts across sessions.