Skip to main content

Context pollution in AI coding agents

Context pollution is when an AI coding agent’s working context degrades over a long session: relevant earlier information is dropped, irrelevant material piles up, or mistakes from earlier turns are treated as fact in later ones. It is a failure of state hygiene across a session, not of a single generation. OWASP’s “Memory & Context Poisoning” covers the adversarial version; context pollution also covers accidental drift. What is often called context rot, a model becoming less reliable as its context grows, is one route into it: one of the detection signals below is a session longer than the model can reliably attend over. Failed attempts that stay in context and bias later ones link it to cascading failure.

Later steps build on whatever is in context, so one stale or wrong fact can steer the rest of the session. The catalog’s incidents include AI agents on GitHub that ran commands hidden in pull-request titles, issues and comments and a support bot that invented a policy that did not exist.

At a glance: Not benchmarked yet (no dataset with confirmed labels exists); the tools below are listed by GitHub stars, not measured. Results ↓

Also called
  • context contamination
  • context leakage
  • irrelevant context
  • cross-task contamination
  • conversation history bleed
  • prompt pollution
  • stale context

How well can context pollution be caught?

Not measured We don't know yet. SAICA-Bench has not benchmarked context pollution: no dataset with confirmed labels exists for it, so no detector has a measured score. No dedicated detector for coding agents yet. The three that detect it are LLM-judge knowledge-retention scores built for chat assistants; several others only work on agents built with them.

Catalog tools that address it, by GitHub stars (not measured)

Stars show popularity, not quality. “Detects” tools flag the failure, “prevents” tools make it less likely, and “records” tools only keep traces for a human to inspect.

Catalog tools that address context pollution, ranked by GitHub stars; none has been measured
Tool Stars What it does How (not measured)
AutoGen 57,354 Records · Hybrid Built-in OpenTelemetry tracing lets a human inspect drift; its buffered/head-tail contexts are token budgeting, not an anti-degradation mechanism.
agno 39,620 Records · Hybrid Traces runs and tool calls so a human can diagnose drift; no built-in context-pollution check.
LangGraph 30,108 Prevents · Free program Memory docs name stale or off-topic context as a failure and ship trim/remove/summarize message utilities that the developer must apply; no detection.
mlflow 25,505 Detects · Hybrid Built-in multi-turn KnowledgeRetention LLM judge (experimental) checks whether the assistant retains information from earlier user inputs.
mastra 23,247 Prevents · Hybrid Observational Memory compresses old messages into observations so the agent reads what matters instead of stale tool output over long sessions.
opik 18,988 Detects · Hybrid Conversation-thread metrics (Knowledge Retention, Degeneration) check that later replies keep earlier user facts and flag repetition/degeneration.
DeepEval 14,938 Detects · AI-based Multi-turn LLM-judge metrics (Knowledge Retention, Turn Relevancy) score whether a conversation keeps earlier facts and stays on task.
Arize Phoenix 9,394 Records · Hybrid OpenTelemetry tracing records the session for human review; no built-in context-degradation metric.
TruLens 3,266 Records · AI-based Trace-level Logical Consistency judge flags hallucinations and unsupported assertions, not stale or irrelevant context, so the audit keeps it as records only.

Leaderboard → How SAICA-Bench works → Build a supervision kit →

How to detect it

Signals that point to context pollution, from the SAICA knowledge graph:

  • Later turns reference facts or code that never existed in user input or tool output.
  • Token-budget pressure truncates prior turns containing still-relevant context.
  • Agent repeats a failed tool call multiple times with identical arguments.
  • Session duration exceeds a threshold at which attention-over-context is empirically unreliable for the model.

Real-world incidents

Studies and reported patterns

Records in the catalog that are not single dated incidents: findings of a study, recurring behaviour that was reported, or scenarios a source describes as hypothetical.

Tools in the catalog that address it

5 tools are tagged with context pollution in the catalog, most-starred first. A tag is the catalog's claim about what a tool is for; it is not a test result. Where SAICA-Bench ran the tool on this mode, the measured J is shown: that row is the evidence.

  • LangGraph Graph-structured state machines for stateful multi-agent orchestration.
  • mastra The TypeScript AI agent framework — assistants, RAG, observability; supports any LLM.
  • mlflow Open-source platform for the machine learning lifecycle — experiments, tracking, models, and LLM evaluation.
  • opik Debug, evaluate, and monitor LLM applications, RAG systems, and agentic workflows.
  • DeepEval The open-source LLM evaluation framework for testing and evaluating LLM applications across 20+ custom and researched metrics.

In other taxonomies

How context pollution maps to other failure taxonomies. “Exact” means the same failure, “partial” an overlap, “related” a neighbouring category.

  • MAST: Multi-Agent System Failure Taxonomy

    Mapping drafted by an AI model from the source text; not yet reviewed by a person.

    • partial FM-1.4 Loss of conversation history — Context truncation that drops recent history is the "relevant prior information is dropped" part of context_pollution. context_pollution is wider (also accumulation of irrelevant material and earlier hallucinations treated as ground truth).

    Related: FM-1.3 Step repetition; FM-2.1 Conversation reset; FM-2.3 Task derailment

  • OWASP Top 10 for Agentic Applications 2026

    Mapping drafted by an AI model from the source text; not yet reviewed by a person.

    • partial ASI06 Memory & Context Poisoning — OWASP covers adversarial manipulation; context_pollution also covers accidental drift. Partial on the adversarial subset.

    Related: ASI01 Agent Goal Hijack; ASI08 Cascading Failures

  • Taxonomy of Failure Mode in Agentic AI Systems (Microsoft AI Red Team, version 1.0)

    Mapping drafted by an AI model from the source text; not yet reviewed by a person.

    • partial MS-ES-01 Memory poisoning and theft — Malicious content in the agent's memory that it "will process each time it is recalled" is context gone bad through an attacker and across sessions. context_pollution is wider on one side (accidental drift, earlier hallucinations taken as fact) and narrower on the other (it is defined within a session).

    Related: MS-ES-02 Targeted knowledge base poisoning; MS-ES-03 Cross domain prompt injection (XPIA)

  • 9 Critical Failure Patterns of Coding Agents

    Mapping drafted by an AI model from the source text; not yet reviewed by a person.

    Related: DAP-08 Codebase Awareness & Refactoring Issues

  • Characterizing Faults in Agentic AI: A Taxonomy of Types, Symptoms, and Root Causes

    Mapping drafted by an AI model from the source text; not yet reviewed by a person.

    Related: SHAH-CAT-01 LLM Interaction; SHAH-CAT-02 Agent Lifecycle and State; SHAH-CAT-03 Memory Management; SHAH-RC-03 State Management Complexity

  • Swiss Cheese Model for AI Safety: Multi-Layered Runtime Guardrail Taxonomy

    Mapping drafted by an AI model from the source text; not yet reviewed by a person.

    Related: SC-QA-02 Efficiency

  • Cascading failures — Each fix introduces new errors and the code drifts further from working.

All 11 AI coding agent failure modes → · Glossary

Research

Questions and answers

What is context pollution in AI coding agents?

Context pollution is when an AI coding agent’s working context degrades over a long session: relevant earlier information is dropped, irrelevant material piles up, or mistakes from earlier turns are treated as fact in later ones. It is a failure of state hygiene across a session, not of a single generation. OWASP’s “Memory & Context Poisoning” covers the adversarial version; context pollution also covers accidental drift. What is often called context rot, a model becoming less reliable as its context grows, is one route into it: one of the detection signals below is a session longer than the model can reliably attend over. Failed attempts that stay in context and bias later ones link it to cascading failure.

How do you detect context pollution?

Signals that point to it: Later turns reference facts or code that never existed in user input or tool output. Token-budget pressure truncates prior turns containing still-relevant context. Agent repeats a failed tool call multiple times with identical arguments. Session duration exceeds a threshold at which attention-over-context is empirically unreliable for the model. No detector has been measured on it yet.

Which tools catch context pollution?

None has been measured. By GitHub stars, the catalog tools that address it are AutoGen (records), agno (records), LangGraph (prevents) and mlflow (detects), and 5 more. No dedicated detector for coding agents yet. The three that detect it are LLM-judge knowledge-retention scores built for chat assistants; several others only work on agents built with them.

Why is context pollution not benchmarked?

There is no dataset with confirmed labels for it yet, so SAICA-Bench cannot score detectors on it. The leaderboard lists the catalog tools that address it by GitHub stars instead, which says nothing about how well they work.

Benchmark results from 2026-11-22. The definition, detection signals and tags come from the SAICA knowledge graph.

View raw YAML on GitHub →