Microsoft AIRT memory-poisoning case study on agentic AI memory corruption
Microsoft's AI Red Team taxonomy whitepaper walks through a worked example in which an attacker corrupts an agent's persistent memory, then uses that corrupted memory as a pivot to exfiltrate data...
Description
Microsoft's AI Red Team (AIRT) 2025 "Taxonomy of Failure Mode in Agentic AI Systems" whitepaper includes an extended worked example of memory poisoning. In the scenario, an attacker injects malicious content into an agent's persistent memory store via an ordinary interaction. The poisoned memory is later recalled in an unrelated session, where it steers the agent into exfiltrating sensitive data to an attacker-controlled endpoint. AIRT uses the example to illustrate how their taxonomy classifies the failure (security + agentic + persistence) and to argue that memory requires the same semantic-validation discipline applied to untrusted input. Although the specific incident is hypothetical, it is authored by a team that red-teams 100+ production GenAI systems, and it maps directly to the msft-airt-2025 taxonomy node that SAICA-KG already ingests.
Exhibited failure modes
- Security vulnerability — Insecure patterns, injection flaws, weak cryptography, secret leakage.
- Context pollution — Trajectory drift, hallucination spirals, overlong or off-task context.
Affected systems
- generic agentic AI with persistent memory
Mitigated by
- Rebuff — LLM Prompt Injection Detector
- Guardrails AI — Output validation framework with a catalog of validators and reask-on-failure loops.
Sources
Notes
`reproducibility: plausible` (not confirmed) — AIRT's example is a worked scenario, not a reported breach. Included to anchor the msft-airt-2025 taxonomy node in an incident-shaped entry.