Skip to main content
SAICA-KG

Incidents

Microsoft AIRT memory-poisoning case study on agentic AI memory corruption

Microsoft's AI Red Team taxonomy whitepaper walks through a worked example in which an attacker corrupts an agent's persistent memory, then uses that corrupted memory as a pivot to exfiltrate data...

2025-04-24security breachplausible

Description

Microsoft's AI Red Team (AIRT) 2025 "Taxonomy of Failure Mode in Agentic AI Systems" whitepaper includes an extended worked example of memory poisoning. In the scenario, an attacker injects malicious content into an agent's persistent memory store via an ordinary interaction. The poisoned memory is later recalled in an unrelated session, where it steers the agent into exfiltrating sensitive data to an attacker-controlled endpoint. AIRT uses the example to illustrate how their taxonomy classifies the failure (security + agentic + persistence) and to argue that memory requires the same semantic-validation discipline applied to untrusted input. Although the specific incident is hypothetical, it is authored by a team that red-teams 100+ production GenAI systems, and it maps directly to the msft-airt-2025 taxonomy node that SAICA-KG already ingests.

Exhibited failure modes

Affected systems

  • generic agentic AI with persistent memory

Mitigated by

  • Rebuff — LLM Prompt Injection Detector
  • Guardrails AI — Output validation framework with a catalog of validators and reask-on-failure loops.

Sources

Notes

`reproducibility: plausible` (not confirmed) — AIRT's example is a worked scenario, not a reported breach. Included to anchor the msft-airt-2025 taxonomy node in an incident-shaped entry.

View raw YAML on GitHub →