Skip to main content
SAICA-KG

Papers

Security Considerations for Artificial Intelligence Agents

Ninghui Li, Kaiyuan Zhang, Kyle Polley, Jerry Ma · 2026 · arXiv

Source →arXiv:2603.12230
failure-mode-taxonomyagentic-aibenchmarkguardrails

TL;DR

This article details observations and recommendations concerning the security of frontier AI agents, and identifies standards and research gaps, including adaptive security benchmarks, policy models for delegation and privilege control, and guidance for secure multi-agent system design aligned with NIST risk management principles.

Abstract

This article, a lightly adapted version of Perplexity's response to NIST/CAISI Request for Information 2025-0035, details our observations and recommendations concerning the security of frontier AI agents. These insights are informed by Perplexity's experience operating general-purpose agentic systems used by millions of users and thousands of enterprises in both controlled and open-world environments. Agent architectures change core assumptions around code-data separation, authority boundaries, and execution predictability, creating new confidentiality, integrity, and availability failure modes. We map principal attack surfaces across tools, connectors, hosting boundaries, and multi-agent coordination, with particular emphasis on indirect prompt injection, confused-deputy behavior, and cascading failures in long-running workflows. We then assess current defenses as a layered stack: input-level and model-level mitigations, sandboxed execution, and deterministic policy enforcement for high-consequence actions. Finally, we identify standards and research gaps, including adaptive security benchmarks, policy models for delegation and privilege control, and guidance for secure multi-agent system design aligned with NIST risk management principles.

Editorial notes

Lightly-adapted version of Perplexity's response to NIST/CAISI RFI 2025-0035 on frontier AI agent security. Maps principal attack surfaces across tools, connectors, hosting boundaries, and multi-agent coordination, with explicit emphasis on indirect prompt injection, confused-deputy behavior, and cascading failures in long-running workflows. Assesses defenses as a layered stack: input/model-level mitigations, sandboxed execution, and deterministic policy enforcement for high-consequence actions. Relates to SAICA-KG failure modes: security_vulnerability, cascading_failure, supply_chain_attack. Useful as an industry-perspective anchor for the layered-defense framing in SAICA-KG's ControlParadigm axis.

View raw YAML on GitHub →