Security vulnerability
Insecure patterns, injection flaws, weak cryptography, secret leakage.
Description
The agent emits code that introduces a security vulnerability — injection primitives, weak cryptographic choices, leaked secrets, or insecure deserialization. Correct-looking code that violates security invariants.
Detection signals
- Static analyzer (semgrep, bandit, CodeQL) fires on the emitted code.
- Secret-scanner (gitleaks, trufflehog) finds hardcoded credentials.
- Generated SQL uses string concatenation instead of parameterization.
- Cryptographic primitives are weak (MD5, SHA1, ECB) or from insecure libraries.
Prior work
- Fang Liu (2024) — Beyond Functional Correctness: Exploring Hallucinations in LLM-Generated Code
- Joseph Spracklen (2024) — We Have a Package for You! A Comprehensive Analysis of Package Hallucinations by Code Generating LLMs
Crosswalks
Agent emits or runs code producing unintended remote execution.
OWASP ASI05 "Unexpected Code Execution" overlaps at the RCE surface; security_vulnerability covers a broader class.
Mitigating tools
- agent-governance-toolkit — AI Agent Governance Toolkit — policy enforcement, zero-trust identity, and execution oversight.
- agentic-security — Agentic LLM vulnerability scanner and AI red-teaming kit.
- AI-Infra-Guard — Full-stack AI red-teaming platform for securing AI ecosystems.
- AI-Red-Teaming-Playground-Labs — AI red-teaming labs: turnkey training environments for attacking AI systems.
- Bandit — Security linter for Python. Catches common code-level vulnerabilities.
- beelzebub — Secure low-code honeypot framework with LLM-backed system virtualization.
- bifrost — Fastest enterprise AI gateway with adaptive load balancing, failover, and guardrails.
- mcp — Browser MCP — an MCP server that lets AI applications control your browser.
- browser-use — Make websites accessible for AI agents.
- DeepTeam — Open-source LLM red-teaming framework from Confident AI with 40+ attack vulnerabilities.
- Daytona — Secure AI sandbox workspaces with sub-second start and stateful dev environments.
- Dependabot — Automated dependency-update PRs that flag outdated and vulnerable packages.
- E2B — Secure sandboxes for AI-generated code execution.
- ESLint — Pluggable JavaScript/TypeScript linter and code-quality enforcer.
- FuzzyAI — A powerful tool for automated LLM fuzzing from CyberArk.
- garak — The LLM vulnerability scanner.
- giskard-oss — Open-source evaluation and testing library for LLM agents and ML models.
- Guardrails AI — Output validation framework with a catalog of validators and reask-on-failure loops.
- helm — Holistic Evaluation of Language Models (HELM).
- litellm — Call any LLM using the OpenAI API format — proxy server and Python SDK.
- llm-guard — The Security Toolkit for LLM Interactions.
- mcp-playwright — Community Playwright MCP server for Claude Desktop, Cursor, Cline, and others.
- mcp-server-browserbase — MCP server to control a browser via Browserbase and Stagehand.
- genai-toolbox — MCP Toolbox for Databases — open-source MCP server for databases.
- Modal — Serverless sandboxed runtime with gVisor-isolated containers for agent code execution.
- NeMo Guardrails — Colang-based programmable guardrails for LLM applications.
- nono — Capability-based sandbox for securely running AI agents with zero setup
- playwright-mcp — Playwright Model Context Protocol server from Microsoft.
- pr-agent — AI-powered tool for automated pull-request analysis, feedback, and suggestions.
- pre-commit — Multi-language framework that runs configurable hooks before commits land — denies the commit on hook failure.
- promptbench — Unified evaluation framework for large language models with adversarial perturbations.
- promptfoo — Test prompts, agents, and RAGs — red-teaming, pentesting, and vulnerability scanning for LLMs.
- promptmap — Security scanner for custom LLM applications.
- PyRIT — The Python Risk Identification Tool for generative AI (PyRIT).
- Rebuff — LLM Prompt Injection Detector
- safe-rlhf — Safe RLHF: constrained value alignment via safe reinforcement learning from human feedback.
- Semgrep — Fast, rule-based static analysis with a pattern syntax that mirrors source code.
- skyvern — Automate browser-based workflows with LLMs and computer vision.
- Snyk — Vulnerability scanning for code, dependencies, containers, and IaC.
- Superagent — Security layer protecting AI apps from prompt injection, data leaks, and harmful outputs.