Skip to main content
SAICA-KG

Tools

garak

7,614

The LLM vulnerability scanner.

DetectionPost-generationFull HITLStableTier 2BEnvironment
Maturity tier details

Computed tier: 2B(raw score 4.55). Effective tier: 2B.

Facets:documented_in=0evaluated_on=0integration_surfaces=1paper_citations=0recent_commit=2stars=1.55

Source of truth:pipeline/maturity/score.py

Description

NVIDIA garak is an open-source LLM vulnerability scanner: applies a library of probes (jailbreaks, prompt injection, data leakage, malware generation, misinformation) against any LLM endpoint and reports per-probe attack success rates. Models the NVIDIA NeMo Guardrails adversarial test plane.

Why it's included

Canonical LLM-vuln-scanner: complements llm-guard/rebuff by running at assessment time rather than at request time.

Addresses failure modes

  • Security vulnerability — Insecure patterns, injection flaws, weak cryptography, secret leakage.
  • Fabrication — Generating reference to non-existent APIs, packages, or symbols.
  • Scope creep — Actions outside declared task boundary.

Techniques

llm-probesattack-success-rate-metricsdetect-generators

View raw YAML on GitHub →