promptbench
2,801Unified evaluation framework for large language models with adversarial perturbations.
DetectionPost-generationFull HITLExperimentalTier 3PromptEnvironment
Maturity tier details
Computed tier: 3(raw score 3.88). Effective tier: 3.
Facets:documented_in=0evaluated_on=0integration_surfaces=0.5paper_citations=0recent_commit=2stars=1.38
Source of truth:pipeline/maturity/score.py
Description
Microsoft PromptBench is a framework to benchmark LLM robustness under adversarial prompt perturbations. Evaluates stability, robustness, and reliability with defined attack taxonomies and measures prompt sensitivity of outputs.
Why it's included
Robustness-under-perturbation cell of LLM evaluation — pairs well with garak/promptmap (security) and helm (holistic).
Addresses failure modes
- Fabrication — Generating reference to non-existent APIs, packages, or symbols.
- Security vulnerability — Insecure patterns, injection flaws, weak cryptography, secret leakage.
- Logic error — Correct API use, incorrect logic.
Techniques
adversarial-perturbationsprompt-sensitivity-metrics