Skip to main content
SAICA-KG

Tools

uqlm

1,146

Uncertainty Quantification for Language Models.

DetectionPost-generationGraduated HITLExperimentalTier 3ModelEnvironment
Maturity tier details

Computed tier: 3(raw score 3.72). Effective tier: 3.

Facets:documented_in=0evaluated_on=0integration_surfaces=0.5paper_citations=0recent_commit=2stars=1.22

Source of truth:pipeline/maturity/score.py

Description

UQLM (by CVS Health) is a Python package for uncertainty quantification of LLM outputs. Implements black-box, white-box, and semantic-similarity-based uncertainty estimators so downstream code can abstain, escalate to human review, or request clarification when confidence is low.

Why it's included

Uncertainty-first branch of detection — enables explicit abstain/escalate routing, valuable for high-stakes medical/finance LLM deployments.

Addresses failure modes

  • Fabrication — Generating reference to non-existent APIs, packages, or symbols.
  • Logic error — Correct API use, incorrect logic.

Techniques

black-box-uqwhite-box-uqsemantic-uqabstain-routing

View raw YAML on GitHub →