Skip to main content
SAICA-KG

Tools

Guidance

21,402

Programmable interleaving of prompt, control flow, and constrained generation.

PreventionIn-generationFully autonomousStableTier 2BModelPrompt
Maturity tier details

Computed tier: 2B(raw score 4.23). Effective tier: 2B.

Facets:documented_in=0evaluated_on=0integration_surfaces=0.5paper_citations=0recent_commit=2stars=1.73

Source of truth:pipeline/maturity/score.py

Description

Python library for programming LLMs with interleaved text, control flow, regex/CFG constraints, and tool calls in a single program. Constraints (select, regex, grammar, JSON) are enforced at decoding time via logit masking against open-weights backends, and via token-healing to avoid tokenizer-boundary artifacts. Supports Transformers, llama.cpp, and remote providers where feasible.

Why it's included

Occupies prevention × in_generation × fully_autonomous alongside Outlines, but with a program-first ergonomics layer (interleaved prompting + constraints) rather than pure FSM compilation. Included to mark that the in-generation cell has multiple distinct approaches.

Security notes

Open-source (MIT), originally from Microsoft Research, now under the guidance-ai org. Requires logit access for full constraint enforcement, so closed-API backends degrade to weaker modes. Programs execute local Python; treat third-party Guidance programs as untrusted code.

Addresses failure modes

  • Fabrication — Generating reference to non-existent APIs, packages, or symbols.
  • Logic error — Correct API use, incorrect logic.

Techniques

constrained-decodinggrammar-guided-generationtoken-healinginterleaved-prompt-program

View raw YAML on GitHub →