Skip to main content
SAICA-KG

Tools

Outlines

13,718

Constrained decoding via finite-state machines over regex, JSON Schema, and CFGs.

PreventionIn-generationFully autonomousStableTier 2BModel
Maturity tier details

Computed tier: 2B(raw score 4.15). Effective tier: 2B.

Facets:documented_in=0evaluated_on=0integration_surfaces=0.5paper_citations=0recent_commit=2stars=1.65

Source of truth:pipeline/maturity/score.py

Description

Python library for structured generation that compiles regex, JSON Schema, or a context-free grammar into a finite-state machine and masks the LLM's logits at every decoding step so only tokens leading to valid output can be sampled. Output is guaranteed to match the schema by construction — no retry loop required. Backends include Transformers, vLLM, llama.cpp, MLX, and a hosted service (.txt).

Why it's included

Canonical prevention × in_generation × fully_autonomous exemplar: logit-masking during decoding prevents schema-invalid tokens from ever being emitted. Distinct from post-generation validators — output is correct-by-construction rather than checked-after-the-fact.

Security notes

Open-source (Apache-2.0). Requires access to model logits, so it works with open-weights backends (Transformers/vLLM/llama.cpp) but not with closed APIs that don't expose logit bias. FSM compilation happens locally; no data leaves the process. Hosted .txt service is a separate SaaS with its own data-handling posture.

Addresses failure modes

  • Fabrication — Generating reference to non-existent APIs, packages, or symbols.
  • Logic error — Correct API use, incorrect logic.

Techniques

constrained-decodinglogit-maskingfsm-guided-generationjson-schema-decodingcfg-guided-generation

View raw YAML on GitHub →