Skip to main content
SAICA-KG

Tools

TruLens

3,266

Evaluation and Tracking for LLM Experiments and AI Agents

DetectionPost-generationGraduated HITLExperimentalTier 3Environment
Maturity tier details

Computed tier: 3(raw score 3.91). Effective tier: 3.

Facets:documented_in=0evaluated_on=0integration_surfaces=0.5paper_citations=0recent_commit=2stars=1.41

Source of truth:pipeline/maturity/score.py

Description

TruLens provides open-source tools for evaluating and tracking LLM experiments with programmable feedback functions. It enables developers to instrument, evaluate, and iterate on LLM applications using feedback functions for quality assessment, including coverage of context relevance, groundedness, and answer relevance.

Why it's included

Occupies detection × post_generation × graduated_hitl with the RAG Triad — context relevance, groundedness, and answer relevance — as programmable feedback functions over instrumented traces. Distinguished from general LLM-observability platforms (langfuse, langsmith, arize-phoenix) by centering RAG-specific groundedness metrics and from deepeval by exposing those metrics as live feedback functions rather than Pytest assertions. Useful as the canonical RAG-evaluation exemplar in the post-generation detection cell.

Addresses failure modes

  • Fabrication — Generating reference to non-existent APIs, packages, or symbols.
  • Context pollution — Trajectory drift, hallucination spirals, overlong or off-task context.

View raw YAML on GitHub →