Skip to main content
SAICA-KG

Tools

AutoRAG

4,722

Open-source framework for retrieval-augmented generation (RAG) evaluation and optimization.

DetectionPost-generationFull HITLExperimentalTier 2BEnvironment
Maturity tier details

Computed tier: 2B(raw score 4.47). Effective tier: 2B.

Facets:documented_in=0evaluated_on=0integration_surfaces=1paper_citations=0recent_commit=2stars=1.47

Source of truth:pipeline/maturity/score.py

Description

AutoRAG is an open-source framework that searches for the best RAG pipeline for a given dataset — grid-searches over retriever × reranker × generator combinations and scores each against ground-truth evals. Complements RAG observability tools by focusing on offline pipeline selection.

Why it's included

RAG-specific evaluation cell: distinct from generic eval harnesses by treating the retriever+generator as the unit of optimization.

Addresses failure modes

  • Fabrication — Generating reference to non-existent APIs, packages, or symbols.
  • Context pollution — Trajectory drift, hallucination spirals, overlong or off-task context.

Techniques

rag-grid-searchretrieval-metricsanswer-metrics

View raw YAML on GitHub →