AutoRAG
4,722Open-source framework for retrieval-augmented generation (RAG) evaluation and optimization.
DetectionPost-generationFull HITLExperimentalTier 2BEnvironment
Maturity tier details
Computed tier: 2B(raw score 4.47). Effective tier: 2B.
Facets:documented_in=0evaluated_on=0integration_surfaces=1paper_citations=0recent_commit=2stars=1.47
Source of truth:pipeline/maturity/score.py
Description
AutoRAG is an open-source framework that searches for the best RAG pipeline for a given dataset — grid-searches over retriever × reranker × generator combinations and scores each against ground-truth evals. Complements RAG observability tools by focusing on offline pipeline selection.
Why it's included
RAG-specific evaluation cell: distinct from generic eval harnesses by treating the retriever+generator as the unit of optimization.
Addresses failure modes
- Fabrication — Generating reference to non-existent APIs, packages, or symbols.
- Context pollution — Trajectory drift, hallucination spirals, overlong or off-task context.
Techniques
rag-grid-searchretrieval-metricsanswer-metrics