Arize Phoenix
9,394Open-source LLM tracing and evaluation platform built on OpenTelemetry.
Maturity tier details
Computed tier: 2B(raw score 4.59). Effective tier: 2B.
Facets:documented_in=0evaluated_on=0integration_surfaces=1paper_citations=0recent_commit=2stars=1.59
Source of truth:pipeline/maturity/score.py
Description
Open-source LLM and agent observability platform from Arize. Ingests OpenInference/OpenTelemetry traces from any framework, provides a local notebook-friendly UI for span-level inspection, and ships built-in LLM-as-judge evaluators (hallucination, relevance, QA correctness, toxicity, agent-tool-call correctness). Integrates with LangChain, LlamaIndex, CrewAI, AutoGen, and the OpenAI/Anthropic SDKs. Self-hostable container plus an Arize AX managed offering sharing the same data model.
Why it's included
Occupies detection × post_generation × fully_autonomous alongside Langfuse and LangSmith but with a tighter OpenTelemetry/ OpenInference standardization posture and a notebook-friendly local mode. Worth modeling as a distinct node because it is the de-facto reference implementation of OpenInference semantic conventions for LLM traces.
Security notes
Core project is source-available under Elastic-2.0 and self-hostable via Docker, which makes it a data-residency-friendly observability option. Prompts and completions are the primary payload — redact secrets and PII via the SDK hooks before ingest. OpenInference instrumentation is framework-agnostic; audit auto-instrumentation patches for any that wrap sensitive call sites.
Addresses failure modes
- Logic error — Correct API use, incorrect logic.
- Context pollution — Trajectory drift, hallucination spirals, overlong or off-task context.
- Fabrication — Generating reference to non-existent APIs, packages, or symbols.