Skip to main content
SAICA-KG

Tools

Arize Phoenix

9,394

Open-source LLM tracing and evaluation platform built on OpenTelemetry.

DetectionPost-generationFully autonomousStableTier 2BEnvironment
Maturity tier details

Computed tier: 2B(raw score 4.59). Effective tier: 2B.

Facets:documented_in=0evaluated_on=0integration_surfaces=1paper_citations=0recent_commit=2stars=1.59

Source of truth:pipeline/maturity/score.py

Description

Open-source LLM and agent observability platform from Arize. Ingests OpenInference/OpenTelemetry traces from any framework, provides a local notebook-friendly UI for span-level inspection, and ships built-in LLM-as-judge evaluators (hallucination, relevance, QA correctness, toxicity, agent-tool-call correctness). Integrates with LangChain, LlamaIndex, CrewAI, AutoGen, and the OpenAI/Anthropic SDKs. Self-hostable container plus an Arize AX managed offering sharing the same data model.

Why it's included

Occupies detection × post_generation × fully_autonomous alongside Langfuse and LangSmith but with a tighter OpenTelemetry/ OpenInference standardization posture and a notebook-friendly local mode. Worth modeling as a distinct node because it is the de-facto reference implementation of OpenInference semantic conventions for LLM traces.

Security notes

Core project is source-available under Elastic-2.0 and self-hostable via Docker, which makes it a data-residency-friendly observability option. Prompts and completions are the primary payload — redact secrets and PII via the SDK hooks before ingest. OpenInference instrumentation is framework-agnostic; audit auto-instrumentation patches for any that wrap sensitive call sites.

Addresses failure modes

  • Logic error — Correct API use, incorrect logic.
  • Context pollution — Trajectory drift, hallucination spirals, overlong or off-task context.
  • Fabrication — Generating reference to non-existent APIs, packages, or symbols.

Techniques

trace-captureopeninference-instrumentationllm-as-judge-evaldataset-regression-evalspan-annotation

View raw YAML on GitHub →