Phoenix

Phoenix is Arize’s open-source platform for tracing, evaluating, and debugging LLM applications, including retrieval-augmented generation (RAG) systems and multi-step agents. It runs on OpenTelemetry, the open standard for collecting traces and metrics, so the data you generate stays portable instead of locked inside one vendor’s dashboard.

Developers get started with a single Docker container or a hosted notebook, then instrument their application to send traces to Phoenix automatically. For AI teams shipping RAG pipelines or agents, that matters because failures rarely show up as a crash. A bad retrieval, a hallucinated answer, or a tool call gone wrong tends to hide inside a working-looking response until someone traces the exact steps that produced it.

How Phoenix works

Phoenix instruments your LLM application using OpenInference, an open standard built on top of OpenTelemetry that captures what happens at each step of a request, from the initial prompt to any tool calls or retrieved documents.

Automatic tracing: Once instrumented, every prompt, model response, retrieval, and tool call gets recorded as a trace you can inspect in the Phoenix UI.
LLM-as-judge evaluation: Phoenix uses a separate LLM to score traces for qualities like relevance, hallucination, and toxicity, so you get automated quality signals instead of reading every transcript by hand.
Datasets and experiments: You version examples into datasets, then run experiments to see how a prompt or model change affects results before shipping it.
Embedding visualization: Phoenix clusters embeddings from your data to surface groups of similar failing queries or documents, which speeds up root-cause analysis.
Prompt playground: A built-in sandbox lets you compare prompts and models side by side using real production traces as test cases.

What you can build with Phoenix

RAG quality dashboard: Trace every retrieval and generation step in a document Q&A system, then use built-in evals to catch hallucinated answers before users see them.
Agent debugging console: Follow a LangGraph or CrewAI agent through every tool call it makes, and pinpoint the exact step where a multi-step task went off track.
Prompt regression testing: Version your prompts, run them against a saved dataset of real user questions, and catch quality regressions before deploying a change.
Self-hosted observability stack: Run Phoenix in a single Docker container alongside your application so all LLM traces and evaluation data stay inside your own infrastructure.
Framework-agnostic monitoring layer: Add Phoenix to a stack that already uses LlamaIndex, LangChain, or the OpenAI Agents SDK without rewriting instrumentation for each one.