Prometheus is an open-source monitoring system that scrapes metrics from your services on an interval, stores them as time-series data, and lets you query and alert on them with PromQL.
Prometheus is an open-source monitoring system and time-series database originally built at SoundCloud in 2012 and now a graduated project under the Cloud Native Computing Foundation (CNCF) — the same foundation that oversees Kubernetes. It collects metrics from your services by pulling (scraping) from HTTP endpoints on a regular interval, stores those metrics as time-series data, and provides a powerful query language called PromQL for analysis and alerting. Prometheus is the standard metrics layer in cloud-native infrastructure and the most widely deployed open-source monitoring system in production today.
Prometheus operates on a pull model. Rather than waiting for your services to push metrics to a central collector, Prometheus reaches out to each service on a schedule — every 15 or 30 seconds by default — and pulls the current metric values from a /metrics HTTP endpoint. This design makes it easy to inspect what Prometheus is actually collecting, debug instrumentation problems, and control scrape frequency per target.
The main components of the Prometheus stack are:
The Prometheus server is the core binary. It handles service discovery (finding the targets to scrape), the scraper (making the HTTP requests), the TSDB (time-series database storing the scraped data on local disk), and the query engine (executing PromQL queries over that data). The server operates standalone without external dependencies, which makes it simple to deploy and reason about.
Exporters are lightweight programs that translate metrics from systems that do not natively expose a /metrics endpoint into Prometheus-compatible format. The node exporter exposes host-level metrics (CPU, memory, disk, network). The Postgres exporter exposes database metrics. The Blackbox exporter probes HTTP, DNS, and TCP endpoints and reports availability. There are hundreds of community-maintained exporters for virtually every common system.
Instrumentation libraries let you add custom metrics directly to your application code. Official libraries exist for Go, Java, Python, Ruby, and many other languages. You define a metric (a counter for HTTP requests, a histogram for response latency), increment or observe it in your code, and Prometheus scrapes it automatically. For AI applications, this is how you instrument token counts, agent loop depth, tool call latency, and LLM API cost per request.
Alertmanager handles the routing and deduplication of alerts. You define alert rules in PromQL (for example: alert if the p95 latency exceeds 2 seconds for 5 consecutive minutes). When a rule fires, Prometheus sends the alert to Alertmanager, which routes it to the right channel — Slack, PagerDuty, email, or a webhook — and suppresses duplicate notifications during an ongoing incident.
Service discovery keeps Prometheus in sync with dynamic infrastructure. In a Kubernetes cluster, Prometheus discovers pods, services, and endpoints automatically using the Kubernetes API. As pods scale up and down, Prometheus adds and removes scrape targets without manual configuration changes.
AI agent observability dashboard. Instrument your LangGraph or CrewAI agent with custom Prometheus metrics: a counter for total LLM API calls, a histogram for per-call latency, a gauge for current active sessions, a counter for tool call errors by tool name. Scrape those metrics with Prometheus and visualise them in Grafana. You get real-time visibility into what your agent is doing, where it is slow, and how much each run costs — the same observability setup the BuildrLabs bootcamp builds in its Capstone projects.
Kubernetes cluster monitoring. Deploy the kube-state-metrics and node-exporter alongside Prometheus in your cluster. You immediately have metrics for pod restarts, CPU and memory usage per namespace, node availability, and persistent volume status. Pair with Grafana’s Kubernetes dashboards (available as importable JSON) and Alertmanager rules for out-of-memory and pending pod alerts. This is the baseline production monitoring setup for any Kubernetes-hosted AI application.
API reliability tracking. Wrap your API server with Prometheus’s HTTP middleware (available for FastAPI, Flask, Express, and most common frameworks). You get per-endpoint request rate, error rate, and latency percentiles automatically. Define an alert that fires when the p95 latency for any endpoint exceeds your SLO, route it to PagerDuty via Alertmanager, and you have a basic reliability feedback loop without a commercial APM tool.
LLM cost monitoring. Instrument each LLM API call with a Prometheus counter that increments by the token count of each request, labelled by model name and operation type (embedding, completion, reranking). Aggregate with PromQL to compute total tokens per model per hour, identify expensive operations, and alert when daily spending projects above budget. Far cheaper than a dedicated LLM observability SaaS for teams with straightforward needs.
Multi-service dependency mapping. Use Prometheus federation to pull summary metrics from multiple Prometheus servers (one per service team or environment) into a top-level aggregator. You get a global view of system health — cross-service error rates, end-to-end latency, and dependency availability — without coupling individual teams’ monitoring configurations.
Prometheus is fully free and open-source under the Apache 2.0 license. There are no paid tiers, no enterprise editions with separate pricing, and no usage caps. You self-host Prometheus on your own infrastructure — Kubernetes, a VM, or a bare-metal server. For teams that do not want to manage Prometheus themselves, managed options include Grafana Cloud (which hosts a Prometheus-compatible remote write endpoint) and Amazon Managed Service for Prometheus, both of which add cost but reduce operational burden.
Download the Prometheus binary from prometheus.io or pull the official Docker image (prom/prometheus). Write a prometheus.yml configuration file that defines your scrape targets. Start the server and open the expression browser at http://localhost:9090. Run PromQL queries against any metrics Prometheus has scraped. For production, deploy Prometheus on Kubernetes using the kube-prometheus-stack Helm chart, which bundles Prometheus, Alertmanager, the node exporter, kube-state-metrics, and a set of default Grafana dashboards as a single installation.
For a hands-on path through building production AI systems with proper observability — Prometheus, Grafana, and LLM-specific instrumentation included — the BuildrLabs Agentic AI Bootcamp covers the full stack in person in Colombo over 16 weeks.
Yes. Prometheus is fully open-source under the Apache 2.0 license and a graduated CNCF (Cloud Native Computing Foundation) project. There is no commercial edition and no usage fee. You provision and operate your own Prometheus server — either on a VM, in a container, or on Kubernetes — and the only cost is the infrastructure you run it on.
PromQL is Prometheus's built-in query language for working with time-series data. It lets you compute rates (requests per second), aggregates (total error count across all pods), percentiles (p95 latency), and arithmetic combinations of any metrics your services expose. PromQL expressions are used to build Grafana dashboards, define alert conditions in Alertmanager, and run ad-hoc analysis. It has a moderate learning curve — the concepts of instant vectors, range vectors, and label matching take a few hours to internalise but become intuitive quickly in practice.
Prometheus is self-hosted and free; Datadog is a managed SaaS platform with per-host or per-metric pricing that can reach thousands of dollars per month at scale. Prometheus gives you full control over your data and costs, but requires you to operate the infrastructure and configure integrations yourself. Datadog provides a polished UI, built-in APM, and managed alerting with less setup. Most teams self-host Prometheus with Grafana for cost control and add a commercial tool later if operational overhead becomes a bottleneck.
Grafana is the open-source dashboarding layer for production systems. It connects to Prometheus, Loki, Tempo, and 60+ data sources and turns raw metrics into the dashboards your team actually looks at.
ObservabilityPromptfoo is an open-source CLI and library for testing, evaluating, and red-teaming LLM applications and agents before they reach production.
Observability
Phoenix is Arize's open-source platform for tracing, evaluating, and debugging LLM, RAG, and agent applications, self-hostable for free.
Observability
Helicone is an open-source AI gateway that logs, traces, and optimizes every LLM API call with one line of code.
ObservabilityUpdates from the AI world — what shipped, what we’re using in production, and what’s worth your attention. Two emails a month, no spam.
