Promptfoo

Promptfoo is an open-source command-line tool and library for testing, evaluating, and red-teaming large language model applications before they reach production. Instead of eyeballing outputs one prompt at a time, you define test cases in a declarative YAML config, run them against 50+ providers including OpenAI, Anthropic, and local models, and compare results side by side in a web viewer or the terminal. The same engine also runs automated red team scans that simulate prompt injection, jailbreaks, and data leakage attempts, the kind of adversarial testing that once only major AI labs had the resources to run. For developers shipping AI features into production, that combination turns quality and security checks into something you can run in CI, not something you hope holds up after launch.

How Promptfoo works

Promptfoo runs from the command line or as a Node.js library, and everything starts with a config file. You write your prompts, list the providers you want to test, and describe the test cases you care about, no notebook or dashboard required to get your first result.

Test cases: You define a set of inputs and, optionally, the expected properties of a good output, based on your real use cases and known failure modes.

Providers: You point the same test cases at any mix of models, OpenAI, Anthropic, Azure, Bedrock, Ollama, or a custom API, so you can compare quality, cost, and latency across them side by side.

Assertions and scorers: Promptfoo grades each output automatically using deterministic checks like regex or JSON schema validation, an LLM-as-a-judge model, or your own custom scoring function.

Red team plugins: For security testing, Promptfoo swaps test cases for adversarial plugins that generate attacks like prompt injection, jailbreaks, and unauthorized tool use, then grades whether your app resisted them.

CI/CD integration: You wire the same commands into GitHub Actions or another pipeline, so every pull request runs the eval or red team suite automatically and fails the build if quality or security regresses.

What you can build with Promptfoo

Regression suite for a customer-facing chatbot: Write test cases from real support conversations and run them on every prompt change, so a tweak that fixes one bug cannot silently break five other cases.

Model comparison matrix before a migration: Point the same test suite at two or three candidate models side by side and compare accuracy, cost, and latency in one view before you commit to a switch.

Automated red team for an internal RAG assistant: Generate adversarial inputs that probe for data leakage and access control failures in a retrieval-augmented app connected to internal documents, and catch a leak before it reaches a real user.

CI gate for an agent that calls tools: Add a GitHub Action that runs a red team scan for tool misuse and privilege escalation on every pull request, so an agent with database or API access cannot ship with an unguarded permission gap.

Compliance report for a regulated industry app: Run red team plugins mapped to OWASP LLM Top 10 and NIST AI RMF categories to produce a risk report you can hand to a compliance or security review board.