Haystack

Haystack is an open-source AI orchestration framework built by deepset for building production-ready LLM applications in Python. It is designed around the idea of context engineering: giving you explicit, transparent control over how information is retrieved, filtered, ranked, and assembled before it reaches a language model. Haystack structures agents and applications as modular pipelines made up of components you connect together, test independently, and swap without rewriting the rest of the system. Enterprises like Airbus, Netflix, and Intel use it in production, and it has over 24,000 GitHub stars.

How Haystack works

The core idea is that every step in an AI application is a component. Components have defined inputs and outputs. You connect them into a pipeline, and data flows through the graph from retrieval to generation. Here is how the key pieces fit together:

• Components: The building blocks of every Haystack application. A component is any Python class that takes input, does something with it, and returns output. Built-in components include embedders, retrievers, prompt builders, generators, routers, and memory stores. You can also write your own.
• Pipelines: Directed graphs of components. You add components to a pipeline and connect their inputs and outputs explicitly. Pipelines can branch conditionally, loop back, and be serialised to YAML so you can store and version them like configuration files.
• Document stores: Where your indexed documents live. Haystack supports in-memory stores for prototyping and integrates with Elasticsearch, OpenSearch, Chroma, Weaviate, Pinecone, and others for production.
• Agents: Components that use an LLM to decide which tools to invoke and in what order, rather than executing a fixed pipeline. Haystack agents maintain state across steps and support streaming output.
• Tools: Any Haystack component or pipeline can be wrapped as a tool and handed to an agent. This means your existing retrieval pipelines can become agent capabilities without being rewritten.
• Hayhooks: A lightweight server that exposes Haystack pipelines and agents as REST API endpoints or MCP servers, so you can integrate them with external systems or AI clients.

What you can build with Haystack

• RAG question-answering system: Build a pipeline that embeds your documents, stores them in a vector index, retrieves relevant chunks at query time, and passes them to an LLM to generate grounded answers with citations.
• Conditional branching search: Wire a router component to fall back to live web search when the document store does not contain enough context to answer a question, so the system never returns an empty or confabulated response.
• Self-reflecting named entity recognition agent: Build a looping pipeline where the agent checks its own output against a validation schema and retries with corrected prompts until the extracted entities meet quality criteria.
• Hacker News summarisation tool: Create a custom component that fetches top posts from an external API and plugs it into a summarisation pipeline, demonstrating how Haystack extends to any data source.
• Document intelligence pipeline: Parse unstructured PDFs and Word files through a preprocessing pipeline that cleans, splits, and indexes them into a searchable store, then surface them through a conversational chat interface.
• MCP server from a pipeline: Use Hayhooks to expose any Haystack retrieval or agent pipeline as an MCP endpoint, making it immediately callable by Claude, Cursor, or any other MCP-compatible AI client.