[Ollama](https://ollama.com)

Ollama is an open-source tool that lets you download and run large language models (LLMs) directly on your own computer instead of calling a hosted API, wrapping the hard parts of local AI, GPU setup, model formats, and memory management, behind one command: ollama run llama3. It ships with a command-line interface, a native GUI, and a local REST API, so you can swap Ollama into any app the same way you would call OpenAI or Anthropic. For developers building production systems, this matters because it keeps sensitive data off third-party servers, drops inference costs to zero once you own the hardware, and gives you a fast local sandbox for testing agents before you touch a cloud bill.

How Ollama works

Ollama runs as a background service on your machine. When you ask it to run a model, it downloads the model weights, the trained parameters that make the model work, from its library, loads them into memory, and exposes them through a local API on port 11434.

Model library: Ollama hosts a catalog of open-weight models, including Llama, Mistral, Gemma, Qwen, and DeepSeek, packaged in a format called GGUF that keeps file sizes small.

Modelfile: Every model ships with a Modelfile, a simple config file that sets the system prompt, temperature, and other defaults, and you can edit it to create your own custom model variants.

Local REST API: Ollama exposes an OpenAI-compatible API on localhost:11434, so existing code that calls a hosted LLM API can point at Ollama with minimal changes.

Quantization: Models are quantized, compressed to use fewer bits per parameter, so a model that needs over 100GB of memory at full precision can run in under 8GB on a laptop.

Cloud burst: When a task needs a model too large for your hardware, Ollama’s optional cloud tier runs it on hosted GPUs and returns the result through the same local interface.

What you can build with Ollama

Private AI chatbot: Build a ChatGPT-style assistant that never sends a message to an external server, useful for healthcare or legal teams handling sensitive records.

Offline coding assistant: Pair Ollama with Claude Code or another agent harness so engineers can get code suggestions on an air-gapped machine or in a low-connectivity environment.

Document summarization pipeline: Feed contracts, invoices, or support tickets to a local model for extraction and summarization, since processing files that can’t leave the network is often the fastest ROI use case for a team.

Retrieval-augmented generation app: Use Ollama’s embedding models to convert your documents into vector embeddings, numeric representations of text used for similarity search, then build a semantic search or FAQ system on top without paying per token for embeddings.

Local AI agent with tool calling: Combine an instruction-tuned model with tool calling, the ability for an LLM to call external functions or APIs, to build an agent that reads files, hits internal APIs, or automates a workflow entirely on your own infrastructure.

Multimodal image analysis: Run vision-capable models like Llama 3.2 Vision locally to describe images, extract text from screenshots, or classify photos without uploading them anywhere.