Modal

Modal is a serverless cloud platform built for AI and data workloads. Instead of provisioning servers or writing Kubernetes configuration, you define your compute needs, GPUs, memory, dependencies, directly inside your Python code, and Modal runs it on demand. Containers start in about a second, scale automatically from zero to thousands of GPUs, and bill by the second, so you never pay for infrastructure sitting idle. Companies like Runway, Suno, and Quora use Modal to run real-time inference and large-scale training without hiring a dedicated infrastructure team. For AI engineers who would rather write application code than manage infrastructure, Modal removes the operations layer entirely.

How Modal works

Modal turns your Python functions into cloud infrastructure using decorators, small annotations you add directly above a function to tell Modal how to run it. Here is how the pieces fit together:

• Functions: Any Python function decorated with @app.function() becomes a unit Modal can run in the cloud, complete with its own container image, GPU type, and memory limits defined right in the code.
• Apps: A collection of functions, classes, and schedules that deploy together as one unit, versioned and managed as a single Modal app.
• Container images: Modal builds a container image, a packaged environment with your code’s dependencies included, for each function automatically, with custom images supported for specific library versions.
• Autoscaling: Modal spins up new containers within about a second of a request arriving and shuts them down when demand drops, scaling all the way to zero between bursts of traffic.
• Sandboxes: Isolated, ephemeral environments for running untrusted or AI-generated code safely, used heavily by coding agents and reinforcement learning workloads.
• GPU scheduling: You request a GPU type like T4, A100, H100, or B200 as an argument on the function itself, and Modal handles provisioning it across its distributed cloud capacity.

What you can build with Modal

• Production LLM inference API: Deploy an open-source or custom language model behind an endpoint that streams tokens in real time and scales to zero when no one is using it.
• Fine-tuning pipeline: Run supervised fine-tuning or LoRA training jobs on a schedule, spinning up multiple GPUs for a training run and releasing them the moment it finishes.
• Coding agent sandbox: Give an AI coding agent an isolated, disposable environment to run and test code changes safely, without risking your production systems.
• Batch data processing job: Fan out a large dataset across hundreds of parallel containers to run embeddings, transcription, or evaluation jobs far faster than a single machine could.
• Multi-modal generation service: Serve image, video, or audio generation models behind an API, using Modal’s GPU autoscaling to handle unpredictable traffic spikes from end users.
• Reinforcement learning environment: Spin up thousands of concurrent rollout environments for RL training, keeping GPU inference resources saturated across every episode.