# LitServe

> Python framework for serving AI models as APIs with batching and scaling

Steal this: An AI server in one Python class, with batching and streaming in code and no config files.

- Category: Agents & dev (`agents`)
- Tags: `open-source`
- Official: https://lightning.ai/litserve
- GitHub: https://github.com/Lightning-AI/LitServe
- Wunderlust: https://wunderlust.xyz/r/litserve
- Image: https://wunderlust.xyz/media/litserve-poster.jpg
- Video preview: https://wunderlust.xyz/media/litserve.webm
- Added: 2026-11-10

## What it is

A minimal Python framework for building your own inference servers, for models, agents, RAG or pipelines. You control batching, routing and streaming in code, with no config files.

## What it’s for

For serving a model or agent as an API with batching and multiple GPUs, without MLOps glue.

## Benefits

- Inference logic written in plain Python
- Batching, streaming and multi-GPU scaling
- Runs locally, self-hosted or on Lightning AI

## Caveats

- According to its maker, it is twice as fast as FastAPI
- Apache-2.0 license; managed hosting on Lightning AI is paid

## Similar on Wunderlust

- [Moonshine Voice](https://wunderlust.xyz/r/moonshine-voice.md): Open source on-device speech to text and text to speech
- [Morphic](https://wunderlust.xyz/r/morphic.md): Open source AI search engine with cited answers and a generated interface
- [Paperclip](https://wunderlust.xyz/r/paperclip.md): Control center for agents organized like a company
- [Ternary Bonsai 2](https://wunderlust.xyz/r/ternary-bonsai-2.md): Family of 2B, 4B and 9B ternary language models for local use
- [Postiz](https://wunderlust.xyz/r/postiz.md): Open-source agentic scheduling tool for social media posts
- [MLX](https://wunderlust.xyz/r/mlx.md): Machine learning array framework for Apple silicon