# Hanzo Engine — Hanzo AI

> High-throughput, Rust-native inference engine for LLMs and embeddings. Accelerated token generation, low latency, and minimal memory footprint.

hanzoai/engine

# Hanzo Engine

The inference engine, in one binary

Name a model on Hugging Face and it runs. Engine quantizes it for the hardware it finds, then serves an HTTP API and a web chat from the same process — text, vision, audio, speech, image and embeddings, without a second stack underneath.

[View on GitHub](https://github.com/hanzoai)

One process, laptop to GPU cluster

- run, serve, bench01
- Built to serve, not just to run02
- More than text03
- It quantizes on the way in04
- The silicon you have05
- Tools, and a UI to watch them06

One binary, three verbs. Chat with a model, put it behind an HTTP API, or measure it — the thing you benchmarked is the thing you shipped.

Paged attention and continuous batching keep the card busy across concurrent requests. A disk-first KV cache carries a session across a restart, so an agent that reuses a long prefix does not pay for it twice.

Vision, audio, speech, image and embeddings go through the same modality pipeline as generation, in the same process. A multimodal app is one deployment, not four.

In-situ quantization of any Hugging Face model, plus GGUF from 2 to 8 bits, GPTQ, AWQ, HQQ, FP8 and bitsandbytes. Ask for a level and it takes a published prebuilt when one exists, or makes one when it doesn't.

CUDA with FlashAttention and multi-GPU tensor parallelism, Metal on Apple Silicon, or plain CPU. Same engine and same commands on each, and a doctor command that reports what it found.

It can call MCP servers over a process, HTTP or a WebSocket. The built-in UI at /ui shows reasoning, code execution and plots inline, and edits branch a conversation with its own state. Pass --no-ui and it is a plain API server.

Paste and ship

Paste this into any agent. It reads the skill manifest and calls Engine from there.

Agent promptCopy

```
Read https://hanzo.ai/skill.md and use Hanzo Engine in my project. Start with: hanzo engine serve
```

## Get started with Hanzo Engine

[View on GitHub](https://github.com/hanzoai)

## Get Engine

AI inference engine

License: Apache-2.0[hanzoai](https://github.com/hanzoai)

[Deploy to Cloud](https://platform.hanzo.ai/deploy)