# Engine prefill on CUDA vs llama.cpp — Hanzo AI

> hanzo-engine against llama.cpp on CUDA, same weights, same box: prefill throughput where the whole 95% interval of the ratio lies above parity.

[Benchmarks](https://hanzo.ai/benchmarks) / Engine prefill on CUDA vs llama.cpp

measured

# Engine prefill on CUDA vs llama.cpp

hanzo-engine against llama.cpp on CUDA, built against the same weights and run on the same box: prefill throughput at the prompt lengths where the whole 95% interval of the ratio lies above parity.

harness

[hanzoai/engine · bench/dossier](https://github.com/hanzoai/engine/tree/bench/dossier/bench-runs)

our paper

[Refutation-Driven Performance Engineering](https://papers.hanzo.ai/refutation-driven-perf)

`scripts/dossier.sh --backend cuda --model -p 512,500,2048,4096 -n 128 -r 7`prints the table on this page — transcribed from the committed run files on branch bench/dossier, the pass of 2026-07-21

## What is measured

hanzo-engine and llama.cpp, built against the same weights, run on the same box, alternating. Prefill is tokens per second at a given prompt length, and the ratio is ours over theirs, so above 1.000 is a win. The models are Zen-1.7B at Q4_K_M and Zen-4B at Q4_K_M; 7 repetitions per cell, as its run records, with the first discarded as warmup, a Student-t 95% interval on each engine, and the ratio's interval formed by adding the two relative intervals in quadrature.

The verdict is the harness's, not a reading of ours: WIN where the whole interval lies above 1.000 and neither engine's coefficient of variation exceeded 5%, which is how the harness says the box was quiet. Every cell below is one.

## Prefill on CUDA, cell by cell

3 cells from 2 runs on branch `bench/dossier` · engine 1.7.87 · each run pins its engine release, its llama.cpp build and its model digest, and is linked below

backend

model

quant

prompt tokens

llama.cpp t/s

hanzo t/s

ratio

95% interval

verdict

CUDA

Zen-1.7B

Q4_K_M

500

7250.2

7961.3

1.098

1.053 – 1.143

WIN

CUDA

Zen-1.7B

Q4_K_M

4096

6871.3

7474.8

1.088

1.051 – 1.125

WIN

CUDA

Zen-4B

Q4_K_M

2048

3592.5

3774.4

1.051

1.017 – 1.085

WIN

## The work that produced it

[Refutation-Driven Performance Engineering](https://papers.hanzo.ai/refutation-driven-perf) is the campaign that produced these kernels, written as a log of numbered hypotheses, each settled by a cheap decisive experiment. A long prefill is compute-bound, which is where that work landed.

Hanzo AI Cloud

### Build on Hanzo Cloud

[Start on Hanzo Cloud](https://cloud.hanzo.ai)[View Models & Pricing](https://hanzo.ai/pricing)

[PreviousSandbox cold start](https://hanzo.ai/benchmarks/sandbox)

[All Benchmarks Board](https://hanzo.ai/benchmarks)

[NextGPQA-Diamond](https://hanzo.ai/benchmarks/gpqa)

Other benchmarks[LoCoMo](https://hanzo.ai/benchmarks/locomo) · [MemoryAgentBench](https://hanzo.ai/benchmarks/memoryagentbench) · [LongMemEval](https://hanzo.ai/benchmarks/longmemeval) · [CronQuestions](https://hanzo.ai/benchmarks/cronquestions) · [RepoBench-R](https://hanzo.ai/benchmarks/repobench-r) · [LoCoMo · subject scope](https://hanzo.ai/benchmarks/locomo-subject-scope) · [LoCoMo-Conv](https://hanzo.ai/benchmarks/locomo-conv) · [Fleet residency](https://hanzo.ai/benchmarks/fleet) · [Live agent footprint](https://hanzo.ai/benchmarks/goroutine) · [Sandbox cold start](https://hanzo.ai/benchmarks/sandbox) · [GPQA-Diamond](https://hanzo.ai/benchmarks/gpqa)

the committed runs, in [bench-runs](https://github.com/hanzoai/engine/tree/bench/dossier/bench-runs): CUDA Zen-1.7B Q4_K_M · CUDA Zen-4B Q4_K_M