Five chips, two runtimes · TTFT and TPOT

Median time to first token and time per output token, from 60 to 90 benchmark prompts per cell (GSM8K, IFEval, BFCL). Prompts run 19 to 772 tokens, median 92.

Latency map

Each line joins one chip's two runtimes. Lower and left is faster. Both axes are logarithmic.

All models, all chips

The same numbers as a grid, both columns on logarithmic scales. llama.cpp ExecuTorch. The model picked above is banded.

Not matched conditions. The Dimensity 9400 ran in hand under a desk fan. The Snapdragon 8 Gen 3 ran on Qualcomm Device Cloud and the 8 Elite, Tensor G5 and Exynos 2400 on Firebase Test Lab, racked, where phones throttle within about 40 seconds.

Two open gaps. Tensor G5 and Exynos 2400 decode at about half speed on llama.cpp against ExecuTorch, cause unknown. SmolLM3 on ExecuTorch prefills at a third of llama.cpp's rate on every chip, which is the compiled model rather than the phones.

Files. llama.cpp ran Q4_K_M GGUFs, Qwen3 at Q8_0. ExecuTorch ran the published XNNPACK INT8/INT4 exports, LFM2.5 as 8da4w and Llama 3.2 as SpinQuant. Runs on 2026-09-03 and 04, single pass, no repeats.