Quality is a constraint.
Compare Q4, Q8 and other GGUF variants without allowing a fast but invalid structured response to win.
Quality gate →Arm64 / CPU-first / Performix-powered
Tune every token to the architecture it runs on.
ArmTune Serve turns an Arm cloud server's hardware fingerprint into a measured LLM deployment configuration. Less guesswork. More useful CPU time.
Built for Graviton, Cobalt, Axion, Ampere and native Arm64 CI.
armtune / benchmarkLIVE
$ armtune detect
architecture
aarch64 / Neoverse-N2
features
NEON I8MM SVE2 BF16
$ armtune recommend --latest
recommendedQ4_K_M / 4 threads / 1 slot
TTFT0.31s-28%
DECODE18.4tok/s
QUALITY0.91pass
$ llama-server -m model.gguf -t 4 -np 1
01DetectISA, cores, memory, NUMA
02Measurereal inference on real hardware
03ExplainArm Performix counters
04Deploycopy-paste launch command
01Why this matters
The same model does not have one best configuration.
Quantization, thread count, memory pressure and concurrency interact with the processor underneath. A setting that wins on a 4-core Neoverse-N2 can lose on a larger Graviton machine.
Threads are not free.
Separate decode and prefill behavior. Find where extra cores stop helping and start competing for bandwidth.
CPU tuning →Numbers need a reason.
Arm Performix adds IPC, cache, branch and memory evidence to explain why a configuration wins.
Performix →02How it works
One workflow from model to production command.
ArmTune sits around the runtime you already trust. It does not replace llama.cpp. It makes the deployment decision observable and repeatable.
01
Fingerprint
Read the CPU's instruction set, memory, cores and NUMA topology.
02
Source
Pull a GGUF model and quantization directly from Hugging Face.
03
Sweep
Run controlled tests across quantization, threads and concurrency.
04
Recommend
Rank the quality-safe configurations and export the command.
03Evidence, not folklore
Make the bottleneck visible.
Arm Performix counterscaptured with the workload
measured on Neoverse-N2 · 4 cores · 15.6 GB · aarch64Llama-3.2-1B-Instruct (GGUF)
Decode throughputbaseline 26.1 → tuned 33 tok/s
26.4% betterTime to first tokenbaseline 0.45 → tuned 0.33 s
-26.7% betterP95 latencybaseline 2.52 → tuned 1.91 s
-24.2% betterPeak process RSSbaseline 1,701 → tuned 1,616 MB
-5.0% betterEvery published number comes from the public ARM64 CI workflow and is stored as JSON, CSV, Markdown and chart artifacts in the repository. Nothing on this page is hand-typed.
04Built for real workflows
Useful before the first request reaches production.
ArmTune turns a performance question into an artifact a team can review, automate and deploy.
01
ML engineer
Compare model size, quality and speed on the actual Arm target before selecting a quantization.
02
Platform team
Run the benchmark in CI and detect a performance regression when a runtime or model changes.
03
Cloud architect
Compare a CPU-only deployment profile with a future CPU-GPU split using the same workload contract.
05Arm cloud, without hardcoding a vendor
Detect the capability. Tune the machine.
ArmTune works from the CPU fingerprint, so the same workflow travels across Arm64 Linux environments.
AWS GravitonAzure CobaltGoogle AxionAmpereNeoverse N1Neoverse N2Neoverse V2Arm64 CI
OPEN SOURCE / ARM64