Docs | ArmTune Serve

ArmTune Serve / v0.1

A measured answer to a hardware-specific question.

Use these commands on native Arm64 Linux. Windows and x86 machines can run tests and the dashboard, but real inference evidence must come from the Arm target.

01 / QUICK START

Install and detect.

Start with the Python control plane, then install the platform-specific runtime and profiler.

git clone https://github.com/luxmikant/Arm-Tune.git
cd Arm-Tune
python3 -m venv .venv
source .venv/bin/activate
pip install -e .
bash scripts/install-performix.sh
bash scripts/build-llama-cpp.sh
armtune detect

What to look forThe output should identify aarch64, the CPU model, physical cores, memory, and features such as NEON, I8MM, SVE or SVE2.

02 / BENCHMARK

Pull a model, sweep the real choices.

The model connector lists GGUF variants and caches selected files locally. The benchmark keeps prompts, warmup, seed and output limits constant.

armtune models list unsloth/Llama-3.2-1B-Instruct-GGUF

export ARMTUNE_LLAMA_SERVER=llama.cpp/build-arm-opt/bin/llama-server
armtune benchmark \
  --profile configs/balanced.yaml \
  --repo unsloth/Llama-3.2-1B-Instruct-GGUF \
  --quant Q4_K_M,Q4_0,Q8_0 \
  --threads 1,2,4 \
  --concurrency 1,2

ModelGGUF quantization and size

Runtimethreads, batch threads, context

Serverslots, queue, P95/P99

03 / CPU TUNING

Why Arm-specific settings matter.

Prefill and decode are different workloads. Prefill is matrix-heavy and responds to batch settings; decode repeatedly touches weights and KV state and is sensitive to memory bandwidth, cache behavior and thread oversubscription.

SettingWhat it changesWhat to measure

-t threadsdecode parallelismdecode tok/s, P95

-tb batch threadsprefill parallelismprompt tok/s, TTFT

quantizationweight size and kernel pathRSS, quality, tok/s

concurrencyserver slots and queueingaggregate tok/s, P95

context / KV cachememory per requestRSS, feasibility

ArmTune builds a generic baseline and a KleidiAI/native Arm variant. The point is not to claim that every lower-bit model wins. The point is to measure which one wins here.

04 / ARM PERFORMIX

Explain the win.

Arm Performix profiles the inference process while the workload is running. The report can connect application metrics to hardware signals.

IPCinstructions per cycle

LLClast-level cache misses

BRbranch misprediction

BWmemory read/write

If a profiler command differs between Performix releases, ArmTune records the attempted command, version, stderr and status instead of presenting empty counters as success.

05 / QUALITY GATE

Speed without quality is cheating.

The reference workload asks for strict JSON. ArmTune scores structure, valid enums and agreement with expected labels. Any configuration below the quality threshold is excluded from recommendations — no matter how fast it is.

Scored fieldssummary, category, priority, recommended_action — plus an expected-category correctness bonus.

06 / RESULTS

Read the recommendation.

Each run creates a timestamped directory under results/.

results/20260813_134346/
├── results.json       # complete machine-readable record
├── results.csv        # spreadsheet-friendly comparison
├── report.md          # shareable written report
├── charts.png         # core metric comparison
├── sweeps.png         # thread/concurrency curves
├── performix.png      # Arm counter comparison
└── improvements.png   # relative change vs baseline

07 / RELEASE

Use it as a package or a workflow.

The Python control plane is packaged as armtune-serve. A trusted-publishing workflow is included for PyPI. The runtime binary and model weights remain separate, platform-specific artifacts.

python -m pip install armtune-serve
armtune --version
armtune dashboard

Publishing responsiblyPublish only after replacing placeholder benchmark values with real artifacts. Never upload gated model weights or credentials.