Posts

Paste it into an agent that runs on your machine, like Claude Code or Codex. It runs one fixed test with llama-bench, asks before it downloads or installs anything, and prints a JSON result without hostnames, paths or serials. What it measures.

You are running the RunItLocal.io benchmark on this machine. Work step by step, show me each command before you run it, and do not install anything, download anything, or use sudo without asking me first. Do not change any system settings.

PRIVACY: Never include hostnames, usernames, IP or MAC addresses, serial numbers, UUIDs, file paths under my home folder, or anything from my environment variables in the output.

If anything this prompt needs is missing, stop and tell me the options. Never substitute another engine, model, format or quant; those results are not comparable with anyone else's.

STEP 1: HARDWARE AND SOFTWARE
Detect, using only read-only commands that fit this OS (Linux, Windows or macOS):
- OS name and version, kernel
- CPU model, physical cores, threads, max clock
- RAM total in GB, speed in MT/s if available, number of DIMMs if available
- Motherboard model if available
- Each GPU: model, VRAM in GB, driver version, CUDA/ROCm/Metal version, power limit in W, current PCIe generation and width under load (nvidia-smi, rocm-smi or system_profiler)
- Whether llama-bench is installed, and whether Ollama is installed. Other engines (LM Studio, MLX, vLLM) do not count for this test.
On Apple Silicon: vram_gb is the unified memory total, runtime is "Metal" plus its version from system_profiler, and driver, power_limit_w and pcie are null.

STEP 2: PICK THE TEST TIER
Pick the largest tier that fits in total GPU memory (or unified memory on Apple Silicon), and tell me which one you picked before continuing:
- Tier S (under 24 GB total): Qwen3-8B Q4_K_M, file Qwen3-8B-Q4_K_M.gguf (5.03 GB, sha256 d98cdcbd03e17ce47681435b5150e34c1417f50b5c0019dd560e4882c5745785) from https://huggingface.co/Qwen/Qwen3-8B-GGUF
- Tier M (24 GB or more total): Qwen3.6-35B-A3B UD-Q4_K_M, file Qwen3.6-35B-A3B-UD-Q4_K_M.gguf (22.13 GB, sha256 ac0e2c1189e055faa36eff361580e79c5bd6f8e76bffb4ce547f167d53e31a61) from https://huggingface.co/unsloth/Qwen3.6-35B-A3B-GGUF
Only that exact GGUF file counts. A model with the same name in another format (MLX, safetensors, EXL2, AWQ) or another quant does not, even if it is already on disk; do not use it.
If the file is not on disk, tell me its name and size and ask before downloading it with:
curl -L -o <file> https://huggingface.co/<repo>/resolve/main/<file>
After downloading, check its sha256 (shasum -a 256, sha256sum, or certutil -hashfile <file> SHA256 on Windows) and stop if it does not match.

STEP 3: GET LLAMA-BENCH
If llama-bench is not installed, ask me before installing llama.cpp:
- macOS with Homebrew: brew install llama.cpp
- Linux or Windows: the prebuilt release for this GPU (CUDA, HIP/ROCm or Vulkan) from https://github.com/ggml-org/llama.cpp/releases
Only if I say no to that and Ollama is installed, use the Ollama fallback at the end of STEP 4.

STEP 4: RUN THE TEST
First record idle GPU power for 10 seconds. Then run llama-bench twice, with all layers on GPU and flash attention on, sampling telemetry every second during each run:
Prefill: llama-bench -m <model> -ngl 99 -fa 1 -p 512,4096,16384 -n 0 -r 3 -o json
Decode:  llama-bench -m <model> -ngl 99 -fa 1 -p 0 -n 128 -r 3 -o json
Take prefill_tps and decode_tps from the "avg_ts" of each test in the JSON output; do not average them yourself. engine.version is "b<build_number> (<build_commit>)" from the same output.
If the model needs more than one GPU, use the default layer split and record how many cards it used.
Telemetry: power_w comes from the decode run only. temp_c, sm_mhz, vram_used_gb and throttle reasons cover both runs.
- NVIDIA: nvidia-smi --query-gpu=index,power.draw,temperature.gpu,clocks.sm,memory.used,clocks_event_reasons.active --format=csv,noheader,nounits -l 1 (on older drivers use clocks_throttle_reasons.active)
- AMD on Linux: rocm-smi --showpower --showtemp --showmeminfo vram --showclocks --json, once a second in a loop (or amd-smi metric on newer ROCm)
- macOS: sudo powermetrics --samplers gpu_power -i 1000 gives GPU power (mW, convert to W) and GPU active frequency (use as sm_mhz). It needs sudo, so ask me first. Temperature and VRAM are not available this way; set them to null.
- Anything else, or if I decline sudo: set the telemetry and derived fields to null and say why in notes.
Ollama fallback: load the same GGUF file with a Modelfile containing "FROM <model file>" and ollama create, then run "ollama run <name> --verbose" three times on one fixed prompt of about 512 tokens. Report the average prompt eval rate under prefill_tps "512" (4096 and 16384 null), the average eval rate as decode_tps, the exact prompt token count in notes, and set engine name to "ollama".

STEP 5: OUTPUT
Print exactly one JSON code block in this shape, with null for anything you could not detect, then a three-line plain summary. Do not invent or estimate numbers; mark anything inferred in "notes".

{
  "schema": "runitlocal-bench/1",
  "date": "YYYY-MM-DD",
  "system": {
    "os": "", "kernel": "",
    "cpu": {"model": "", "cores": 0, "threads": 0, "max_mhz": 0},
    "ram": {"total_gb": 0, "speed_mts": null, "dimms": null},
    "motherboard": null
  },
  "gpus": [
    {"model": "", "vram_gb": 0, "driver": "", "runtime": "", "power_limit_w": 0, "pcie": "Gen3 x16"}
  ],
  "engine": {"name": "llama.cpp", "version": "", "flags": "-ngl 99 -fa 1"},
  "test": {
    "tier": "M",
    "model": "Qwen3.6-35B-A3B",
    "quant": "UD-Q4_K_M",
    "gpus_used": 2,
    "prefill_tps": {"512": 0, "4096": 0, "16384": 0},
    "decode_tps": 0
  },
  "telemetry": {
    "idle_w": 0,
    "power_w": {"avg": 0, "max": 0},
    "temp_c": {"avg": 0, "max": 0},
    "sm_mhz": {"avg": 0, "min": 0},
    "vram_used_gb": {"max": 0},
    "throttled": false,
    "throttle_reasons": []
  },
  "derived": {
    "joules_per_token": 0,
    "kwh_per_million_tokens": 0
  },
  "notes": ""
}

Compute joules_per_token as average GPU power during decode divided by decode tok/s. Compute kwh_per_million_tokens as joules_per_token times 1,000,000 divided by 3,600,000.

When you're done, tell me I can send the JSON to RunItLocal.io by pasting it into the form at https://runitlocal.io/learn/posts#share, if I want to.