X local-model bench digest — Sep 24, 2026
As of Sep 24, 2026 — 12 posts collected from X. 30 measured runs catalogued, 1 queued for review, 11 kept to the digest only. Latest source post Sep 24, 2026 01:31 UTC.
Every number below is what its author published, transcribed with the engine, quantization and context length they reported. Sources are linked on each entry. Numbers without a full methodology stay out of the benchmark board until they are reviewed.
The 12 most recent posts are listed below; 3 older entries from this window are in the daily collection report (a digest-only entry never reaches the database).
Unspecified platform × MiMo-v2.6-Flash
Sep 24, 2026 · by @ayayalar
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | — | — | 33.7 tok/s | — |
| — | — | — | 51.9 tok/s | — |
I already like MiMo-v2.6-Flash better than DeepSeek v4.1f. It has a great personality too! MiMo-v2.6-Flash / decode tok/s results (Prose): ×1 33.7 agg 33.7/str · TTFT 348ms ×2 51.9 agg 26.2/str · TTFT 546ms https://t.co/Gu3w81LqXg
Digest-only: hardware.name missing; incomplete methodology (engine+version, quantization, context length, hardware state, measurement method).
RTX 3060 × MiMo-V2.6-Distill-Qwen-9B
Sep 24, 2026 · by @Oluwaphilemon1
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| 262144 | Q5_K_M | llama.cpp | 47.0 tok/s | — |
| 262144 | Q5_K_M | llama.cpp | — | 1600.0 tok/s |
A 12GB RTX 3060 is now running a 9B reasoning model with the full 262K context window. The model is Xiaomi’s MiMo-V2.6-Distill-Qwen-9B, running at Q5_K_M on a single RTX 3060. The numbers from this setup: 12GB VRAM 262K context ~47 tok/s decode ~1,600 tok/s prefill Q5_K_M quantization Flash Attention enabled Q8 KV cache Reasoning enabled And the setup is surprisingly straightforward with llama.cpp. llama-server \ -m…
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, hardware state, measurement method).
M5 Max × Qwen3.8-Flash-Next
Sep 24, 2026 · by @Oluwaphilemon1 · third-party report
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | — | — | 90.0 tok/s | — |
| — | — | — | 45.0 tok/s | — |
Qwen3.8-Flash-Next is shaping up to be seriously fast on Apple Silicon. The upcoming package is expected to be around 49GB, which means a 64GB+ M5 Max machine should have enough unified memory to keep the model comfortably resident. The current target is around 70 to 90 tokens per second on M5 Max. That is the part that caught my attention. A model in this class running locally at that speed changes what local AI…
Digest-only: confidence 0.35 below 0.60; incomplete methodology (engine+version, quantization, context length, hardware state, measurement method).
Snapdragon X2 Elite × Pokee-Isaac-fast
Sep 23, 2026 · by @Pokee_AI
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | — | — | — | 1000.0 tok/s |
| — | — | — | 20.0 tok/s | — |
Pokee AI on the main stage at #SnapdragonSummit today 🔴 Previewing Pokee-Isaac-fast in live demos with Qualcomm: a 36B agent model running fully local on Snapdragon X2 Elite with even 32GB RAM. ~1,000 tok/s prefill, ~20 tok/s generation. Come see it live. Thanks @Snapdragon https://t.co/kSaUNKQCmN
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, quantization, context length, hardware state, measurement method).
Spark × GLM-5.3-Flash
Sep 23, 2026 · by @filicroval
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| 262K | — | — | 30.0 tok/s | — |
GLM-5.3-Flash running smoothly at 30 tok/s on a single Spark (262k context + vision + tool calls) quality is maintained. recipe dropping soon! https://t.co/bVG6mSSAwC
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, quantization, hardware state, measurement method).
GB200
Sep 23, 2026 · by @SemiAnalysis_ · third-party report
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | — | vllm | 700.0 tok/s | — |
ALERT ALERT ALERT 🚨 🚨 🚨 VLLM MAINTAINERS HAVE JUST SHOWN THAT TPUv7 CAN GET 700 tok/s/user, 56% BETTER PERFORMANCE THAN NVIDIA GB200 NVL72 THROUGH MEGAKERNEL OPTIMIZATION ON KIMI K3. As we said awhile ago, the TPU externalization of software is full steam ahead. This is ultra important to follow the progress of this.
Digest-only: confidence 0.35 below 0.60; model.name missing; incomplete methodology (engine+version, quantization, context length, hardware state, measurement method).
RTX 3060 × Bonsai 2 27B + MTP
Sep 23, 2026 · by @sudoingX
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| 125K | — | — | 22.0 tok/s | — |
| 125K | — | — | 50.0 tok/s | — |
this is what 12gb of vram builds in 2026, absolute magic rtx 3060 12gb, #1 gpu on steam bonsai 2 27b + mtp, 5.95 gb of weights hermes agent, 5 hours, 328k tokens written 8 js files, 2,368 lines, zero hand written code 50 tok/s fresh, 22 tok/s average, 125k context watch the full video, 5 hours in 12 minutes of pure dance of a local ai model on rtx 3060 12gb vram, and stay till the end for the full gameplay. this…
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, hardware state, measurement method).
TPU × Kimi K3
Sep 23, 2026 · by @vllm_project
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | — | pallas | 709.0 tok/s | — |
| — | — | — | 450.0 tok/s | — |
The @inferact team open sourced a TPU megakernel for Kimi K3 achieving 709 tokens/s, against 450 tokens/s on GB200. All 92 of K3's MoE layers run in a single Pallas kernel, with weight prefetching that reaches across layer boundaries so transfers for one layer overlap with computation in the previous one. Shoutout to the team! Writeup and repo link in the thread.
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, quantization, context length, hardware state, measurement method).
TPU × Kimi K3
Sep 23, 2026 · by @inferact
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | — | pallas megakernel | 709.0 tok/s | — |
| — | — | gb200 baseline | 450.0 tok/s | — |
Our first TPU megakernel for Kimi K3 reaches 709 tokens/s on low-concurrency decode, against 450 tokens/s for our GB200 baseline, both with DSpark speculative decoding. To our knowledge, this is the first TPU inference megakernel. The whole model runs in a single Pallas kernel, and without spec decoding it is roughly 1.4 to 2x the GB200 baseline at batch sizes 1 through 8. We are open sourcing it today. 1/2
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, quantization, context length, hardware state, measurement method).
Unspecified platform × Qwen 3.8 27B
Sep 23, 2026 · by @CopenDeCamp
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | 4bpw | dflash2 | 73.5 tok/s | — |
Qwen 3.8 27B 独自4bpw量子化 DFlash2 73.50 tok/s https://t.co/7ipP0KzrSr
Digest-only: hardware.name missing; incomplete methodology (engine+version, context length, hardware state, measurement method).
DGX Spark x4 × DeepSeek-V4.1-Flash
Sep 23, 2026 · by @ViC305
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | EXL3 4.75 bpw SAGE | vllm | 30.2 tok/s | — |
| — | EXL3 4.75 bpw SAGE | vllm | 33.4 tok/s | — |
| — | EXL3 4.75 bpw SAGE | vllm | 27.8 tok/s | — |
| — | EXL3 4.75 bpw SAGE | vllm | 43.9 tok/s | — |
| — | EXL3 4.75 bpw SAGE | vllm | 57.7 tok/s | — |
| — | EXL3 4.75 bpw SAGE | vllm | 61.5 tok/s | — |
DeepSeek-V4.1-Flash EXL3 is finally alive across 4× DGX Sparks. 🔥 I built the quality-first 4.75 bpw SAGE quant. @cfontes and @Blackwellboy spent more than a week, 14 PRs, dozens of commits and 40+ bring-up iterations making it actually run. I don’t own one Spark, let alone four. This is what open-source collaboration looks like. 𝗧𝗛𝗘 𝗥𝗘𝗖𝗘𝗜𝗣𝗧𝗦 Single-stream decode: 30.2 tok/s prose 33.4 tok/s code 33.37 tok/s best…
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version).
Tesla V100 x2 × Qwen3.8-Flash-Next-GSQ-RCO-IQ3XXS
Sep 23, 2026 · by @PeasantSmith
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| 131K | IQ3XXS | llama.cpp b11030-mix-5ff778e | 80.9 tok/s | — |
| 131K | IQ3XXS | llama.cpp b11030-mix-5ff778e | 103.3 tok/s | — |
| — | IQ3XXS | llama.cpp b11030-mix-5ff778e | 42.8 tok/s | — |
| 512 | IQ3XXS | llama.cpp b11030-mix-5ff778e | 87.5 tok/s | — |
| 4K | IQ3XXS | llama.cpp b11030-mix-5ff778e | 96.1 tok/s | — |
| — | IQ3XXS | llama.cpp b11030-mix-5ff778e | 88.1 tok/s | — |
| — | IQ3XXS | llama.cpp b11030-mix-5ff778e | 103.3 tok/s | — |
Two used Tesla V100s. The GPUs the modern inference stack abandoned (not for us peasants!) Qwen3.8-Flash-Next-GSQ-RCO-IQ3XXS +Vision +131k ctx 103.3 tok/s peak 80.9 tok/s stable Same model. Same cards. The gap is context length, load balance and more nuances, not magic. What actually 2.4×’d the run: The MTP draft head 🏅 Without it: 42.8 tok/s. 🪫 With it: 103.3 tok/s 🔋 Settled Draft acceptance: 87.8% 🎯 Then the…
Full methodology stated (engine, quantization, context, hardware state, method) — 3 added to the benchmark board (3 of 7 measured rows queued; the rest are published here only).
Verification status
This digest is an automated transcription of public posts. Runs with a stated engine build, quantization, context length, hardware state and measurement method are published straight to the benchmark board with a link to their source; the rest stay here. These are author-reported numbers, not an independent DiggTop re-measurement: every row carries its provenance so you can check the original, and numbers that fail review are removed. Our own testing follows the methodology. Spotted an error? Reply on the source post or submit a corrected run at /submit.
Sources
- @ayayalar — Sep 24, 2026
- @Oluwaphilemon1 — Sep 24, 2026
- @Oluwaphilemon1 — Sep 24, 2026
- @Pokee_AI — Sep 23, 2026
- @filicroval — Sep 23, 2026
- @SemiAnalysis_ — Sep 23, 2026
- @sudoingX — Sep 23, 2026
- @vllm_project — Sep 23, 2026
- @inferact — Sep 23, 2026
- @CopenDeCamp — Sep 23, 2026
- @ViC305 — Sep 23, 2026
- @PeasantSmith — Sep 23, 2026
FAQ
Are the numbers in this digest verified?
They are transcribed from public X posts and the repositories those posts link to, with attribution and a source link on every entry. Rows that state the full methodology are published to the benchmark board automatically, so treat them as author-reported until a dispute is raised: the source link on every row is the check.
What does it take for a run to reach the verified board?
Engine and exact version, quantization, context length, hardware state and a described measurement method. Author-reported posts that state all five are queued as pending and can be promoted by a reviewer; anything missing a condition stays out of the database.
Where does the data come from?
Public X posts about local-model inference measured on owned hardware, discovered by keyword search every two hours and linked back to the original thread. The digest is regenerated daily.