X local-model bench digest — Sep 16, 2026
As of Sep 16, 2026 — 12 posts collected from X. 36 measured runs catalogued, 12 kept to the digest only. Latest source post Sep 16, 2026 12:29 UTC.
Every number below is what its author published, transcribed with the engine, quantization and context length they reported. Sources are linked on each entry. Numbers without a full methodology stay out of the benchmark board until they are reviewed.
The 12 most recent posts are listed below; 6 older entries from this window are in the daily collection report (a digest-only entry never reaches the database).
DGX Spark x2 × DeepSeek-V4.1-Flash
Sep 16, 2026 · by @wei_wang
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | EXL3 2.9bpw | custom cooperative moe kernel | 42.0 tok/s | — |
| — | EXL3 2.9bpw | custom cooperative moe kernel | 32.0 tok/s | — |
| — | EXL3 2.9bpw | custom cooperative moe kernel | 63.7 tok/s | — |
| — | EXL3 2.9bpw | custom cooperative moe kernel | 42.5 tok/s | — |
双 DGX Spark 跑 DeepSeek V4.1 Flash 又提速了。 单路 Prose 从约 32 tok/s 提升到 42 tok/s，快了约 33%。 两路并发总吞吐从 42.5 tok/s 提升到 63.7 tok/s，快了约 50%。 这次最关键的改进是 Cooperative MoE 内核。 MoE 模型生成每个 token 时，只会调用少量专家。单路或低并发解码时，分到每个专家的 token 很少，GPU 需要执行大量很小的矩阵运算。计算本身不大，Kernel 启动、专家分组和数据搬运的开销反而很明显。 新的 Cooperative MoE 内核针对 DeepSeek V4.1 的固定专家形状，把这些零散的小任务集中处理，让多个 CUDA 线程块协作完成专家计算，减少频繁调度和中间数据读写。 模型权重没有改变，仍然是双 Spark 上的 EXL3 2.9 bpw…
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, context length, hardware state, measurement method).
Tesla PG500-216 × Qwen3.8-27B
Sep 16, 2026 · by @UtaAoya
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| 16384 | — | vllm 1.5.0 | 24.4 tok/s | 620.4 tok/s |
| 16384 | — | vllm 1.5.0 | 33.8 tok/s | 620.4 tok/s |
| 16384 | — | vllm 1.5.0 | 38.5 tok/s | 621.1 tok/s |
Local LLM - V100 (の別SKU: PG500-216) ”評価0ショップから届いたのはV100の派生、別SKU: PG500-216(違いは図を参照)”でした。。 EVO-X2 + OCulink経由 PG500-216 x 1 で最低限の動作テストが終わったのでシェアします😌 暑いの嫌いなので150W縛り。暑くても平気な人はもっと高速化してくださいな〜 チャッピー解説 EVO-X2 128GBにTesla PG500-216 32GBをOCuLink接続し、Qwen3.8-27Bを1Cat-vLLM 1.5.0で検証しました。 PG500-216は標準V100よりSM数は少ない一方、HBM2帯域は約1.13TB/s。実測readも 1070.54GB/s を確認。 16K実コードでは MTP OFF 24.36 tok/s MTP1 33.77 tok/s MTP2 38.52 tok/s MTP2で…
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (quantization, measurement method).
M4 Max 128GB × Qwen3.8 Flash-Next mixed 4-8b
Sep 16, 2026 · by @ddalcu
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | — | mlx v26.9.3 | 80.0 tok/s | — |
| — | — | mlx v26.9.2 | 80.0 tok/s | — |
| — | — | mlx v26.9.1 | 68.0 tok/s | — |
MLX-Serve 26.9.3 is out. Fastest engine to run local models on a Mac: LLM's, Video, Image, Voice Clone, Music Gen. https://t.co/XSGv1OjOo7 Qwen Flash Next goes over 100 tok/s on M4/M5Max and insane prefill speeds. This release is huge, consider it "beta". Next release will be focused on polish and refactoring. Thanks: @Beamsters1, @TheDavidTai, @Lojza3D, ViRb3, d-b, perretv, @Damn_Lino, Nikolai Vysotskyi, t2tx…
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (context length).
RTX 4060 × Qwen 3.8 27B
Sep 16, 2026 · by @cyrilXBT
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| 65536 | IQ4_XS | unsloth | — | 150.0 tok/s |
| 65536 | IQ4_XS | unsloth | 5.0 tok/s | — |
UNREAL. Qwen 3.8 27B is now cruising locally on an RTX 4060 with a mere 8GB of VRAM. That’s a 64k context window, courtesy of Unsloth’s brand-new IQ4_XS quant, and the whole thing lands at just 14.6GB on disk. Prefill clocks in around 150 tok/s, while decode hovers near ~5 tok/s thanks to native MTP. To stay under the 8GB cap, you only have to offload 25 layers—no spills, no headaches. The quantized KV cache…
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, measurement method).
RTX 4060 × Qwen3.8-27B
Sep 16, 2026 · by @Oluwaphilemon1
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| 64K | IQ4_XS | llama.cpp | — | 150.0 tok/s |
| 64K | IQ4_XS | llama.cpp | 5.0 tok/s | — |
Qwen3.8-27B on an 8GB RTX 4060 gaming laptop sounds like a memory error waiting to happen. But it actually works. The machine: • RTX 4060 with 8GB VRAM • Intel i7-12700H • 16GB dual-channel DDR4 • PCIe 4 • Qwen3.8-27B Q4 And the target isn’t some tiny context window. It’s 64,000 tokens. That’s the minimum context used by the Hermes agent setup being tested. The trick is that the entire 27B model doesn’t need to live…
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, measurement method).
DGX Spark × qwen
Sep 16, 2026 · by @sudoingX · third-party report
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| 256K | fp8 | — | 45.0 tok/s | — |
qwen 3.8 flash next built this landing page in 29 minutes and served it on my tailnet on its own. i am running official fp8 on 2x dgx spark, full 256k context loaded, 45 tok/s with mtp on. i have run deepseek and glm on these boxes and it was good, this one just feels right. qwen 3.8 flash next is multimodal native, a vision encoder in the same weights, so the serve that wrote this page can read the screenshot it…
Digest-only: confidence 0.35 below 0.60; incomplete methodology (engine+version, hardware state, measurement method).
RTX 4090 × Qwen 3.8 27B
Sep 16, 2026 · by @0x0SojalSec
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | — | — | 141.0 tok/s | — |
| 38K | — | — | 130.0 tok/s | — |
A 4-year-old RTX 4090 Single card just hit 140 tok/s on Qwen 3.8 27B. - 141-tok/s single-stream on real agent workloads - 95.5% GSM8K - The optimizations (speculative decoding & requantized int4 output head, lookup drafting) compound. Same recipe that got 133 on a 3090 now runs even faster on the extra bandwidth. Real agent turns with tools and 38k context still stay above 130. Hardware from 2022 is not finished yet.
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, quantization, hardware state, measurement method).
RTX 4090D x200 × Qwen 3.8 27b
Sep 15, 2026 · by @qtum
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| 262K | FP16 | deepseek harness | 60.0 tok/s | — |
Here's Qwen 3.8 27b in action using the DeepSeek harness. This is self-hosted from our facility. 55-65 tokens per second. Text go woosh. This is FP16 precision 262k context vision enabled. We will be adding more open weight models soon, next up will be Qwen 3.8 Flash Next hosted on our own hardware. We're using a few hundred RTX 4090D 48gb cards to make this work. The price for this model is $1.18 per 1m tokens…
Digest-only: confidence 0.40 below 0.60; incomplete methodology (engine+version, hardware state, measurement method).
MacBook × Qwen 3.8 Flash Next
Sep 15, 2026 · by @mfranz_on
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | — | — | 6.1 tok/s | — |
| — | — | — | — | 10.0 tok/s |
Qwen 3.8 Flash Next at 6.1 tok/s on my 24GB Macbook. To be honest what makes me happier that the decode is the prefill speed, from 2 tok/s to 10 tok/s. I made a 5x in prefill and a 2x in decode. It is still no usable but we are making it! https://t.co/fsDG5yoaSs
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, quantization, context length, hardware state, measurement method).
RTX 4070 × Qwen3.8-Flash-Next
Sep 15, 2026 · by @ai_hakase_ · third-party report
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | — | — | 6.0 tok/s | — |
| — | — | — | 20.0 tok/s | — |
【RTX 4070でQwen3.8-Flash-Nextを爆速化する技術！】 中位スペックの環境（RTX 4070 12GB ＋ 64GB DDR5）で、超巨大なMoEモデル「Qwen3.8-Flash-Next」をローカルで動かすア熱いテクニックが話題になっていますよ！✨ そのままじゃとてもVRAMに乗り切らない巨大モデルですが、AtomicChatによる「AD-4.27bpw」などの量子化や、Gen4 NVMe SSDへ巨大なn-gramテーブルを「Lazy mmap」でオフロードする技を組み合わせることで、省メモリと高速化を見事に両立しているそうです！🚀 さらに、MTP（Multi-Token Prediction）パッチの適用によって、生成速度が初期の約6 tok/sから一気に約20…
Digest-only: confidence 0.35 below 0.60; incomplete methodology (engine+version, quantization, context length, hardware state, measurement method).
DGX Spark × DeepSeek-V4.1-Flash
Sep 15, 2026 · by @yume_arasaki
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| 35K | 3-bit codebook (profiled, stream | custom profiled inference | 36.6 tok/s | 337.0 tok/s |
| 35K | EXL3 2.9bpw | sglang | 48.5 tok/s | 787.0 tok/s |
| — | MXFP4/FP8 | sglang | 37.9 tok/s | — |
| — | MXFP4 | native | 52.0 tok/s | — |
| — | EXL3 3.5bpw | custom | 73.8 tok/s | — |
| — | — | custom | 61.1 tok/s | — |
| — | — | custom | 178.7 tok/s | — |
| — | — | custom | 190.0 tok/s | — |
| — | — | custom | 92.7 tok/s | — |
| — | — | custom | 131.0 tok/s | — |
| — | — | custom | 86.0 tok/s | — |
| — | — | custom | 495.0 tok/s | — |
DeepSeek shipped V4.1 Flash on September 10. By the next morning the local scene had it figured out: 510GB of weights, 3 to 4 DGX Sparks, nothing less. Nobody runs this below four boxes. Five days later, one builder is running it on a RTX 5090... I tracked every measured V4.1 Flash run up to this date so you don't have to. Here is the map as of today, every number sourced in the reply. From this point on remember…
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version).
Radeon AI PRO R9700 x2 × Qwen3.8-27B
Sep 15, 2026 · by @bkuyper
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| 262144 | NVFP4 | vllm 0.9.3 | — | 5809.0 tok/s |
| 262144 | NVFP4 | vllm 0.9.3 | 276.0 tok/s | — |
NVFP4 Qwen3.8 27b - AMD R9700's running @ 5,809 tok/s Prefill and 276 tok/s decode. This leverages the MXFP4 fast path I built on vLLM Radiance. https://github.com/GGZ14/vllm-mxfp4 #AMDAI @AIatAMD #localLLM #amd https://t.co/g44w9XEbwr
Digest-only: incomplete methodology (measurement method).
Verification status
This digest is an automated transcription of public posts. Runs with a stated engine build, quantization, context length, hardware state and measurement method are published straight to the benchmark board with a link to their source; the rest stay here. These are author-reported numbers, not an independent DiggTop re-measurement: every row carries its provenance so you can check the original, and numbers that fail review are removed. Our own testing follows the methodology. Spotted an error? Reply on the source post or submit a corrected run at /submit.
Sources
- @wei_wang — Sep 16, 2026
- @UtaAoya — Sep 16, 2026
- @ddalcu — Sep 16, 2026
- @cyrilXBT — Sep 16, 2026
- @Oluwaphilemon1 — Sep 16, 2026
- @sudoingX — Sep 16, 2026
- @0x0SojalSec — Sep 16, 2026
- @qtum — Sep 15, 2026
- @mfranz_on — Sep 15, 2026
- @ai_hakase_ — Sep 15, 2026
- @yume_arasaki — Sep 15, 2026
- @bkuyper — Sep 15, 2026
FAQ
Are the numbers in this digest verified?
They are transcribed from public X posts and the repositories those posts link to, with attribution and a source link on every entry. Rows that state the full methodology are published to the benchmark board automatically, so treat them as author-reported until a dispute is raised: the source link on every row is the check.
What does it take for a run to reach the verified board?
Engine and exact version, quantization, context length, hardware state and a described measurement method. Author-reported posts that state all five are queued as pending and can be promoted by a reviewer; anything missing a condition stays out of the database.
Where does the data come from?
Public X posts about local-model inference measured on owned hardware, discovered by keyword search every two hours and linked back to the original thread. The digest is regenerated daily.