X local-model bench digest — Oct 27, 2026
As of Oct 27, 2026 — 34 posts collected from X. 97 measured runs catalogued, 34 kept to the digest only. Latest source post Oct 27, 2026 08:58 UTC.
Every number below is what its author published, transcribed with the engine, quantization and context length they reported. Sources are linked on each entry. Only posts that state the full methodology (engine build, quantization, context length, hardware state, measurement method) reach the benchmark board; the rest stay here as leads.
DGX Spark x8 × GLM-5.3
Oct 27, 2026 · by @jaiball7
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| 1M | — | r3-102 | 47.1 tok/s | 1370.0 tok/s |
| 1M | — | r3-102 | 37.8 tok/s | 930.0 tok/s |
GLM-5.3（755B）を NVIDIA DGX Spark で動かすレシピ、r3-102 を公開しました。 ・TP=8:kv 1M token x 2, decode 47.1 tok/s, prefill 1,370 tok/s ・TP=4:kv 1M token, decode 37.8 tok/s, prefill 930 tok/s ・Expert Atlas:19,200のエキスパートを使われ方に応じて 2〜5bit で載せ分け、動いているエンジンのままエキスパートを最適化できます。同じメモリのまま、よく選ばれるエキスパートほど多くのビットになります。 速さや精度などの細かい数字は GitHub にまとめています。必要な重み（Hugging Face）の一覧と、重みを https://t.co/k8rP9NhMbk 公式の BF16 から自分で作る手順もあります。 https://t.co/Ba1Y5uxMQZ
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (hardware state, measurement method).
RTX 5070 × DeepSeek V4.1 Flash
Oct 27, 2026 · by @VeknQAQ
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | — | — | 4.7 tok/s | — |
763B 的 DeepSeek V4.1 Flash 写着 12GB 显卡就能跑。 显卡是 12GB。 内存要 46 到 64GB，硬盘 257GB。 RTX 5070 上测出来 4.7 tok/s。 一秒出四个多字 😂😂😂这能干啥 cos gpt 6.1sol吗？？
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, quantization, context length, hardware state, measurement method).
RTX 3090 × Qwen3.8-27B
Oct 27, 2026 · by @Oluwaphilemon1 · third-party report
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | — | hyperqwen | — | 250.0 tok/s |
| — | — | llama.cpp | — | 700.0 tok/s |
Qwen3.8-27B spent 21 days trying to build a CUDA inference engine that could outperform llama.cpp on a single RTX 3090. It failed to beat the established runtime, but the experiment reveals some interesting challenges in long-running AI engineering. The project statistics: 21 days of development 180 subagents 230 million tokens processed 1.7 billion cache-read tokens 699 context compactions 83 hours spent on…
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, quantization, context length, hardware state, measurement method).
RTX 4060 Ti 16GB × Strata
Oct 27, 2026 · by @UtaAoya
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| 128K | — | — | — | 1013.0 tok/s |
Local LLM - Strataの傾向 (RTX4070S/4060Ti/3060) まじかー！自分の環境で驚きw 同一環境のグラボ比較でRTX4060Ti が1番になるとは。。 StrataはVRAM容量が重要というのが裏が取れた感じ😅 チャッピー解説 ミニPCでも128KのローカルAI Agentは実用的！ MINISFORUM M1-1390＋DDR4 64GB＋OCuLink eGPUで、GPUだけを交換してStrataを比較しました。 ✅ 3GPUとも100W制限で128K推論に成功 🚀 RTX 4060 Ti 16GB：長Context Prefill最速（1,013 tok/s） ⚡ RTX 4070 SUPER 12GB：Decode最速 💡 RTX 3060 12GB：旧世代でも128K完走 特に驚いたのは、4060 Tiが128K Prefillで4070 SUPERより約39%高速だったこと。 AI…
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, quantization).
M5 Ultra 256GB × DeepSeek-V4.1-Flash-1017
Oct 27, 2026 · by @ksuniri
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | — | tensorfold (deepseek-v41-main br 8557236a | 46.3 tok/s | — |
| — | — | tensorfold (deepseek-v41-main br 8557236a | 50.0 tok/s | — |
Development Environment ##### Spec : M5 Ultra 256g / 4TB Recipe : @BobClawblaw , https://github.com/BobClawblaw/deepseek-v41-flash-mlx-m5-ultra Model : DeepSeek-V4.1-Flash-1017 https://huggingface.co/BobClawblaw/DeepSeek-V4.1-Flash-1017-TensorFold-e2e3-p4-h8 CTX : 1M Decode 1M 장문 -> 46.3 tok/s , Normal Prose -> 50 tok/s 결론 : M5 Ultra 512g 도 소유하고 싶다^^ https://t.co/Uoc4PvRezJ
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (quantization, measurement method).
DGX Spark x2 × MiniMax 3.1 Flash
Oct 27, 2026 · by @jayleaton
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| 32768 | — | tensorfold | — | 720.0 tok/s |
| 2048 | — | tensorfold | — | 786.0 tok/s |
| — | — | tensorfold | 21.8 tok/s | — |
| — | — | tensorfold | 21.3 tok/s | — |
Final @MiniMax_AI 3.1 Flash update for DGX Spark TP=2 on @tensorfold_dev Prompt processing: 32K 580 → 720 2K 692 → 786 Split decode attention: The kernel alone went from 406 → 41 µs per layer in a microbenchmark. 19.3 / 18.2 → 21.8 / 21.3 tok/s on a long-context bench. This model config is not really suited for the low bandwidth of the DGX Spark. This work will go into upstream TensorFold, and then I will work on a…
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, quantization, hardware state, measurement method).
RTX 5090 × Qwen3.8-Flash-Next
Oct 27, 2026 · by @Smlay_ero
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | Q2_0 | strata | 229.0 tok/s | — |
| 4K | Q2_0 | strata | 221.0 tok/s | — |
| 16K | Q2_0 | strata | 218.0 tok/s | — |
| 16K | Q2_0 | strata | — | 6436.0 tok/s |
I’ve been testing the https://t.co/WrbDh4rCJi project with a fresh config, running Qwen3.8-Flash-Next at Q2_0. Shoutout to @coldniko — incredibly creative work. Just a year ago, you wouldn’t dare expect speeds like this on a 12 GB card. Here are my benchmark results. 229 tok/s median decode on the standard-v1 suite (short context), against 87-93 tok/s on the official RTX 5070 12 GB reference: about 2.5x. Hardware…
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version).
DGX Spark × Qwen3.8-27B
Oct 27, 2026 · by @wei_wang
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| 2K | — | tensorfold 1.1.4 | — | 1281.0 tok/s |
| 8K | — | tensorfold 1.1.4 | — | 1266.0 tok/s |
| 32K | — | tensorfold 1.1.4 | — | 1165.0 tok/s |
| 64K | — | tensorfold 1.1.4 | — | 1045.0 tok/s |
同一台 DGX Spark，长 prompt 的 prefill 速度，换个配方能差 5 倍。 TensorFold 1.1.4 只改了 Qwen3.8-27B 在 GB10 上的默认配方，冷启动 prefill 越长提速越多： 1. 2k：834 → 1,281 tok/s 2. 8k：612 → 1,266 3. 32k：326 → 1,165 4. 64k：203 → 1,045 粗算一下，64k 的 prompt 以前要读 5 分多钟，现在 1 分钟出头。decode 只快了 2% 左右，硬件带宽决定的那部分没变。 跑 agent 每轮都要读几万 token 的上下文，这一步比 decode 更影响体感。 所以拿 Spark 和别的机器比长上下文速度，先看用的是哪个引擎、哪个版本。
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (quantization, hardware state, measurement method).
DGX Spark x4 × DeepSeek-V4.1-Flash-1017
Oct 26, 2026 · by @Tech2Wild
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | EXL3 4.0 bpw | tensorfold 0a0b69a | 97.2 tok/s | — |
| — | EXL3 4.0 bpw | tensorfold 0a0b69a | 143.7 tok/s | — |
| — | EXL3 4.0 bpw | tensorfold 0a0b69a | 230.7 tok/s | — |
DeepSeek V4.1 Flash on 4x DGX Spark, now on @bertholomusai's TensorFold v0.3.5 recipe + his 4.0bpw pack: • 29.4M-token KV pool (was 2M) • 97 tok/s at 1 user, 144 on code, 231 at 16 users • 93K prompt read in 37s, ~2s replies during long reads https://github.com/tonyd2wild/DeepSeek-V4.1-Flash-TP2-and-TP4-DGX-Sparks
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (measurement method).
RTX 4090 × Qwen3.8-27B
Oct 26, 2026 · by @QwenAIFans · third-party report
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| 131K | f16 KV cache (Ollama default) | ollama | 16.0 tok/s | — |
| 131K | q4_0 KV cache | ollama | 102.0 tok/s | — |
| 262K | q4_0 KV cache | ollama | 21.0 tok/s | — |
💥 Your Qwen3.8-27B "fits" 131K on a 4090… then crawls at 16 tok/s. Ollama's default KV is f16. At 131K it spills ~30% to CPU and speed falls off a cliff. Two lines in a systemd override: OLLAMA_FLASH_ATTENTION=1 OLLAMA_KV_CACHE_TYPE=q4_0 Restart, same 131K stays 100% GPU at ~102 tok/s (measured 512-token runs, MTP on). Catch: even q4_0 still spills a bit at 262K (~21 tok/s). 131K is the sweet spot on 24 GB. via…
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version).
RTX 5070 × DeepSeek-V4.1-Flash-1017
Oct 26, 2026 · by @wei_wang
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | — | — | 4.7 tok/s | — |
| — | — | — | — | 121.0 tok/s |
| — | — | — | 8.0 tok/s | — |
| — | — | — | 48.0 tok/s | — |
| — | — | — | 80.0 tok/s | — |
763B 的 DeepSeek-V4.1-Flash-1017，现在一张 12GB 显卡就能跑起来。靠的是两件事：每个张量单独挑位宽，再把模型拆到显存、内存、SSD 三层。 12GB 只是显存的门槛。实际还要 46 到 64GB 内存、257GB 硬盘，RTX 5070 上回答速度大约 4.7 tok/s。 先说直觉。 普通的 2 bit 量化是一刀切，所有张量压到同一个位宽。GSQ-RCO 先定一个总预算，比如平均 2.00 bpw，再让搜索决定每个张量分多少。 对误差敏感的多给，不敏感的少给，总大小正好卡在预算上。 2.00 版的分配表是公开的。40 层路由专家，每层 gate、up、down 三块，一共 120 个张量，位宽差得很开： 1. 60 个 IQ1_M，42 个 IQ2_XXS，18 个 IQ2_XS 2. 第 0 到 6 层的专家全是最低的 IQ1_M 3. 第 39 层三块都是 IQ2_XS，全表最高 4…
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, context length, measurement method).
RTX 5070 12GB + RTX 5060 Ti 16GB
Oct 26, 2026 · by @murasametech
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| 14K | IQ3_XXS | strata v0.2.6 | — | 487.5 tok/s |
| 14K | IQ3_S | strata v0.2.6 | — | 576.9 tok/s |
同じRAMの枠でも、モデルが少し大きいだけで、長文1件あたりのSSDの読み込み量が2桁変わりました。 構成はRTX 5070 12GB×1＋RTX 5060 Ti 16GB×1（計2枚）、RAM 64GB、Strata v0.2.6で、expertをRAMに置く上限は33GiBです。 ・IQ3_XXS：GPUに載らないexpertは24.66GiBで、33GiBの枠に全部収まりました。SSD読込は4.6〜34.6MB ・IQ3_S：GPUに載らないexpertは32.13GiB。処理中に借りる分まで含めると枠に入りきらず、エンジンからも「RAMに収まらない分はファイルから読む」という警告が出ました。SSD読込は660〜3,226MB （SSD読込は入力の長さごとの中央値） Prefillの差（約1.4万tokensで487.5と576.9…
Digest-only: model.name missing; incomplete methodology (measurement method).
Also in this window
The 12 most recent posts are shown in full above; 22 posts from the same window are listed here in short form, each with its source post.
- Unspecified platform × DeepSeek V4.1 Flash — 943.0 tok/s decode (2 measurements) · @sgl_project
- RX 9070 16GB (eGPU over Thunderbolt) × Qwen2.5-7B — 33.0 tok/s decode (4 measurements) · @JoshuaSWarren
- DGX Spark x4 × DeepSeek-V4.1-Flash-1017 — 157.0 tok/s decode (7 measurements) · @FT_IOxCS
- P100/V100 server — 20.0 tok/s decode (1 measurement) · @iamMrDuncan
- RTX 4090 × Qwen3.8-27B — 2185.0 tok/s prefill (2 measurements) · @QwenAIFans
- RTX 3080 — 2000.0 tok/s decode (1 measurement) · @BitsBeTrippin
- DGX Spark × nanbeige4.3-33b — 213.0 tok/s decode (3 measurements) · @filicroval
- RTX 5090 × Qwen3.8-27B GGUF — 136.7 tok/s decode (3 measurements) · @QwenAIFans
- DGX Spark x4 × DeepSeek-V4.1-Flash-1017 — 165.3 tok/s decode (5 measurements) · @niklaslenz_ai
- Unspecified platform × Nanbeige4.3-33B — 13.0 tok/s decode (2 measurements) · @Raullen
- Ryzen AI Max+ 395 (Strix Halo) + RX 7900 XTX x2 × DeepSeek-V4.1-Flash-1017 Maya-S — 41.3 tok/s decode (2 measurements) · @jgattermayer
- Unspecified platform × DeepSeek-v4.1-flash-1017 — 750.0 tok/s decode (1 measurement) · @0xSero
- RTX 4090 48GB × Nanbeige4.3-33B — 45.0 tok/s decode (2 measurements) · @MinLiBuilds
- DGX Spark x2 × Deepseek v4.1 Flash 1017 — 49.0 tok/s decode (1 measurement) · @nunomo1
- Mac mini × Qwen3-Next 80B — 350.0 tok/s decode (2 measurements) · @pratikg
- Unspecified platform × Qwen3.8-flash-next — 50.0 tok/s decode (2 measurements) · @ItsmeAjayKV
- RTX 3090 x4 × Qwen 3.8 27B — 2020.0 tok/s prefill (2 measurements) · @gospaceport
- DGX Spark x4 × DeepSeek-V4.1-Flash-1017 — 141.2 tok/s decode (6 measurements) · @niklaslenz_ai
- DGX Spark × Qwen3.8-Flash-Next — 68.5 tok/s decode (7 measurements) · @wei_wang
- CMP 170HX × Qwen 3.8 27B — 33.0 tok/s decode (2 measurements) · @VRAMCalculator
- DGX Spark x4 × DeepSeek-V4.1-Flash-1017 — 165.3 tok/s decode (5 measurements) · @niklaslenz_ai
- M5 Ultra Spark × Nanbeige4.3-33B — 41.0 tok/s decode (2 measurements) · @jtdavies
How to read this page
This digest is an automated transcription of public posts. Runs with a stated engine build, quantization, context length, hardware state and measurement method are published straight to the benchmark board with a link to their source; the rest stay here. These are author-reported numbers, not an independent DiggTop re-measurement: every row carries its provenance so you can check the original, and a number that turns out to be misreported is corrected or removed. Our own testing follows the methodology. Spotted an error? Reply on the source post or submit a corrected run at /submit.
Sources
- @jaiball7 — Oct 27, 2026
- @VeknQAQ — Oct 27, 2026
- @Oluwaphilemon1 — Oct 27, 2026
- @UtaAoya — Oct 27, 2026
- @ksuniri — Oct 27, 2026
- @jayleaton — Oct 27, 2026
- @Smlay_ero — Oct 27, 2026
- @wei_wang — Oct 27, 2026
- @Tech2Wild — Oct 26, 2026
- @QwenAIFans — Oct 26, 2026
- @wei_wang — Oct 26, 2026
- @murasametech — Oct 26, 2026
- @sgl_project — Oct 26, 2026
- @JoshuaSWarren — Oct 26, 2026
- @FT_IOxCS — Oct 26, 2026
- @iamMrDuncan — Oct 26, 2026
- @QwenAIFans — Oct 26, 2026
- @BitsBeTrippin — Oct 26, 2026
- @filicroval — Oct 26, 2026
- @QwenAIFans — Oct 26, 2026
- @niklaslenz_ai — Oct 26, 2026
- @Raullen — Oct 26, 2026
- @jgattermayer — Oct 26, 2026
- @0xSero — Oct 26, 2026
- @MinLiBuilds — Oct 26, 2026
- @nunomo1 — Oct 26, 2026
- @pratikg — Oct 26, 2026
- @ItsmeAjayKV — Oct 26, 2026
- @gospaceport — Oct 26, 2026
- @niklaslenz_ai — Oct 26, 2026
- @wei_wang — Oct 26, 2026
- @VRAMCalculator — Oct 26, 2026
- @niklaslenz_ai — Oct 26, 2026
- @jtdavies — Oct 26, 2026
FAQ
Are the numbers in this digest sourced?
They are transcribed from public X posts and the repositories those posts link to, with attribution and a source link on every entry. Rows that state the full methodology are published to the benchmark board automatically, so treat them as author-reported until a dispute is raised: the source link on every row is the check.
What does it take for a row to reach the sourced tier?
Engine and exact version, quantization, context length, hardware state and a described measurement method. Posts that state all five are published to the benchmark board as soon as the collector reads them, each row carrying its source link — nothing waits in a human review queue. Anything missing a condition stays in this digest.
Where does the data come from?
Public X posts about local-model inference measured on owned hardware, discovered by keyword search every two hours and linked back to the original thread. The digest is regenerated daily.