X local-model bench digest — Oct 29, 2026
As of Oct 29, 2026 — 31 posts collected from X. 90 measured runs catalogued, 31 kept to the digest only. Latest source post Oct 29, 2026 10:39 UTC.
Every number below is what its author published, transcribed with the engine, quantization and context length they reported. Sources are linked on each entry. Only posts that state the full methodology (engine build, quantization, context length, hardware state, measurement method) reach the benchmark board; the rest stay here as leads.
RTX 5090 × QwenFlashNext
Oct 29, 2026 · by @orenoa7
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | IQ1_M | — | — | 5562.0 tok/s |
| — | IQ1_M | — | 161.0 tok/s | — |
| — | IQ1_M | — | 199.4 tok/s | — |
| — | Q2_0 | — | — | 6123.0 tok/s |
| — | Q2_0 | — | 211.5 tok/s | — |
| — | Q2_0 | — | 296.5 tok/s | — |
| — | IQ2_XS | — | — | 5465.0 tok/s |
| — | IQ2_XS | — | 212.9 tok/s | — |
| — | IQ2_XS | — | 284.2 tok/s | — |
| — | IQ3_XXS | — | — | 5196.0 tok/s |
| — | IQ3_XXS | — | 182.0 tok/s | — |
| — | IQ3_XXS | — | 221.9 tok/s | — |
Strata + RTX5090 + DDR5-3600 128GB QwenFlashNextの量子化６種で実測 ■ Prefill｜Prose｜Code (tok/s) の実測は？ ・IQ1_M 5562｜161.0｜199.4 ・Q2_0 6123｜211.5｜296.5 ・IQ2_XS 5465｜212.9｜284.2 ・IQ3_XXS 5196｜182.0｜221.9 ・IQ3_S 5407｜163.2｜178.5 ・UD-IQ4_XS 4225｜96.5｜106.6 Q8_0とのTop-1一致率を考えると、IQ3_Sがバランス良さそう ■ Prefillは入力長で変化するか？ （２枚目）8k~127kまでPrefill速度は大きな変化なし ■ 並列性能は？ （３枚目）Proseは８並列で1.3〜1.5倍に合計デコード速度が伸びる…
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, context length, hardware state, measurement method).
DGX Spark × Ling-3.0-flash
Oct 29, 2026 · by @NFT_Chen · third-party report
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | MXFP4 | sglang | 37.8 tok/s | — |
| — | MXFP4 | sglang | 53.9 tok/s | — |
| — | MXFP4 | sglang | 102.9 tok/s | — |
💥厉害！单台 DGX Spark 把 Ling-3.0-flash 解码速度干到 53.89 tok/s！+42.5% 吞吐，延迟直降 24.5%，准确率几乎零损失！ 蚂蚁百灵 × NVIDIA 联合优化，三项黑科技叠满： 🔹Humming MXFP4 MoE 内核 🔹在线 FP8 LM-head 量化 🔹NEXTN MTP 投机解码 GSM8K 全量实测：37.81→53.89 tok/s，合成负载最高 102.88 tok/s。 上手极简： 1️⃣拉 SGLang + pip install humming-kernels 2️⃣设环境变量 SGLANG_ENABLE_FP8_LM_HEAD=1 3️⃣开启 NEXTN MTP（注意 LM-head 模块共享） 4️⃣跑官方 notebook：dgx-spark-sglang-ling-3-flash-mxfp4-humming.ipynb 124B…
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, context length, hardware state).
DGX Spark x4 × Qwen3.8-Flash-Next
Oct 29, 2026 · by @Isichan_Hitori
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | — | — | 57.0 tok/s | 3350.0 tok/s |
| — | — | — | 67.0 tok/s | 1675.0 tok/s |
| — | — | — | 38.7 tok/s | 1098.0 tok/s |
| 6500 | — | — | 75.5 tok/s | 4465.0 tok/s |
| 6500 | — | — | 66.0 tok/s | 2887.0 tok/s |
| 6500 | — | — | 35.0 tok/s | 1440.0 tok/s |
| — | — | — | 290.0 tok/s | — |
| — | — | — | 132.0 tok/s | — |
| — | — | — | 46.0 tok/s | — |
| — | — | unsloth | 6970.0 tok/s | — |
| — | — | mlx | 33.3 tok/s | — |
【 M5U 512GBが400万円。本当にいいの？ 】 ～DGXとMac Studio 用途別の比較～ ・DGX Spark 128GB×4台 ・M5Ultra 256GB×2台 ・M5Ultra 512GB×1台 合計メモリは512GBにそろえて10個の用途で比較 ✅先に結論 巨大なモデルを速く → M5 Ultra 256GB×2 AI作業・何本も回す・CUDA → DGX Spark×4 1台で済ませたい → M5 Ultra 512GB×1 ✅なぜ2台の方が速いのか Apple公式（10/15）式の測定 ・モデルを2台で順番に処理→速さは1台のときと同じ （Qwen3.8 27Bで、1台 19.69 → 2台 19.44） ・モデルを2台で同時に処理→1.7倍速い （同じモデルで 19.69 → 33.27） ━━━━━━━━━━ ✅LLMの用途 ※直接比較がない場合は推定値になります。 ～～①1個のLLMを使用～～…
Digest-only: confidence 0.55 below 0.60; incomplete methodology (engine+version, quantization, hardware state, measurement method).
DGX Spark x2 × DeepSeek V4.1 Flash
Oct 29, 2026 · by @ivanfioravanti · third-party report
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | — | — | 77.0 tok/s | — |
| — | — | — | 105.0 tok/s | — |
| — | — | — | 150.0 tok/s | — |
| — | — | — | 194.0 tok/s | — |
| — | — | — | 63.0 tok/s | — |
| — | — | — | 78.0 tok/s | — |
| — | — | — | 98.0 tok/s | — |
| — | — | — | 99.0 tok/s | — |
| — | — | — | 27.0 tok/s | — |
| — | — | — | 51.0 tok/s | — |
The question that is stopping me from buying an M5 Ultra: why does a Mac with 1.2 TB/s of bandwidth lose to DGX Spark at long context and with multiple users? With one user, decode is memory-bound. Every token means reading all the active weights once, and the GPU mostly waits for data. More bandwidth means more tok/s, and that's where the Mac shines. 👍 Batching changes everything. With 8 users you read the weights…
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, quantization, context length, hardware state, measurement method).
DGX Spark x4 × DeepSeek V4.1 Flash 1017
Oct 29, 2026 · by @masa_ai_labo
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| 256K | — | vllm | 70.3 tok/s | — |
| 256K | — | sglang | — | 6180.0 tok/s |
前回の3レシピ比較の続き。今回はCRS804を使ったスイッチあり構成で再比較してみた。さらに、気になっていたspark-ds41fも追加して、4レシピで比較。 DeepSeek V4.1 Flash 1017の公式重みを使用。DGX Spark×4／TP4／200G接続。256K入力・同時1件・GPUクロック上限2300MHzで、SparkDashを使って各3回測定しました。 今回の条件では、decodeはspark-ds41fが平均70.29 tok/sでトップ。キャッシュなしのprefillはknapcioが約6,180 tok/sで最速でした。 個人的には、選ぶならこの2つかなという感想です。spark-ds41fはdecodeもprefillも速く、普段はこちらをメインに使っていこうと思います。 1枚目がdecode・prefillの平均値、2枚目がTTFTの中央値と測定条件です。…
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, quantization).
DGX Spark × Ling-3.0-flash
Oct 29, 2026 · by @AntLingAGI
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | — | — | 37.8 tok/s | — |
| — | — | — | 53.9 tok/s | — |
In technical collaboration with the @NVIDIAAI team, we optimized inference for the open-weight Ling-3.0-flash model on a single NVIDIA DGX Spark using three techniques. Across 1,319 GSM8K questions, decode throughput increased 42.5%—from 37.81 to 53.89 tok/s—while mean latency fell 24.5%, from 15.21s to 11.49s, with no statistically detectable accuracy change. 🧵
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, quantization, context length, hardware state).
RTX 3060 × Qwen3.8-27B
Oct 29, 2026 · by @QwenAIFans · third-party report
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| 96K | Q4_K_S | llama.cpp | 9.7 tok/s | 225.0 tok/s |
💥12G 的 3060，硬塞千问3.8-27B，上下文还能开到 96K 社区实测：Q4_K_S，KV 用 q4_0，再把前 46 层 FFN 甩到内存。 显存峰值约 11GB 生成约 9.7 tok/s，提示处理约 225 tok/s MTP 接受率约 88% 关键大概是： --override-tensor "blk.([0-9]/[1-3][0-9]/4[0-5]).ffn_.*=CPU" 再加 -ctk q4_0 -ctv q4_0 新版 llama.cpp 也可以试 -ncffn 40 温馨提示：这是社区帖里的数，不是官方表。显卡还驱动桌面的话，可用显存会更紧。 你们 12G 卡最长开到多少上下文？ 来源：Hugging Face Unsloth 讨论👇 https://t.co/sFDo2Hk5mD
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, measurement method).
RTX 3060 12GB (OCuLink) + Ryzen 7 PRO 8845HS × Qwen3.8-Flash-Next Coder
Oct 29, 2026 · by @aliexp_store
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| 32768 | IQ1_M | strata v0.1.52 | 20.8 tok/s | 118.0 tok/s |
ローカルLLMを動かしてみた！第二弾 AMD Ryzen 7 PRO 8845HS（AVX-512対応） RAM31GB NVIDIA GeForce RTX 3060 12GB（OCuLink接続） Strata（v0.1.52、MIT ライセンス、llama.cpp ベース） Qwen3.8-Flash-Next の Coder 版（IQ1_M、合計125B／実働6Bの MoE） 設定コンテキスト 32K、KV キャッシュ 8-bit、MTP（先読み生成）あり 生成速度（平均）20.8 tok/s プロンプト読み込み（平均）118 tok/s プロンプト読み込み（範囲）57〜229 tok/s 1回の所要時間27.7〜61.7 秒 システムRAM30.7 / 31 GB VRAM10.9 / 12 GB GPU 温度44〜48°C 電力約18 W（待機時）
Digest-only: incomplete methodology (measurement method).
RTX Pro 6000 x4 × deepseek-v4.1-flash-1017
Oct 29, 2026 · by @0xSero · third-party report
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | — | — | 766.0 tok/s | — |
766 tok/s on deepseek-v4.1-flash-1017 concurrency 1 on 4x rtx pro 6000 These cards are going to the moon.. Credit: kacper-daftcode https://t.co/Q1Hq581HfN
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, quantization, context length, hardware state, measurement method).
RTX 3060 × LFM3-1.7B
Oct 29, 2026 · by @DogukanUrker
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| 16384 | Q8_0 | llama.cpp | 144.0 tok/s | — |
| 16384 | Q8_0 | llama.cpp | 382.0 tok/s | — |
running all 6 new LFM3 models from @liquidai on a single RTX 3060. 12GB VRAM. configs below, one model per post 🧵 1/ LFM3-1.7B 68 parallel slots, 16K context each. ~144 tok/s on one request, ~382 tok/s total with all 68 busy. 11.3GB VRAM. these are pipeline models (tagging, extraction, captions), not long-context agents, so i went with many short slots instead of one 256K slot. config: llama-server -m…
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, measurement method).
V100 × deepseek-v4.1-flash
Oct 29, 2026 · by @VectorCrossProd
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | IQ2_XXS | — | 15.0 tok/s | — |
llm-visuals now supports Strata's DeepSeek packs. This is deepseek-v4.1-flash at IQ2_XXS, 6 of 384 experts per token, decoding at about 15 tok/s on a 2016 Dell R730 with a V100. The V100 is full at 15.3 of 16 GB and 137 GB of system RAM is in use. Which model should I visualise https://t.co/MY9mjvdU22
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, context length, measurement method).
DGX Spark × Qwen3.8 Flash Next
Oct 29, 2026 · by @_nodelay · third-party report
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | — | — | 130.0 tok/s | — |
밤 사이에 mia ai lab 에서 올라온 dgx spark 한대에서 돌아가는 qwen3.8 flash next 가 미친 속도를 130 tok/s 가 나왔다. 한대만 있어도 이제 어지간한간 충분하다. 다른 한대는 멀티모달 처리를 하면 되겠다!!
Digest-only: confidence 0.55 below 0.60; incomplete methodology (engine+version, quantization, context length, hardware state, measurement method).
Also in this window
The 12 most recent posts are shown in full above; 19 posts from the same window are listed here in short form, each with its source post.
- RTX 5090 Laptop GPU × Qwen 3.6 27B — 35.3 tok/s decode (2 measurements) · @sudoingX
- RTX PRO 6000 — 3.0 tok/s decode (1 measurement) · @JoeGuglielmucci
- P100 × Qwen 3.8 27b — 30.0 tok/s decode (1 measurement) · @iamMrDuncan
- DGX Spark × Qwen3.8-Flash-Next — 133.0 tok/s decode (1 measurement) · @0xBakeer
- Mac mini M6 × Qwen3.6-35B-A3B-Splash — 234.1 tok/s decode (3 measurements) · @swarogan
- RTX 4080 SUPER × Qwen3.8-27B — 65.0 tok/s decode (2 measurements) · @QwenAIFans
- Unspecified platform × DeepSeek V4.1 Flash 1017 — 367.0 tok/s decode (1 measurement) · @BoundlessHQ
- RTX 3090 × Qwen3.8 — 42.9 tok/s decode (1 measurement) · @QwenAIFans
- DGX Spark x2 × GLM-5.3-Flash — 165.4 tok/s decode (12 measurements) · @yume_arasaki
- RTX 5090 × Qwen3.8 MTP — 107.6 tok/s decode (2 measurements) · @QwenAIFans
- DGX Spark × Qwen3.8-Flash-Next — 2269.0 tok/s prefill (2 measurements) · @jtdavies
- Unspecified platform × DeepSeek 4.1 Flash — 2300.0 tok/s decode (2 measurements) · @jiayq
- M3 Max — 140.0 tok/s decode (2 measurements) · @steeve
- Intel Arc Pro B70 x2 × Nanbeige4.3-33B — 41.0 tok/s decode (2 measurements) · @SergiiioBS
- 8GB GPU × Recursive Flint — 18.6 tok/s decode (2 measurements) · @GuideboardLabs
- M5 Max × MiniCPM-OCR 1.0 — 227.5 tok/s decode (1 measurement) · @OpenBMB
- RTX 3090 x2 × Qwen3.8-27B — 262.2 tok/s decode (3 measurements) · @geldeki
- iPhone 18 Pro Max × MiniCPM-V 4.7 — 20.5 tok/s decode (1 measurement) · @OpenBMB
- RTX 3090 x4 × DeepSeek V4.1 Flash Abliterated — 45.5 tok/s decode (2 measurements) · @superalesha
How to read this page
This digest is an automated transcription of public posts. Runs with a stated engine build, quantization, context length, hardware state and measurement method are published straight to the benchmark board with a link to their source; the rest stay here. These are author-reported numbers, not an independent DiggTop re-measurement: every row carries its provenance so you can check the original, and a number that turns out to be misreported is corrected or removed. Our own testing follows the methodology. Spotted an error? Reply on the source post or submit a corrected run at /submit.
Sources
- @orenoa7 — Oct 29, 2026
- @NFT_Chen — Oct 29, 2026
- @Isichan_Hitori — Oct 29, 2026
- @ivanfioravanti — Oct 29, 2026
- @masa_ai_labo — Oct 29, 2026
- @AntLingAGI — Oct 29, 2026
- @QwenAIFans — Oct 29, 2026
- @aliexp_store — Oct 29, 2026
- @0xSero — Oct 29, 2026
- @DogukanUrker — Oct 29, 2026
- @VectorCrossProd — Oct 29, 2026
- @_nodelay — Oct 29, 2026
- @sudoingX — Oct 29, 2026
- @JoeGuglielmucci — Oct 28, 2026
- @iamMrDuncan — Oct 28, 2026
- @0xBakeer — Oct 28, 2026
- @swarogan — Oct 28, 2026
- @QwenAIFans — Oct 28, 2026
- @BoundlessHQ — Oct 28, 2026
- @QwenAIFans — Oct 28, 2026
- @yume_arasaki — Oct 28, 2026
- @QwenAIFans — Oct 28, 2026
- @jtdavies — Oct 28, 2026
- @jiayq — Oct 28, 2026
- @steeve — Oct 28, 2026
- @SergiiioBS — Oct 28, 2026
- @GuideboardLabs — Oct 28, 2026
- @OpenBMB — Oct 28, 2026
- @geldeki — Oct 28, 2026
- @OpenBMB — Oct 28, 2026
- @superalesha — Oct 28, 2026
FAQ
Are the numbers in this digest sourced?
They are transcribed from public X posts and the repositories those posts link to, with attribution and a source link on every entry. Rows that state the full methodology are published to the benchmark board automatically, so treat them as author-reported until a dispute is raised: the source link on every row is the check.
What does it take for a row to reach the sourced tier?
Engine and exact version, quantization, context length, hardware state and a described measurement method. Posts that state all five are published to the benchmark board as soon as the collector reads them, each row carrying its source link — nothing waits in a human review queue. Anything missing a condition stays in this digest.
Where does the data come from?
Public X posts about local-model inference measured on owned hardware, discovered by keyword search every two hours and linked back to the original thread. The digest is regenerated daily.