X local-model bench digest — Oct 23, 2026
As of Oct 23, 2026 — 36 posts collected from X. 103 measured runs catalogued, 1 published to the board, 35 kept to the digest only. Latest source post Oct 23, 2026 10:34 UTC.
Every number below is what its author published, transcribed with the engine, quantization and context length they reported. Sources are linked on each entry. Only posts that state the full methodology (engine build, quantization, context length, hardware state, measurement method) reach the benchmark board; the rest stay here as leads.
NVIDIA Rubin NVL72 × GLM5.3 744B
Oct 23, 2026 · by @MLBear2 · third-party report
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | — | — | 862.0 tok/s | — |
【10月23日 夜のニュースまとめ】 ・RSNA膝コンペ終了、LLM擬似ラベルの品質が勝敗分ける ・Meta Museが日本で提供開始、1Password連携も展開へ ・Moonshot AIがKimi K3.1を公式予告、Arenaで試験か ・Grok Botがスマホのシステムアプリ操作に対応 ・Claude on Google Cloud（Anthropic管理）が一般提供開始 ・パーソナルAI Agent乱立、通知増幅と先行者有利の指摘 ・AIと回すバイブ形式手法、Z3/TLA+/Leanの使い分け ・中国当局、DeepSeekとMiniMaxのモデル海外展開を承認 ・Rubin NVL72、GLM5.3で1ユーザー862 tok/sを達成 ・Liquid AI、超低遅延推論エンジンRIPLを発表 ・Ramp、会計向けstackと汎用glassで全社Agent運用…
Digest-only: confidence 0.00 below 0.60; incomplete methodology (engine+version, quantization, context length, hardware state, measurement method).
RTX 5070 12GB × Qwen3.8-Flash-Next
Oct 23, 2026 · by @jun1228909
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | — | strata 0.1.48 | 45.0 tok/s | — |
もうリンクを貼るまでもなくインストール済みですよねってこと？！😂 https://t.co/69ZYfwAPh1 Strata v0.2.3 リリース 🚀 Strix Halo高速化 + Agent/Terminal安定化かな😃 改善続きでありがたや！ ・Strix Halo（Ryzen AI Max）を高速化 ・回答速度 +6〜19%、code +18%、tool call +19% ・prompt速度も +5〜14% ・Intel Arc用engineを正式release package化 ・Arc B-seriesも利用可能に ・DFlash2 drafterをTerminal限定で追加 ・Agent chatのcontext overflow対策を強化 ・/compact対応、過去会話の自動summarize改善 ・64K contextテストでprompt使用率83%→61% ・RTX 5070 12GBで31→45…
Digest-only: confidence 0.55 below 0.60; incomplete methodology (quantization, context length, hardware state, measurement method).
DGX Spark x4 × MiniMax-M3.1-Flash
Oct 23, 2026 · by @majewskizby
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| 262K | NVFP4 experts + MXFP8 weights | — | 39.7 tok/s | — |
| 262K | NVFP4 experts + MXFP8 weights | — | 62.2 tok/s | — |
| 262K | NVFP4 experts + MXFP8 weights | — | 166.0 tok/s | — |
| 262K | NVFP4 experts + MXFP8 weights | — | 261.0 tok/s | — |
| — | NVFP4 experts + MXFP8 weights | speedbench | 38.4 tok/s | — |
MiniMax-M3.1-Flash TP4 on 4× DGX Spark, day 0 🐇 • Prose decode: 39.7 tok/s at c1 • Structured decode: 62.2 tok/s at c1 • Aggregate: 166 tok/s prose, 261 tok/s structured at c16 • speedbench quick: 38.4 tok/s across 30 prompts • 262k context, ~1.37M-token FP8 KV cache • MiniMax's own NVFP4 experts + MXFP8 weights, bundled DSpark draft • Runs on GB10 (sm_121): FP8 KV, Triton sparse attention, Marlin experts (the NVFP4…
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version).
Fireworks cloud (unspecified) × DeepSeek V4.1 Flash
Oct 23, 2026 · by @dingyi · third-party report
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | — | — | 1521.0 tok/s | — |
隔壁家 Fireworks 也上线了 DeepSeek V4.1 Flash，1017 runs at 1,521+ tok/s 这是多么恐怖的速度！以后就应该用这些云服务平台的，官方的太拉胯了。
Digest-only: confidence 0.40 below 0.60; incomplete methodology (engine+version, quantization, context length, hardware state, measurement method).
RTX 3090 × DeepSeek-v4.1-flash
Oct 23, 2026 · by @ItsmeAjayKV
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| 192 | — | project maya v1.1.24 | — | 778.0 tok/s |
| 131072 | — | project maya v1.1.24 | 5.0 tok/s | — |
| 131072 | — | project maya v1.1.24 | 16.7 tok/s | — |
DeepSeek-v4.1-flash is now faster on my 3090 ! Tested new DeepSeek-v4.1-flash Maya-XS quant with Project-Maya System Specs: - Single RTX 3090 - 64GB DDR5 RAM - Ryzen 9 9900x Prefill: 192 - 778 tok/s 4K - 64K, 64K read in 82 s Decode: 3–5 tok/s cold 8.7–16.7 tok/s warm 276 GB model, experts streamed from NVMe Project Maya v1.1.24, 128K ctx
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (measurement method).
Unspecified platform × MiniMax M3.1 Flash
Oct 23, 2026 · by @NFT_Chen · third-party report
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | — | sglang | 435.9 tok/s | — |
| — | — | sglang | 123.6 tok/s | — |
🤯什么？！让 MiniMax M3.1 Flash 生成速度直接提升 3.5倍，而且还是完全无损的！ Inco AI 为刚开源的 MiniMax M3.1 Flash 放出了 DFlash 2 drafter，Apache 2.0 开源。 核心亮点： 🔹最高 3.53× 提速（GSM8K 单并发：435.9 tok/s vs 自回归 123.6 tok/s） 🔹完全无损：贪心解码与目标模型输出完全一致，采样严格保持原分布 🔹Block Diffusion 一次并行起草整块 token（block size=8），比模型自带 DSpark 接受长度更长 🔹轻量 2B 草稿器，可直接挂到现有推理栈 快速上手步骤： 1️⃣准备 SGLang + MiniMax-M3.1-Flash 目标模型（参考官方 cookbook） 2️⃣启动时加上：--speculative-algorithm DFLASH…
Digest-only: hardware.name missing; incomplete methodology (engine+version, quantization, context length, hardware state).
RTX 5070 12GB + RTX 5060 Ti 16GB × Qwen3.8-Flash-Next GSQ-RCO IQ3_S
Oct 23, 2026 · by @murasametech
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | — | strata v0.2.0-beta | 66.0 tok/s | — |
ローカルLLMは、手持ちのGPUを組み合わせて使えるのも良いところです。型番やVRAM容量が違っていても組み合わせられます。RTX 5070とRTX 5060 Tiは、VRAMの容量も処理の速さも違います。 自宅ではRTX 5070 12GBとRTX 5060 Ti 16GBを1枚ずつ、計2枚で層を分担させています。VRAMは合計28GBで、足りない部分はRAM 64GBとSSDに置いています。 この構成で、推論エンジンStrata v0.2.0-betaを使い、ファイル合計約84GBのQwen3.8-Flash-Next GSQ-RCO IQ3_Sが、Decode 65〜67 tok/sで動いています。 最初から大容量VRAMのGPUを1枚そろえなくても、GPUを1枚足す、メモリを増やす、と段階的に広げていけるのは、家庭で始めるにはありがたい点です。…
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (context length, measurement method).
RTX 4090 48G × DeepSeek V4.1 Flash 1017
Oct 23, 2026 · by @xiangxiang103 · third-party report
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | 1-bit | — | 26.4 tok/s | — |
| — | 1-bit | — | 65.0 tok/s | — |
| — | — | — | 2.0 tok/s | — |
🥹 有人开源了：家用电脑跑 748B 的 DeepSeek！ Project Maya，MIT 协议，免费。10-07 才建的库，已经 1500 多星，DeepSeek 量化文件下载 3 万次。 它让 DeepSeek V4.1 Flash 1017 这种平时得进机房的模型，在你家电脑上跑起来。 原理很白：常用的专家放显卡，次常用放内存，冷门的丢固态，聊天时自动搬 单张 4090 48G，日常聊天每秒 26.4 个 token，写代码 65。同一台机器换 Unsloth 1-bit，只有 2 个左右 自带网页聊天，兼容 OpenAI 和 Anthropic 接口，Claude Code 改个地址就能接本地 DeepSeek 本地跑，不按 token 付钱。 丑话说前头：这是 1-bit 量化，跟原版选同一个词的比例 87.5% Mac 不支持，显存加内存不到 115G 的也别硬上。 有N 卡的朋友，想本地跑 DeepSeek…
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, context length, measurement method).
DGX Spark × ThinkingCap Qwen3.8-27B
Oct 23, 2026 · by @0xBakeer · third-party report
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | EXL3 4.0 bpw | tandemllm | 66.0 tok/s | — |
@0xWhiteMage built a full recipe for ThinkingCap Qwen3.8-27B and made TandemLLM the default way to run it on a DGX Spark 🙏 EXL3 4.0 bpw, 16.35 GB of weights 97.8% top-1 token agreement with BF16 @niklaslenz_ai on one Spark: 66 tok/s, 0.16s to first token This is how a recipe should look! Great job 👏 https://t.co/zIKbHKnrWm
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, context length, hardware state, measurement method).
RTX 4090 48GB × DeepSeek V4.1-Flash-1017
Oct 23, 2026 · by @servasyy_ai
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | — | maya-xs | 65.0 tok/s | — |
| — | — | maya-xs | 26.4 tok/s | — |
我的天！！！70+ tok/s ！！ 单台 4090/48G 跑 DeepSeek V4.1-Flash-1017 的速度彻底被突破了！ 之前我们手工打了 5 层补丁，写代码一直卡在 17 tok/s。 今天直接上原版 Maya-XS 引擎，一个补丁没打 实测直接突破 70 tok/s： • 代码生成飙到 65 tok/s（峰值 72.4），暴涨 3.8 倍 • 日常聊天 26.4 tok/s，提速 55% • 16K 超长文本首字从 23.5 秒缩到 8.6 秒，快了 2.7 倍 底层的工程思路极其漂亮：48GB 显存吃满高频热专家（代码命中率高达 72%） 冷专家走内存 mmap 按需流式补页，搭配 DSpark 投机解码，磁盘读取仅 12-15MiB/token，丝毫不卡顿。 折腾一圈终于有标准答案了：4090 跑 1017 别再魔改，原版 Maya-XS 就是目前的最优解！
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, quantization, measurement method).
DGX Spark x4 × Ling 3.1 Flash
Oct 23, 2026 · by @sudoingX
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| 143K | Q4 GGUF | llama.cpp | 7.5 tok/s | — |
i gave ling 3.1 flash one prompt and never touched it again the prompt was the floating tree, and the model ran on the 4x dgx sparks, 560b moe in q4 over llama[.]cpp rpc, with hermes agent driving 12 hours 26 min on one prompt 86 requests, 325k tokens out 7.5 tok/s decode, q4 over rpc (painful) peak context 143k of 262k 12/12 tests pass, smoke green 2,166 lines across 12 modules it built the island, the tree with a…
Digest-only: confidence 0.00 below 0.60; incomplete methodology (engine+version, quantization, context length, hardware state, measurement method).
Apple M5 Max × Bonsai 2 9B
Oct 23, 2026 · by @pratikg
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | — | mlx | 144.8 tok/s | — |
| — | — | mlx | 959.6 tok/s | — |
EXPLAINED: How Bonsai 2 9B got ~6x faster in a week [VIDEO] Kydo has the final numbers. Here’s how it actually worked. @PrismML gave Day 0 access to Bonsai 2 9B, an open-weight ternary model. @YukonResearch froze a reference engine for it, MLX on Mac / Apple Silicon (https://t.co/uFpHiT9o9m) and llama.cpp on DGX Spark (https://t.co/tuLkHUh7Qx). The weights stayed locked. Everything that runs the model was open to…
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, context length, hardware state, measurement method).
Also in this window
The 12 most recent posts are shown in full above; 24 posts from the same window are listed here in short form, each with its source post.
- RTX 3060 × Nemotron-3.5-Bolt-74B — 30.0 tok/s decode (2 measurements) · @DogukanUrker
- RTX 5070 12GB x1 + RTX 5060 Ti 16GB × Qwen3.8-Flash-Next GSQ-RCO — 84.5 tok/s decode (2 measurements) · @murasametech
- DGX Spark × DeepSeek V4.1 Flash — 143.0 tok/s decode (12 measurements) · @yume_arasaki
- DGX Spark × ThinkingCap Qwen3.8-27B Kearuga — 125.2 tok/s decode (6 measurements) · @niklaslenz_ai
- Tesla V100 32GB x3 × DeepSeek-v4.1-flash-1017 — 70.0 tok/s decode (2 measurements) · @ItsmeAjayKV
- DGX Spark x4 × DeepSeek v4.1 Flash 1017 — 222.0 tok/s decode (1 measurement) · @norsyx
- DGX Spark × qwen3.8-flash-next — 75.7 tok/s decode (3 measurements) · @filicroval
- Unspecified platform × Bonsai 2 9B — 1229.0 tok/s decode (2 measurements) · @TheDavidTai
- RTX 5090 × Recursive Flint 40B — 224.9 tok/s decode (10 measurements) · @niklaslenz_ai
- RTX 5090 × abliterated model (unnamed) — 65.0 tok/s decode (1 measurement) · @MrHydeDev
- DGX Spark × Qwen — 100.0 tok/s decode (1 measurement) · @AkitaOnRails
- DGX Spark x4 × DeepSeek-V4.1-Flash-1017 — 95.0 tok/s decode (3 measurements) · @calvarado2004
- M4 Max 48GB × Qwen3-1.7B — 300.8 tok/s decode (7 measurements) · @atomic_chat_hq
- DGX Spark x2 × DeepSeek-V4.1-Flash-1017 — 99.6 tok/s decode (7 measurements) · @niklaslenz_ai
- Unspecified platform × DeepSeek v4.1 Flash — 1700.0 tok/s decode (1 measurement) · @bingxu_
- DGX Spark × Ornith-1.5-122B-EXL3-SAGE-2.58bpw — 90.8 tok/s decode (6 measurements) · @ViC305
- M5 Max 36GB × Flint — 193.0 tok/s decode (3 measurements) · @rapidmlx
- DGX Spark × Ornith-1.5-122B — 44.6 tok/s decode (2 measurements) · @niklaslenz_ai
- M5 Ultra 256GB × DeepSeek-V4.1-Flash-1017 — 107.0 tok/s decode (2 measurements) · @NFT_Chen
- Fireworks AI Ultrafast platform × DeepSeek V4.1 Flash — 1521.0 tok/s decode (1 measurement) · @Lonely__MH
- Unspecified platform × DeepSeek V4.1 Flash — 1521.0 tok/s decode (1 measurement) · @MinLiBuilds
- RTX 3090 × Qwen3.8 Flash-Next — 77.0 tok/s decode (2 measurements) · @coffeecup2020
- RTX 5090 × Flint — 216.0 tok/s decode (2 measurements) · @witcheer
- DGX Spark x2 × deepseek v4.1 flash 1017 — 147.6 tok/s decode (1 measurement) · @sudoingX
How to read this page
This digest is an automated transcription of public posts. Runs with a stated engine build, quantization, context length, hardware state and measurement method are published straight to the benchmark board with a link to their source; the rest stay here. These are author-reported numbers, not an independent DiggTop re-measurement: every row carries its provenance so you can check the original, and a number that turns out to be misreported is corrected or removed. Our own testing follows the methodology. Spotted an error? Reply on the source post or submit a corrected run at /submit.
Sources
- @MLBear2 — Oct 23, 2026
- @jun1228909 — Oct 23, 2026
- @majewskizby — Oct 23, 2026
- @dingyi — Oct 23, 2026
- @ItsmeAjayKV — Oct 23, 2026
- @NFT_Chen — Oct 23, 2026
- @murasametech — Oct 23, 2026
- @xiangxiang103 — Oct 23, 2026
- @0xBakeer — Oct 23, 2026
- @servasyy_ai — Oct 23, 2026
- @sudoingX — Oct 23, 2026
- @pratikg — Oct 23, 2026
- @DogukanUrker — Oct 23, 2026
- @murasametech — Oct 23, 2026
- @yume_arasaki — Oct 23, 2026
- @niklaslenz_ai — Oct 22, 2026
- @ItsmeAjayKV — Oct 22, 2026
- @norsyx — Oct 22, 2026
- @filicroval — Oct 22, 2026
- @TheDavidTai — Oct 22, 2026
- @niklaslenz_ai — Oct 22, 2026
- @MrHydeDev — Oct 22, 2026
- @AkitaOnRails — Oct 22, 2026
- @calvarado2004 — Oct 22, 2026
- @atomic_chat_hq — Oct 22, 2026
- @niklaslenz_ai — Oct 22, 2026
- @bingxu_ — Oct 22, 2026
- @ViC305 — Oct 22, 2026
- @rapidmlx — Oct 22, 2026
- @niklaslenz_ai — Oct 22, 2026
- @NFT_Chen — Oct 22, 2026
- @Lonely__MH — Oct 22, 2026
- @MinLiBuilds — Oct 22, 2026
- @coffeecup2020 — Oct 22, 2026
- @witcheer — Oct 22, 2026
- @sudoingX — Oct 22, 2026
FAQ
Are the numbers in this digest sourced?
They are transcribed from public X posts and the repositories those posts link to, with attribution and a source link on every entry. Rows that state the full methodology are published to the benchmark board automatically, so treat them as author-reported until a dispute is raised: the source link on every row is the check.
What does it take for a row to reach the sourced tier?
Engine and exact version, quantization, context length, hardware state and a described measurement method. Posts that state all five are published to the benchmark board as soon as the collector reads them, each row carrying its source link — nothing waits in a human review queue. Anything missing a condition stays in this digest.
Where does the data come from?
Public X posts about local-model inference measured on owned hardware, discovered by keyword search every two hours and linked back to the original thread. The digest is regenerated daily.