X local-model bench digest — Nov 2, 2026
As of Nov 2, 2026 — 31 posts collected from X. 85 measured runs catalogued, 31 kept to the digest only. Latest source post Nov 2, 2026 12:48 UTC.
Every number below is what its author published, transcribed with the engine, quantization and context length they reported. Sources are linked on each entry. Only posts that state the full methodology (engine build, quantization, context length, hardware state, measurement method) reach the benchmark board; the rest stay here as leads.
RTX 5070 Ti × Qwen3.6-35B
Nov 2, 2026 · by @QwenAIFans · third-party report
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | — | llama.cpp | 105.0 tok/s | — |
| — | — | llama.cpp | 143.0 tok/s | — |
💥 Qwen3.6-35B 塞不进显存？别先砍量化。 llama.cpp 把专家扔 CPU，再开 --moe-cache-mib：单卡 5070 Ti，大约 105→143 tok/s，约 1.4 倍。 温馨提示：这个开关默认关着，不写等于 0。 你这张卡开多大 cache 最顺？ 来源：High U👇 https://note.com/high_u/n/nf64f0b62a91f https://t.co/ovfnuesXqp
Digest-only: confidence 0.55 below 0.60; incomplete methodology (engine+version, quantization, context length, measurement method).
RTX 3060 12GB × Qwen 3.8 Flash Next
Nov 2, 2026 · by @ai_hakase_ · third-party report
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | — | llama.cpp | 1.5 tok/s | — |
| — | — | betelgeuse | 22.0 tok/s | — |
| — | — | betelgeuse | 48.0 tok/s | — |
| — | Q8_K_XL | betelgeuse | 12.0 tok/s | — |
【制限の厳しい環境で巨大MoEモデルを動かすカスタム推論エンジン「Betelgeuse」】 ミドルレンジGPU「RTX 3060 12GB」と16GBの小容量RAM環境下で、68GB規模のMoEモデル「Qwen 3.8 Flash Next」（GSQ-RCO-IQ2_XS量子化ビルド）をエキスパートの枝刈りなし・100% bit-exact（完全ビット一致）の品質を維持したまま実用速度で動作させることに成功したそうです！ 標準のllama.cpp（mmap）ではページキャッシュのスラッシングにより1.47 tok/sだった処理速度が、Betelgeuseエンジン（
bel-cli/bel-server）によってウォームデコード時に22.03 tok/s、最大48 tok/sまで大幅に向上していますね。 主な技術的アプローチとしては、以下の仕組みが挙げられます！…
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, context length, measurement method).
CMP 100-100 x2 × Qwen3.8-27B
Nov 2, 2026 · by @QwenAIFans · third-party report
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | — | — | 32.0 tok/s | — |
💥 两张挖矿废卡，每张大约 25 刀，跑起了 Qwen3.8-27B。 CMP 100-100：Linux 下 6GB 解锁到 12GB HBM2，双卡约 32 tok/s。 温馨提示：作者测的，PCIe 还卡在 Gen1 x1。 来源：@Roberty35570209👇 https://t.co/CY9NVmj41u
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, quantization, context length, measurement method).
RTX 3060 × Spark-X2.6-4B
Nov 2, 2026 · by @DogukanUrker
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| 288K | Q8_0 | llama.cpp | 67.0 tok/s | 3500.0 tok/s |
| — | BF16 | llama.cpp | 39.0 tok/s | — |
| — | Q8_0 | llama.cpp | 69.0 tok/s | — |
| 448K | Q8_0 | llama.cpp | 12.0 tok/s | — |
running Spark-X2.6-4B on a single RTX 3060. 12GB VRAM. (config below) 288K context. ~67 tok/s decode, ~3500 tok/s prefill. only 9 of its 36 layers keep a full kv cache, so long context is cheap. found a needle hidden at 280K tokens. Q8 over BF16: almost lossless at 4B, and decode is 69 vs 39 tok/s. config: llama-server -m Spark-X2.6-4B-Q8_0.gguf -ngl 99 -c 294912 -fa on --jinja -np 1 --cache-type-k q8_0…
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, measurement method).
Unspecified platform × GLM-4.7-Flash-abliterated i1-Q3_K_M
Nov 2, 2026 · by @Isichan_Hitori
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | — | — | 200.0 tok/s | — |
【無検閲AI、脱がせるのは得意でも小説は？】 ～無検閲モデルにエロ小説を90本、書かせてみた～ 引用元の無検閲モデル上位から5本を選び、同じお題でNSFW小説の一節を書かせてみました。 ✅比較 お題は3つ。 第一：結婚10年目の夫婦、記念日に温泉旅館で過ごす夜（妻の視点） 第二：残業後、20代の同僚同士が初めて結ばれる夜（三人称） 第三：雨の夜のアトリエで、30代の作家と写真家（写真家の視点） 1つのお題（800字程度）を6回ずつで18本、合計90本。 評価項目は「具体さ」「文章」「魅力」で、採点役は別のLLM2種にお願いしました。 対象はこの５モデル ①Qwen3.8-Flash-Next-GSQ-RCO-abliterated IQ2_XS（SC117） 約68.47GB ②Qwen3.8-Flash-Next-Abliterated-Uncensored IQ3_S（RVN） 約88.88GB…
Digest-only: confidence 0.55 below 0.60; hardware.name missing; incomplete methodology (engine+version, context length, hardware state, measurement method).
RTX 4090 × Qwen3.8-Flash-Next
Nov 2, 2026 · by @QwenAIFans · third-party report
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | IQ3_S | llama.cpp f5222f25c | 29.8 tok/s | — |
| — | IQ3_S | llama.cpp f5222f25c + tandem.patch | 97.3 tok/s | — |
| — | IQ3_S | llama.cpp f5222f25c + tandem.patch | 96.1 tok/s | — |
| 65536 | IQ3_S | llama.cpp f5222f25c + tandem.patch | — | 860.0 tok/s |
| 65536 | IQ3_S | llama.cpp f5222f25c | — | 600.0 tok/s |
| — | IQ3_S | llama.cpp f5222f25c + tandem.patch | 100.9 tok/s | — |
| — | IQ3_S | llama.cpp f5222f25c + tandem.patch | 182.5 tok/s | — |
| — | IQ3_S | llama.cpp f5222f25c + tandem.patch | 131.1 tok/s | — |
| — | IQ3_S | llama.cpp f5222f25c + tandem.patch | 91.3 tok/s | — |
| — | IQ2_XS | llama.cpp f5222f25c + tandem.patch | 56.9 tok/s | — |
| — | IQ2_XS | llama.cpp f5222f25c + tandem.patch | 89.6 tok/s | — |
| — | IQ2_XS | llama.cpp f5222f25c + tandem.patch | 132.4 tok/s | — |
🚀 普通 llama.cpp 跑 Qwen3.8-Flash-Next，4090 上也能到 97 tok/s 了。 新补丁 tandem.cpp：热专家留 GPU，冷的交给 CPU。IQ3_S 约 30→97，还是每个专家都算满。 温馨提示：目前 Linux+CUDA。 来源：GitHub ewhacc/tandem.cpp👇 https://github.com/ewhacc/tandem.cpp https://t.co/Dm9uJIaPDL
Full methodology stated.
RTX 3090 × Qwen3.8 Flash-Next
Nov 2, 2026 · by @superalesha
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| 32K | — | — | 88.0 tok/s | — |
| 130K | — | — | 88.0 tok/s | — |
| 32K | — | — | — | 1932.0 tok/s |
| — | — | — | 123.0 tok/s | — |
| 500K | — | — | 70.0 tok/s | — |
The numbers Qwen4 has to beat on my rig. Qwen3.8 Flash-Next on 1 RTX 3090: 88 tok/s decode at 32K. 88 tok/s decode at 130K. 1932 tok/s prefill at 32K. The same model on 2 cards: 123 tok/s. DeepSeek V4.1 Flash 1017 on 4x 3090: 70+ tok/s on top of a 500K prompt. 0.8 GB of VRAM for 512K of context. I will post the same lines for Qwen4 on the day it lands. If the 27B or Flash beats these on 1 card, that is the model I…
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, quantization, hardware state, measurement method).
single GPU × 9B model (unnamed)
Nov 2, 2026 · by @mikenevermiss
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | — | — | 185.0 tok/s | — |
a 9B model just beat qwen's own 27B on 7 out of 7 benchmarks and it's uncensored. the stock version refused 100 out of 100 test prompts. this one refuses 6. 256K context window native vision 185 tokens/sec on a single GPU apache 2.0 check it out : https://t.co/7ajQNVU7vM I broke down every other free model worth running in my article 👇
Digest-only: confidence 0.55 below 0.60; incomplete methodology (engine+version, quantization, context length, hardware state, measurement method).
DGX Spark × Gemma 4 31B
Nov 2, 2026 · by @ashxhart
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | — | tensorfold 1.2.1 | 28.3 tok/s | — |
| — | — | tensorfold 1.2.1 | 12.2 tok/s | — |
TensorFold engine 1.2.1 is out. A few highlights: • MiniMax-M3.1-Flash on CUDA across two GB10s, using Mia-AiLab's EXL3 pack. • Gemma 4 31B on GB10 with Google's MTP assistant: 28.3 tok/s versus 12.2 undrafted, at one stream. • Qwen3.5/3.6 windows of 1,048,576 tokens on CUDA with YaRN. • MOSS-Image end to end on Metal, via the CLI. • Gemma 4 E4B prompt processing 2 to 2.6 times faster on Intel Arc Pro B70. There are…
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (quantization, context length, hardware state, measurement method).
V100 × Qwen3.8-Flash-Next
Nov 2, 2026 · by @QwenAIFans · third-party report
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| 128K | IQ3_S | strata | 63.2 tok/s | 1460.0 tok/s |
| 256K | IQ3_S | strata | 58.5 tok/s | — |
🔥 老 V100 限 150W，Qwen3.8-Flash-Next 128K 还能 63.2 tok/s Strata 对着旧 CUDA 调过。预填充 1460，解码 63.2；256K 也有 58.5。 温馨提示：IQ3_S，作者自己那台的数。 来源：@yama_boku👇 https://t.co/GIRTWihsqu
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, measurement method).
H200 x4 × DeepSeek V4.1 Flash
Nov 2, 2026 · by @Zaddyzaddy
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | — | — | 653.0 tok/s | — |
| — | — | — | — | 42000.0 tok/s |
We published our recipe for DeepSeek V4.1 Flash, on one 4xH200 we are able to hit 653 tok/s C1 and up to 42k tok/s prefill
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, quantization, context length, hardware state, measurement method).
M4 Mac Mini + RTX 3090
Nov 1, 2026 · by @0xSero
Broke 1000 tok/s prefill on an M4 Mac Mini (16gb ram/256gb ssd) + RTX 3090 The cost to run one of the best models ever is like 2200 https://t.co/Wm87Naughd
Digest-only: confidence 0.40 below 0.60; model.name missing; incomplete methodology (engine+version, quantization, context length, hardware state, measurement method).
Also in this window
The 12 most recent posts are shown in full above; 19 posts from the same window are listed here in short form, each with its source post.
- Cerebras × Qwen3.8-27B — 1279.0 tok/s decode (3 measurements) · @QwenAIFans
- DGX Station × GLM 5.3 Flash — 1000.0 tok/s decode (1 measurement) · @tomcupr
- RTX 3090 x4 × MiniMax M3.1 Flash — 27.0 tok/s decode (2 measurements) · @superalesha
- Tesla P40 x2 × Qwen MoE 125B — 71.1 tok/s decode (3 measurements) · @QwenAIFans
- MacBook Pro M5 × Qwen3.5 9B — 92.1 tok/s decode (3 measurements) · @ronixtec
- RTX 5090 D × Qwen3.8-Flash-Next-Gyro-M — 405.0 tok/s decode (2 measurements) · @QwenAIFans
- CMP 170HX x2 × GLM-5.3-Flash — 90.0 tok/s decode (1 measurement) · @eduarD3V_
- H200 x4 × DeepSeek-V4.1-Flash — 653.0 tok/s decode (2 measurements) · @HatforceSec
- RTX PRO 6000 x2 × Qwen3.8-Flash-Next — 325.4 tok/s decode (3 measurements) · @QwenAIFans
- RTX PRO 6000 × Qwen3.8-Flash-Next — 325.4 tok/s decode (3 measurements) · @Dataphil3
- RTX 5090 × LFM3-1.7B — 402.0 tok/s decode (3 measurements) · @BennyDaBall_OG
- Tesla P40 x2 × 125B MoE (unspecified) — 71.1 tok/s decode (3 measurements) · @lyesbahi
- RTX 3090 × DeepSeek v4.1 flash — 126.0 tok/s decode (3 measurements) · @RubenTorney
- RTX 3090 x4 × DeepSeek V4.1 Flash — 81.5 tok/s decode (2 measurements) · @superalesha
- RTX 5090 × Qwen3.8-Flash-Next-GSQ-RCO-GGUF — 185.0 tok/s decode (2 measurements) · @jun1228909
- DGX Spark x2 × DeepSeek-V4.1-Flash-1017 — 166.1 tok/s decode (9 measurements) · @wei_wang
- RTX 5090 × SenseNova 27B — 157.0 tok/s decode (1 measurement) · @BennyDaBall_OG
- DGX Spark × DeepSeek v4.1 Flash — 107.2 tok/s decode (2 measurements) · @jayleaton
- RTX 4090 × Spark-X2.6-1.7B-DSpark — 769.0 tok/s decode (1 measurement) · @SparkLLM
How to read this page
This digest is an automated transcription of public posts. Runs with a stated engine build, quantization, context length, hardware state and measurement method are published straight to the benchmark board with a link to their source; the rest stay here. These are author-reported numbers, not an independent DiggTop re-measurement: every row carries its provenance so you can check the original, and a number that turns out to be misreported is corrected or removed. Our own testing follows the methodology. Spotted an error? Reply on the source post or submit a corrected run at /submit.
Sources
- @QwenAIFans — Nov 2, 2026
- @ai_hakase_ — Nov 2, 2026
- @QwenAIFans — Nov 2, 2026
- @DogukanUrker — Nov 2, 2026
- @Isichan_Hitori — Nov 2, 2026
- @QwenAIFans — Nov 2, 2026
- @superalesha — Nov 2, 2026
- @mikenevermiss — Nov 2, 2026
- @ashxhart — Nov 2, 2026
- @QwenAIFans — Nov 2, 2026
- @Zaddyzaddy — Nov 2, 2026
- @0xSero — Nov 1, 2026
- @QwenAIFans — Nov 1, 2026
- @tomcupr — Nov 1, 2026
- @superalesha — Nov 1, 2026
- @QwenAIFans — Nov 1, 2026
- @ronixtec — Nov 1, 2026
- @QwenAIFans — Nov 1, 2026
- @eduarD3V_ — Nov 1, 2026
- @HatforceSec — Nov 1, 2026
- @QwenAIFans — Nov 1, 2026
- @Dataphil3 — Nov 1, 2026
- @BennyDaBall_OG — Nov 1, 2026
- @lyesbahi — Nov 1, 2026
- @RubenTorney — Nov 1, 2026
- @superalesha — Nov 1, 2026
- @jun1228909 — Nov 1, 2026
- @wei_wang — Nov 1, 2026
- @BennyDaBall_OG — Nov 1, 2026
- @jayleaton — Nov 1, 2026
- @SparkLLM — Nov 1, 2026
FAQ
Are the numbers in this digest sourced?
They are transcribed from public X posts and the repositories those posts link to, with attribution and a source link on every entry. Rows that state the full methodology are published to the benchmark board automatically, so treat them as author-reported until a dispute is raised: the source link on every row is the check.
What does it take for a row to reach the sourced tier?
Engine and exact version, quantization, context length, hardware state and a described measurement method. Posts that state all five are published to the benchmark board as soon as the collector reads them, each row carrying its source link — nothing waits in a human review queue. Anything missing a condition stays in this digest.
Where does the data come from?
Public X posts about local-model inference measured on owned hardware, discovered by keyword search every two hours and linked back to the original thread. The digest is regenerated daily.