X local-model bench digest — Oct 26, 2026
As of Oct 26, 2026 — 34 posts collected from X. 105 measured runs catalogued, 2 published to the board, 32 kept to the digest only. Latest source post Oct 26, 2026 08:30 UTC.
Every number below is what its author published, transcribed with the engine, quantization and context length they reported. Sources are linked on each entry. Only posts that state the full methodology (engine build, quantization, context length, hardware state, measurement method) reach the benchmark board; the rest stay here as leads.
Mac Studio M5 Ultra 256GB × DeepSeek-V4.1-Flash-1017
Oct 26, 2026 · by @niklaslenz_ai
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| 524288 | — | tensorfold deepseek-v41-main (72a552ed) | 80.5 tok/s | — |
| 524288 | — | tensorfold deepseek-v41-main (72a552ed) | 70.5 tok/s | — |
| 524288 | — | tensorfold deepseek-v41-main (72a552ed) | 88.3 tok/s | — |
DeepSeek-V4.1-Flash-1017 decodes code at 80.5 tok/s on one Mac Studio M5 Ultra 256 GB with BobClawblaw's recipe on TensorFold. Setup: – Recipe and TensorFold branch deepseek-v41-main (72a552ed), his own tool and settings – Context 524,288 tokens, 4 streams, DSpark drafts on, temperature 0 – Speed and quality with thinking off, except GPQA with thinking Speed: – Decode, code: 80.5 tok/s – Decode, prose: 70.5 tok/s –…
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (quantization, hardware state).
RTX 5070 × DeepSeek-V4.1-Flash
Oct 26, 2026 · by @murasametech
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | — | strata | 100.0 tok/s | — |
これはヤバすぎる！ DeepSeek-V4.1-FlashをRTX 5070 12GBを使ってStrataで実行するとまさかの100+ tok/sを達成。 システムRAMをどれだけ積めるかが影響しそうですが、コンシューマ向けカードでこれだけの性能が出るのは本当に凄い。
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, quantization, context length, hardware state, measurement method).
RTX 5080 + RTX 5060 Ti × DeepSeek; Qwen
Oct 26, 2026 · by @rlaope
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | — | — | 4.8 tok/s | — |
| — | — | — | 48.7 tok/s | — |
OH MY .......... HERMES !! FOR TOKKENMAXXER. oh-my-hermes (OMH): an operating layer for hermes-agent. become a master of graph engineering. u keep talking to Hermes. OMH picks the skill and model for each lane, runs them in parallel, and never reports planned work as done. 📸 The screenshot: one Ryzen 7800X3D box running DeepSeek on an RTX 5080 (4.8 tok/s, experts offloaded to RAM) and Qwen on an RTX 5060 Ti (48.7…
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, quantization, context length, measurement method).
DGX Spark x2 × MiniMax-M3.1-Flash
Oct 26, 2026 · by @jayleaton
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| 131K | — | — | 29.4 tok/s | — |
| 131K | — | — | 27.8 tok/s | — |
This one feels a bit disappointing, honestly. @MiniMax_AI MiniMax-M3.1-Flash on two DGX Sparks ALPHA: 29.4 tok/s on code 27.8 on tool calls 131K context. My DeepSeek recipe does ~90 on the same TP=2 setup. MiniMax reads about 2x the bytes per token, and Spark decode is bandwidth-bound. Built on @MiaAI_lab's MiniMax-M3.1 TensorFold patch and @ashxhart TensorFold. Model and DSpark drafter by MiniMax. Decoding on a…
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, quantization, measurement method).
tinybox red × GLM-5.3-Flash
Oct 26, 2026 · by @tinygrad
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | IQ4_XS | — | 1000.0 tok/s | — |
The tinybox red flash gets faster! This is on GLM-5.3-Flash IQ4_XS with a single user at 262k context. Multi user decode is ~1000 tok/s aggregate. Running Terminal-Bench and DeepSWE now, looking good. https://t.co/fnznGbDCGa
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, hardware state, measurement method).
Mac mini M5 Pro 64GB × DeepSeek V4.1 Flash
Oct 26, 2026 · by @pratikg
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | — | — | 5.5 tok/s | — |
I made 11 open-weight models rewrite Abraham Lincoln's Gettysburg Address 1,000 times each on my Mac mini (M5 Pro, 64GB) Every rewrite fed the next one. THE WHOLE RUN LASTED FOR 45 HOURS. Here's the raw footage, timelapsed to 40 seconds. I wanted to know how many rewrites it takes before it stops being the same speech. Held on for all 1000 rounds: - DeepSeek V4.1 Flash (343GB) - GLM-5.3 Flash (88GB) Both are too big…
Digest-only: confidence 0.55 below 0.60; incomplete methodology (engine+version, quantization, context length).
B200 x4 × GLM-5.3-Flash
Oct 26, 2026 · by @ekzhang1
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | — | resin | — | 60000.0 tok/s |
| — | — | resin | 16000.0 tok/s | — |
Alright I finally have more to share on my personal project! Resin is a runtime for building fast LLM inference engines. The idea is to make all the generic parts of inference reusable, and leave only the model-specific forward pass + kernels to be written for each architecture / hardware target. To build an engine: 1. Import Resin's core. 2. Have an agent write a forward pass for the model you’re interested in, for…
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, quantization, context length, hardware state, measurement method).
DGX Spark × Qwen3.8-Flash-Next
Oct 26, 2026 · by @niklaslenz_ai
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| 262144 | NVFP4 | tensorfold | 43.2 tok/s | — |
| 262144 | NVFP4 | tensorfold | 87.6 tok/s | — |
| 32768 | NVFP4 | tensorfold | — | 2163.0 tok/s |
| 262144 | EXL3 3.05 bpw | exllamav2 | 58.4 tok/s | — |
| 262144 | EXL3 3.05 bpw | exllamav2 | 41.5 tok/s | — |
| 32768 | EXL3 3.05 bpw | exllamav2 | — | 662.0 tok/s |
Qwen3.8-Flash-Next on one DGX Spark: @MiaAI_lab's TensorFold recipe decodes code at 43.2 tok/s, @ViC305's two-model EXL3 recipe at 58.4 tok/s. Setup: – Same voxel-world prompt, both as an agent in Qwen Code 0.25.2, one prompt, one try, no help, thinking on – Context: 262,144 tokens – Mia's recipe: NVFP4 with MTP on @ashxhart's TensorFold, 5 slots, answer limit 131,072 tokens – ViC305's recipe: EXL3 3.05 bpw on…
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version).
RTX 3090 x2 × Nanbeige4.3-33B
Oct 26, 2026 · by @Tech2Wild
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| 131072 | W4A16 (INT4 AutoRound, GPTQ, gro | vllm v0.27.1 | 66.2 tok/s | — |
| 131072 | W4A16 (INT4 AutoRound, GPTQ, gro | vllm v0.27.1 | 354.0 tok/s | — |
| 17K | W4A16 (INT4 AutoRound, GPTQ, gro | vllm v0.27.1 | — | 1770.0 tok/s |
| — | W4A16 (INT4 AutoRound, GPTQ, gro | vllm v0.27.1 | 53.2 tok/s | — |
| — | W4A16 (INT4 AutoRound, GPTQ, gro | vllm v0.27.1 | 58.6 tok/s | — |
🔧 Nanbeige4.3-33B dropped with no int4 that vLLM can run on RTX 3090s. So we made one. 📦 Full recipe: AutoRound W4A16 quant, vLLM launcher, C1-C6 bench, 69-scenario eval 🖥️ 2x RTX 3090, NVLink, TP2 ⚡ 66 tok/s single stream https://github.com/tonyd2wild/Nanbeige4.3-33B-W4A16-2x-RTX-3090
Full methodology stated (engine, quantization, context, hardware state, method) — 1 added to the benchmark board (1 of 5 measured rows queued; the rest are published here only).
RTX A6000 × Nanbeige4.3-33B
Oct 25, 2026 · by @Alex_tra_memory
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| 113K | FP8 weights, NVFP4 KV cache | vllm 0.32 | 19.0 tok/s | — |
Nanbeige4.3-33B as a live coding agent on an A6000: Given a Dijkstra function with 5 planted bugs + 4 tools, it fixed all 5 and passed 8/8 hidden edge-case tests it never saw. The map replays its code after every edit. vLLM 0.32 · FP8 weights · NVFP4 KV cache on Ampere (113K-token pool) · ~19 tok/s
Digest-only: confidence 0.00 below 0.60; incomplete methodology (engine+version, quantization, context length, hardware state, measurement method).
DGX Spark x4 × DeepSeek-V4.1-Flash-1017
Oct 25, 2026 · by @niklaslenz_ai
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | FP8/MXFP4 | tensorfold 6beaad9 | 70.2 tok/s | — |
| — | FP8/MXFP4 | tensorfold 6beaad9 | 69.5 tok/s | — |
| 32768 | FP8/MXFP4 | tensorfold 6beaad9 | — | 3743.0 tok/s |
| — | FP8/MXFP4 | tensorfold 6beaad9 | 115.9 tok/s | — |
| — | FP8/MXFP4 | tensorfold 6beaad9 | 139.0 tok/s | — |
| — | FP8/MXFP4 | tensorfold 6beaad9 | 119.9 tok/s | — |
| — | FP8/MXFP4 | tensorfold 6beaad9 | 206.6 tok/s | — |
| 32768 | FP8/MXFP4 | tensorfold 6beaad9 | — | 3872.0 tok/s |
| 32768 | FP8/MXFP4 | tensorfold 6beaad9 | — | 4019.0 tok/s |
| — | FP8/MXFP4 | tensorfold 6beaad9 | 157.0 tok/s | — |
I interrupted every run and got straight to it: DeepSeek-V4.1-Flash-1017 on four DGX Sparks: 70.2 tok/s on prose with @MiaAI_lab's TensorFold recipe. Setup: – One model split across all four Sparks (TP4), the FP8/MXFP4 checkpoint as DeepSeek published it – The recipe as of Oct 25, commit 6beaad9, with the 8 TF_DS_* switches her README table sets on for her cluster, and her server defaults for context and streams –…
Full methodology stated.
DGX Spark × Qwen3.8-Flash-Next
Oct 25, 2026 · by @niklaslenz_ai
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| 262144 | EXL3 3.05 bpw | exllamav2 0fc9145 | 58.4 tok/s | — |
| 262144 | EXL3 3.05 bpw | exllamav2 0fc9145 | 53.9 tok/s | — |
| 262144 | EXL3 3.05 bpw | exllamav2 0fc9145 | 41.5 tok/s | — |
Qwen3.8-Flash-Next at 3.05 bpw decodes code at 58.4 tok/s on one DGX Spark, with a second model, MiniCPM 4.7, loaded beside it. Setup: – Recipe: @ViC305's TWOMODELS-ONESPARK (0fc9145) – Engine: ExLlamaV3 and TabbyAPI, MTP drafts – Context: 262,144 tokens, 1 slot Speed: – Decode, code: 58.4 tok/s – Decode, prose: 53.9 tok/s – 500 words: about 12.6 s – 8 requests at once: 41.5 tok/s in total Quality: – MMLU-Pro: 68.3%…
Digest-only: incomplete methodology (hardware state, measurement method).
Also in this window
The 12 most recent posts are shown in full above; 22 posts from the same window are listed here in short form, each with its source post.
- Strix Halo × Qwen — 255.0 tok/s decode (3 measurements) · @KireiStudios
- M5 Pro 48GB × Qwen3.8-Flash-Next — no throughput stated (2 measurements) · @QwenAIFans
- RTX 5060 Ti x2 × Qwen3.8-27B — 143.1 tok/s decode (6 measurements) · @niklaslenz_ai
- DGX Spark x4 × DeepSeek V4.1 Flash — 75.0 tok/s decode (1 measurement) · @christopherowen
- RTX 5090 × Nanbeige4.3-33B — 70.0 tok/s decode (2 measurements) · @lxe
- DGX Spark × qwen3.8 flash-next — 130.0 tok/s decode (3 measurements) · @filicroval
- DGX Spark x4 × DeepSeek V4.1 Flash 1017 — 210.0 tok/s decode (10 measurements) · @MiaAI_lab
- RTX 3080 × KRAFTON Raon-1 Mini (30B A4B) Preview — 45.0 tok/s decode (1 measurement) · @PaulGugAI
- Mac mini M5 Pro 64GB × Nanbeige4.3-33B — 14.4 tok/s decode (4 measurements) · @pratikg
- RTX 3090 × DeepSeek-V4.1-Flash-1017 — 27.8 tok/s decode (2 measurements) · @NFT_Chen
- RTX 5070 × DeepSeek-V4.1-Flash — 100.0 tok/s decode (1 measurement) · @Forhanvv
- RTX 4080 Super 16G × DeepSeek V4.1-Flash-1017 — 83.3 tok/s decode (6 measurements) · @NFT_Chen
- RTX 5090 × DeepSeek-V4.1-Flash — 36.2 tok/s decode (2 measurements) · @lyesbahi
- RTX 3090 x4 × DeepSeek V4.1 Flash — 97.0 tok/s decode (9 measurements) · @superalesha
- RTX 3060 × Nanbeige4.3-33B — 20.0 tok/s decode (1 measurement) · @DogukanUrker
- RTX 4090 48GB × DeepSeek — 73.8 tok/s decode (2 measurements) · @MinLiBuilds
- Unspecified platform × DeepSeek V4.1 Flash — 3.0 tok/s decode (1 measurement) · @yuhasbeentaken
- Radeon AI PRO R9700 × DeepSeekV4.1-Flash-1017 — 27.6 tok/s decode (2 measurements) · @dextune
- Unspecified platform — 264.0 tok/s decode (2 measurements) · @0xBakeer
- DGX Spark × Nemotron 3.5 Super VL — 84.2 tok/s decode (3 measurements) · @niklaslenz_ai
- R9700 × DeepSeek — 84.7 tok/s decode (4 measurements) · @jun1228909
- Unspecified platform × GLM-5.3-Flash-FP4-local — 90.0 tok/s decode (1 measurement) · @vr8vr8
How to read this page
This digest is an automated transcription of public posts. Runs with a stated engine build, quantization, context length, hardware state and measurement method are published straight to the benchmark board with a link to their source; the rest stay here. These are author-reported numbers, not an independent DiggTop re-measurement: every row carries its provenance so you can check the original, and a number that turns out to be misreported is corrected or removed. Our own testing follows the methodology. Spotted an error? Reply on the source post or submit a corrected run at /submit.
Sources
- @niklaslenz_ai — Oct 26, 2026
- @murasametech — Oct 26, 2026
- @rlaope — Oct 26, 2026
- @jayleaton — Oct 26, 2026
- @tinygrad — Oct 26, 2026
- @pratikg — Oct 26, 2026
- @ekzhang1 — Oct 26, 2026
- @niklaslenz_ai — Oct 26, 2026
- @Tech2Wild — Oct 26, 2026
- @Alex_tra_memory — Oct 25, 2026
- @niklaslenz_ai — Oct 25, 2026
- @niklaslenz_ai — Oct 25, 2026
- @KireiStudios — Oct 25, 2026
- @QwenAIFans — Oct 25, 2026
- @niklaslenz_ai — Oct 25, 2026
- @christopherowen — Oct 25, 2026
- @lxe — Oct 25, 2026
- @filicroval — Oct 25, 2026
- @MiaAI_lab — Oct 25, 2026
- @PaulGugAI — Oct 25, 2026
- @pratikg — Oct 25, 2026
- @NFT_Chen — Oct 25, 2026
- @Forhanvv — Oct 25, 2026
- @NFT_Chen — Oct 25, 2026
- @lyesbahi — Oct 25, 2026
- @superalesha — Oct 25, 2026
- @DogukanUrker — Oct 25, 2026
- @MinLiBuilds — Oct 25, 2026
- @yuhasbeentaken — Oct 25, 2026
- @dextune — Oct 25, 2026
- @0xBakeer — Oct 25, 2026
- @niklaslenz_ai — Oct 25, 2026
- @jun1228909 — Oct 25, 2026
- @vr8vr8 — Oct 25, 2026
FAQ
Are the numbers in this digest sourced?
They are transcribed from public X posts and the repositories those posts link to, with attribution and a source link on every entry. Rows that state the full methodology are published to the benchmark board automatically, so treat them as author-reported until a dispute is raised: the source link on every row is the check.
What does it take for a row to reach the sourced tier?
Engine and exact version, quantization, context length, hardware state and a described measurement method. Posts that state all five are published to the benchmark board as soon as the collector reads them, each row carrying its source link — nothing waits in a human review queue. Anything missing a condition stays in this digest.
Where does the data come from?
Public X posts about local-model inference measured on owned hardware, discovered by keyword search every two hours and linked back to the original thread. The digest is regenerated daily.