X local-model bench digest — Oct 21, 2026

As of Oct 21, 2026 — 43 posts collected from X. 121 measured runs catalogued, 43 kept to the digest only. Latest source post Oct 21, 2026 09:26 UTC.

Every number below is what its author published, transcribed with the engine, quantization and context length they reported. Sources are linked on each entry. Only posts that state the full methodology (engine build, quantization, context length, hardware state, measurement method) reach the benchmark board; the rest stay here as leads.

M3 Ultra 512GB × DeepSeek V4.1

Oct 21, 2026 · by @jundotkim

Context	Quant	Engine	Decode	Prefill
—	—	mlx 0.7.2	68.2 tok/s	949.0 tok/s
128K	—	mlx 0.7.2	—	900.0 tok/s
—	—	mlx 0.7.2	28.0 tok/s	—

It's only been a day since 0.7.1, but oMLX 0.7.2 is out with changes worth shipping now. https://t.co/s0GBOcbus1 This release brings faster DeepSeek V4.1 on every oQ build, Lightning MTP in distributed inference, and Qwen3.8-Flash-Next GGUFs on Splash 1.4.0. Faster DeepSeek V4.1 on oQ2e to oQ4e - oQ3e with Lightning MTP + CED: decode 49.4 - 68.2 tok/s (+38%), prefill 700 - 949 tok/s (+36%). Every quant from oQ2e to…

Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (hardware state, measurement method).

DGX Spark × Ornith-1.5-122B

Oct 21, 2026 · by @MiaAI_lab

Context	Quant	Engine	Decode	Prefill
—	EXL3 EAQ 4 bpw	tensorfold	52.0 tok/s	—
—	EXL3 EAQ 4 bpw	tensorfold	103.0 tok/s	—
—	EXL3 EAQ 4 bpw	tensorfold	90.0 tok/s	—
—	EXL3 EAQ 4 bpw	tensorfold	—	2000.0 tok/s

Run Ornith-1.5-122B on your DGX Spark ✨ Uses my Ornith-1.5-122B-EXL3-EAQ 4 bpw quant. Recipe can easily be adapted for the RTX PRO 6000 too. - Runs on TensorFold - 256K context, total 1M KV cache - 4 concurrent streams by default - 86 GB total memory - Uses DFlash2 drafts Decode speed (thinking off): - 52 tok/s on prose, single stream - 103 tok/s on code, single stream - 90 tok/s on prose, 4 concurrent streams…

Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, context length, hardware state, measurement method).

RTX 3090 × DeepSeek-V4.1-Flash-1017

Oct 21, 2026 · by @ItsmeAjayKV

Context	Quant	Engine	Decode	Prefill
128K	—	maya	8.5 tok/s	—
64K	—	maya	—	240.0 tok/s

RTX 3090を買っておいて本当によかった！😭 今、自分の3090で動かしているモデルが本当にすごいです。まだテスト段階なので、速度は遅いし、改善できるところもたくさんあります。でも、これが今の結果です！ DeepSeek-V4.1-Flash-1017を、Maya inference engineを使ってRTX 3090で動かしています。 Maya-S quantの結果がかなり良くて、今は128K contextで動かしています。前は64Kでした！現在の速度：・Decode：7〜10 tok/s（まだ遅い）・Prefill：64K contextで240 tok/s（なかなか良い！）これからinference側でもっと改善が進むことを期待しています。最終的には、ハードウェアの限界だけがボトルネックになるところまで行きたいです。まだまだ改善の余地はたくさんありますね！

Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, hardware state, measurement method).

DGX Spark × Ornith 1.5 122B

Oct 21, 2026 · by @0xBakeer

Context	Quant	Engine	Decode	Prefill
—	—	tandemllm	38.0 tok/s	—
—	—	tandemllm	123.0 tok/s	—
—	—	tandemllm	85.0 tok/s	—
—	—	tandemllm	110.0 tok/s	—
—	—	tandemllm	114.0 tok/s	—
—	—	tandemllm	125.0 tok/s	—

Spent the last day trying to support Ornith 1.5 122B fast on one DGX Spark with TandemLLM. Plain decoding sat at ~38 tok/s no matter what I asked it. Usable, but slow for agent work. First I tried a DFlash2 drafter plus lookup. Big jump, 67 to 123 tok/s. Then I switched to the model's own MTP layer. It won on every task. Prose 85, chat 110, code 114, copy 125. The part I checked most: every drafted output matches…

Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, quantization, context length, hardware state, measurement method).

Tenstorrent TT-QuietBox × gpt-oss-120b

Oct 21, 2026 · by @Fixstars_JP

Context	Quant	Engine	Decode	Prefill
—	—	—	6.9 tok/s	—
—	—	—	10.1 tok/s	—

【ホワイトペーパー】Tenstorrent TT-QuietBoxでgpt-oss-120bのデコード速度を6.9→10.05 tok/s（+45.7%）に高速化しました。Wormhole 8基 GDDR6 計96GB、同時実行数1、入出力各128トークンで実測。 ⏱TPOT：144.85ms → 99.49ms（31.3%短縮） 🔥MoEのexpert行列積：22.8ms → 6.2ms（3.68倍） TP＋EPで演算コアの活用を促し、Operator Fusionで複数演算を1カーネル化。ハードウェアは変えず、ソフトウェア最適化だけで実用的な速度に引き上げました。詳細は本文をダウンロードしてご確認ください。 https://t.co/uINAEGEFzg #Tenstorrent

Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, quantization, context length).

DGX Spark x2 × DeepSeek-V4.1-Flash 1017

Oct 21, 2026 · by @Blackwellboy

Context	Quant	Engine	Decode	Prefill
128K	—	—	—	2008.0 tok/s
—	—	—	113.0 tok/s	—

BLACKFOLD IS LIVE. 🤯 Uncensored DeepSeek-V4.1-Flash 1017 running locally on two DGX Sparks. No restrictions. Blazing speeds: ⚡️ 2,008 tok/s prefill (128K context) 💻 113 tok/s code decode Virtually zero refusals matching api quality. Everything I fixed to get there 🧵 https://t.co/L9ZNbvH3xB

Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, quantization, hardware state, measurement method).

B300 x4 × Qwen3.8-27B

Oct 21, 2026 · by @Joses_Iker

Context	Quant	Engine	Decode	Prefill
—	—	vllm	961.6 tok/s	—
—	—	vllm	28000.0 tok/s	—

EAGLE-4 is a serious step up in speculative decoding. 961.6 tokens/s on Qwen3.8-27B (4× B300, single request) is roughly 4× vanilla. More importantly, it still holds an edge at high concurrency (28k+ aggregate tokens/s at 512 requests) and delivers strong gains on AMD hardware and even a stock phone CPU. The phone numbers are the most striking part: 3.6× on HumanEval with a 4-bit Gemma-4-E2B and lower energy per…

Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, context length, hardware state, measurement method).

DGX Spark x2 × DeepSeek v4.1 1017

Oct 21, 2026 · by @WescheNex1q

Context	Quant	Engine	Decode	Prefill
131072	—	exllamav2	43.0 tok/s	—

DeepSeek v4.1 1017 by @Blackfrost_AI Refusal-reduced build, quantized to run on 2 DGX Sparks. 204 GB · 88.0 score on Spark-Bench v7.0 (stock 1017: 88.6) · 43 tok/s code · 128K ctx exllamav3 can't convert DeepSeek V4.1 Flash out of the box: drops Engram, skips the ratio-1 indexers, pads 3 compressors wrong. Five converter patches + a repair script later Weights, patches, repair script all in the repo…

Digest-only: confidence 0.00 below 0.60; incomplete methodology (engine+version, quantization, context length, hardware state, measurement method).

GB200 × DeepSeek-V4.1-Flash

Oct 21, 2026 · by @vllm_project

Context	Quant	Engine	Decode	Prefill
—	—	vllm	826.0 tok/s	—

🚨🚨🚨 vLLM is all you need to reproduce DeepSeek’s performance. With Wide-EP and EPLB, vLLM can match DeepInfer’s decode performance within 6% and generate $39/gpu/hr in revenue at DeepSeek’s off-peak pricing, at 200 P90 TPS on the AgentX workload. Specifically: • Under DeepSeek’s decode configuration (DeepSeek-V4.1-Flash, 131K context, nextn=3), vLLM on GB200 achieves throughput within 6% of DeepInfer’s published…

Digest-only: confidence 0.55 below 0.60; incomplete methodology (engine+version, quantization, hardware state, measurement method).

RTX 3090 × Qwen3.8-27B

Oct 21, 2026 · by @Oluwaphilemon1 · third-party report

Context	Quant	Engine	Decode	Prefill
—	—	—	82.0 tok/s	—
—	—	—	114.0 tok/s	—
—	—	—	138.0 tok/s	—
—	—	—	381.0 tok/s	—
—	—	—	133.0 tok/s	—

Qwen3.8-27B reportedly went from 82 to 381 tokens/sec on a single RTX 3090 with 24GB VRAM. No GPU upgrade. The gains came from optimizing the inference stack. Performance progression: Baseline: 82 tok/s Optimized MTP: 114 tok/s DFlash2 + context lookup: 138 tok/s Optimized lookup verification: 381 tok/s That’s roughly 4.6× the original throughput, but there’s an important catch: the 381 tok/s result came from…

Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, quantization, context length, hardware state, measurement method).

B300 x4 × Qwen 27B

Oct 20, 2026 · by @0xCVYH

bruto! tem que tirar o chapéu, quase 1k tok/s no Qwen 27b em 4 b300

Digest-only: confidence 0.40 below 0.60; incomplete methodology (engine+version, quantization, context length, hardware state, measurement method).

DGX Spark x2 × DeepSeek-V4.1-Flash-1017

Oct 20, 2026 · by @fumi_KPFstudio

Context	Quant	Engine	Decode	Prefill
256K	EXL3 2.9bpw	tensorfold	—	2000.0 tok/s
256K	EXL3 2.9bpw	tensorfold	—	1800.0 tok/s
256K	EXL3 2.9bpw	tensorfold	95.0 tok/s	—
256K	EXL3 2.9bpw	tensorfold	145.0 tok/s	—
512K	EXL3 2.9bpw	tensorfold	155.0 tok/s	—
512K	EXL3 2.9bpw	tensorfold	10.0 tok/s	—

DeepSeek-V4.1-Flash-1017の2×dgx sparkで、実用的な256kや512kで、速度や並列性能を確認したので共有します！ Mia最新レシピで非検閲との比較も同時にしています。 TP4のMiaのdeepseek v4.1flashレシピから変えたら、prefillは半分くらいになったけど、decodeはほとんど変わらないか、むしろ早いのがすごい！速度の頭打ちは8並列あたりでした。・Spark1+2：通常版・Spark3+4：非検閲版（Abliterated）【1枚目】256K文脈・4並列・prefill：通常版約2,000 tok/s、非検閲版約1,800 tok/s ・コード生成：1本で約91〜99 tok/s、4並列の合計で約145 tok/s 【2枚目】512K文脈・16並列設定・16並列まで安定、エラー0 ・合計速度は8並列あたりで約155 tok/sが頭打ち…

Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, hardware state, measurement method).

Also in this window

The 12 most recent posts are shown in full above; 31 posts from the same window are listed here in short form, each with its source post.

Unspecified platform — 961.6 tok/s decode (3 measurements) · @thedailyblock
K420 × Bonsai 2B Ternary — 22.9 tok/s decode (1 measurement) · @amirhusain_tx
M5 Pro 64GB × Qwen3.8-Flash-Next — 171.0 tok/s decode (3 measurements) · @jianchen1799
B300 x4 × Qwen 3.8 27B — 961.6 tok/s decode (3 measurements) · @mattknowsai
Mac Studio M5 Ultra 256GB × DeepSeek-V4.1-Flash-1017 — 63.8 tok/s decode (2 measurements) · @niklaslenz_ai
AMD Instinct MI355X × Qwen3.8-27B — 300.0 tok/s decode (1 measurement) · @AIatAMD
DGX Spark × Qwen3.8-Flash-Next — 42.8 tok/s decode (1 measurement) · @niklaslenz_ai
RTX 3090 × Ornith-1.5-122B — 27.0 tok/s decode (4 measurements) · @ItsmeAjayKV
DGX Spark × Ornith-1.5-122B — 30.8 tok/s decode (3 measurements) · @filicroval
Wally × DeepSeek-v4.1-flash — 615.0 tok/s decode (1 measurement) · @RunAnywhereAI
RTX 5060 Ti × Qwen3.8-27B — 52.9 tok/s decode (1 measurement) · @niklaslenz_ai
Unspecified platform × Qwen3.8-Flash-Next-NVFP4 — 142.2 tok/s decode (2 measurements) · @AlicanKiraz0
CMP 170HX x4 × DeepSeek-V4.1-Flash-1017 — 671.0 tok/s decode (9 measurements) · @seanphan
TPU v7 × Kimi K3 — 391.0 tok/s decode (1 measurement) · @inferact
DGX Spark x2 × RED-SNOW-5.3-Flash — 45.0 tok/s decode (2 measurements) · @justinn_builds
RTX 5090 × Qwen3.8-27B — 149.0 tok/s decode (1 measurement) · @niklaslenz_ai
DGX Spark × qwen3.8 flash next — 1568.0 tok/s prefill (2 measurements) · @filicroval
DGX Spark × Qwen3.8-Flash-Next-FP4-Local — 297.0 tok/s decode (9 measurements) · @vr8vr8
M3 Ultra × DeepSeek V4.1 Flash — 98.0 tok/s decode (3 measurements) · @jundotkim
RTX 3090 × Qwen3.8 Flash-Next 2-bit (S / S-small, Strata format) — 109.0 tok/s decode (10 measurements) · @superalesha
RTX 3090 × DeepSeek-v4.1-flash-1017 — 4.0 tok/s decode (5 measurements) · @ItsmeAjayKV
DGX Spark x4 × DeepSeek-V4.1-Flash-1017 — 65.0 tok/s decode (3 measurements) · @niklaslenz_ai
Unspecified platform × Nemotron 3.5 Super VL — 112.0 tok/s decode (2 measurements) · @0xBakeer
Unspecified platform × GLM 5.3 Maya-L — 55.0 tok/s decode (1 measurement) · @gospaceport
DGX Spark x2 × DeepSeek-V4.1-Flash-1017 — 103.6 tok/s decode (5 measurements) · @sfxnz
Unspecified platform — 710.0 tok/s prefill (2 measurements) · @witcheer
DGX Spark x4 × DeepSeek V4.1 Flash — 93.6 tok/s decode (2 measurements) · @plotarmordev
Unspecified platform — 250.0 tok/s decode (1 measurement) · @zephyr_z9
Unspecified platform × Qwen3.8-27B Coder390 — 180.0 tok/s decode (1 measurement) · @Oluwaphilemon1
RTX 3060 12GB × Qwen3.5-9B — no throughput stated · @CY160909
DGX Spark × DeepSeek V4.1 Flash 1017 — 107.0 tok/s decode (3 measurements) · @durantfan5789

How to read this page

This digest is an automated transcription of public posts. Runs with a stated engine build, quantization, context length, hardware state and measurement method are published straight to the benchmark board with a link to their source; the rest stay here. These are author-reported numbers, not an independent DiggTop re-measurement: every row carries its provenance so you can check the original, and a number that turns out to be misreported is corrected or removed. Our own testing follows the methodology. Spotted an error? Reply on the source post or submit a corrected run at /submit.

Sources

FAQ

Are the numbers in this digest sourced?

They are transcribed from public X posts and the repositories those posts link to, with attribution and a source link on every entry. Rows that state the full methodology are published to the benchmark board automatically, so treat them as author-reported until a dispute is raised: the source link on every row is the check.

What does it take for a row to reach the sourced tier?

Engine and exact version, quantization, context length, hardware state and a described measurement method. Posts that state all five are published to the benchmark board as soon as the collector reads them, each row carrying its source link — nothing waits in a human review queue. Anything missing a condition stays in this digest.

Where does the data come from?

Public X posts about local-model inference measured on owned hardware, discovered by keyword search every two hours and linked back to the original thread. The digest is regenerated daily.