DiggTop
Menu
⌕

X local-model bench digest — Nov 4, 2026

As of Nov 4, 2026 — 22 posts collected from X. 57 measured runs catalogued, 1 published to the board, 21 kept to the digest only. Latest source post Nov 4, 2026 10:53 UTC.

Every number below is what its author published, transcribed with the engine, quantization and context length they reported. Sources are linked on each entry. Only posts that state the full methodology (engine build, quantization, context length, hardware state, measurement method) reach the benchmark board; the rest stay here as leads.

Mac (large-memory host) + M5 Max × Qwen3.8-Flash-Next

Nov 4, 2026 · by @QwenAIFans · third-party report

Context Quant Engine Decode Prefill
65536 — mlx v0.7.4.dev1 — 1309.0 tok/s
65536 — mlx v0.7.4.dev1 — 3666.0 tok/s

💥两台 Mac 串起来，Qwen3.8-Flash-Next 预填充 1309→3666 tok/s。 oMLX Prefill Accelerator：大内存机扛模型，M5 Max 只加速预填充，64K 约 +180%。Decode 不变。 温馨提示：开发版；作者本机。 来源：GitHub oMLX👇 https://github.com/jundot/omlx/releases/tag/v0.7.4.dev1 https://t.co/8ojuIF4VIg

Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (quantization, hardware state, measurement method).

RTX 3090 × Qwen3.8-Flash-Next

Nov 4, 2026 · by @QwenAIFans · third-party report

Context Quant Engine Decode Prefill
— — bloomery 83.4 tok/s —
— — bloomery 74.3 tok/s —

🔥单卡3090跑 Qwen3.8-Flash-Next，decode ~83 tok/s。 bloomery 让 experts 在 GPU、内存、NVMe 间搬家。Status：UD-Q4_K_XL，83.36 / 74.34。 温馨提示：256GB 内存，3090@250W。 来源：bloomery👇 https://github.com/midagedev/bloomery https://t.co/fZR891CJfM

Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, context length, measurement method).

V100 32GB x2 × Qwen3.8-Flash-Next

Nov 4, 2026 · by @pentacoxian_dev

Context Quant Engine Decode Prefill
— IQ3E-Q8D-MTP llama.cpp pentaCoxian/Qwen-V100-llama.cpp fork — 2400.0 tok/s
— IQ3E-Q8D-MTP llama.cpp pentaCoxian/Qwen-V100-llama.cpp fork 180.0 tok/s —

My llama.cpp fork for 2× V100 32GB now supports plain PCIe (no NVLink) 🎉 Run Qwen3.8-Flash-Next (IQ3E-Q8D-MTP), same model, but now faster prefill at up to 2400tok/s and 90~180 tok/s decode on dual V100 (x8 PCIe). Repo: https://github.com/pentaCoxian/Qwen-V100-llama.cpp #LocalLLM #V100 #QwenFN

Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (context length, hardware state, measurement method).

DGX Spark x4 × GLM-5.3

Nov 4, 2026 · by @ViC305 · third-party report

Context Quant Engine Decode Prefill
1M — tensorfold 69.2 tok/s —
1M — tensorfold 55.7 tok/s —
1M — tensorfold 73.6 tok/s —

🔥 @cfontes got the full 753B GLM‑5.3 generating code at 73.6 tokens/sec on four DGX Sparks at 1M CTX. 🚀 That’s the FASTEST single-stream code result I found across the four public recipes I compared. When I released my 3.38 bpw SAGE‑EXL3 quant, we had all 19,200 routed experts in 319 GB. Chris funded the compute himself, then spent four weeks working on what happens after the weights load. Here’s what his TensorFold…

Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, hardware state).

RTX 5090 (simulated 12GB) × Qwen3.8-Flash-Next-12G32G-GGUF

Nov 4, 2026 · by @QwenAIFans · third-party report

Context Quant Engine Decode Prefill
— — llama.cpp 56.0 tok/s —

💥 12G 显卡跑 Qwen3.8-Flash-Next：每层砍 64 专家，解码反而 56 tok/s。 L448 自炼版比 Unsloth：解码 +30%，8K 预填 +33%，还小 5GB。空回复靠校准时加入自家 chat 轮次修好。 温馨提示：5090 模拟 12G，作者数字。 来源：Hugging Face zephel01👇 https://huggingface.co/zephel01/Qwen3.8-Flash-Next-12G32G-GGUF https://t.co/IC2CXtibHj

Digest-only: confidence 0.55 below 0.60; incomplete methodology (engine+version, quantization, measurement method).

Intel B70 x2 × Qwen3.8-27B

Nov 3, 2026 · by @QwenAIFans · third-party report

Context Quant Engine Decode Prefill
— — — 109.0 tok/s 1700.0 tok/s

💥 Dual Intel B70 (32GB each) just hit 109 tok/s decode on Qwen3.8-27B. 1,700 tok/s prefill. 64GB VRAM. One local box, no cloud bill. Caveat: his run, not ours. Still wild for affordable dual cards. via @tolewis 👇 https://t.co/dQ1u5Vz3gE

Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, quantization, context length, hardware state, measurement method).

H100 × 400B model

Nov 3, 2026 · by @RoyRinberg

Context Quant Engine Decode Prefill
— — flashprove — 100.0 tok/s

Sometimes, all you have to do is ask. We are open-sourcing a research prototype which we hope leads to more progress in this field. FlashProve is open source and proves a 400B model at over 100 tok/s on one H100 in prefill. https://x.com/AnthropicAI/status/2117710942826123294?a=YW8M1x3fQrxkTNhD5eM&s=20

Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, quantization, context length, hardware state, measurement method).

M3 Ultra x2 × Mistral Large 4

Nov 3, 2026 · by @Prince_Canuma

Context Quant Engine Decode Prefill
— — mlx 30.0 tok/s —

Well, we made it even faster! Mistral Large 4 (1T A52b) now running at ~30 tok/s with DSpark on MLX-VLM using 2 x M3 Ultras. Release is coming out today with lots of improvements 🚀 https://t.co/QUcYf1PKnB

Digest-only: confidence 0.00 below 0.60; incomplete methodology (engine+version, quantization, context length, hardware state, measurement method).

DGX Spark × Ling-3.0-flash

Nov 3, 2026 · by @Marktechpost · third-party report

Context Quant Engine Decode Prefill
— — — 37.8 tok/s —
— — — 53.9 tok/s —

▲ NVIDIA just published 3 ways to speed up MoE inference on a single DGX Spark, tested on Ant Group inclusionAI's 124B Ling-3.0-flash. The key idea: at batch size 1, decoding is memory-bound. Reading weights, not math, sets the pace. So NVIDIA cut the bytes moved per step (Humming MoE kernels, an FP8 LM head) and the steps per answer (built-in MTP speculative decoding). On all 1,319 GSM8K questions, decode…

Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, quantization, context length, hardware state).

RTX 5090 × MiniMax-M3.1-Flash

Nov 3, 2026 · by @niklaslenz_ai

Context Quant Engine Decode Prefill
— IQ1_S (~1.6 bpw) maya 1.1.50 24.0 tok/s —
— IQ1_S (~1.6 bpw) maya 1.1.50 24.4 tok/s —
— IQ1_S (~1.6 bpw) maya 1.1.50 23.6 tok/s —
249K IQ1_S (~1.6 bpw) maya 1.1.50 18.7 tok/s —

MiniMax-M3.1-Flash (428B) decodes at 24.0 tok/s on one RTX 5090 plus 92 GiB RAM, and 18.7 with a 249K prompt RTX 5090, 32 GB, plus 92 GiB RAM Decode, code and prose: 24.0 tok/s Decode, code: 24.4 tok/s Decode, prose: 23.6 tok/s 500 words (code and prose): about 27.5 s First token at 16K: 36.6 s First token at 128K: 465.8 s Decode with a 249K prompt: 18.7 tok/s Project Maya 1.1.50, Maya-XS (93 GB, IQ1_S experts at…

Full methodology stated (engine, quantization, context, hardware state, method) — 1 added to the benchmark board (1 of 4 measured rows queued; the rest are published here only).

R9700 × Qwen3.8-27B

Nov 3, 2026 · by @QwenAIFans · third-party report

Context Quant Engine Decode Prefill
— — — 36.0 tok/s —
— — — 92.0 tok/s —

💥 One R9700 took Qwen3.8-27B from ~36 to 88-96 tok/s. MTP k=3 + a few RDNA4 Vulkan patches. Code/JSON hit the high end. Caveat: stock used a different quant, so whole-setup, not one knob. Still parking AMD cards? via Chris Novick 👇 https://chrisnovick.com/p/r9700-amd-inference https://t.co/aLfxq5DcHT

Digest-only: confidence 0.55 below 0.60; incomplete methodology (engine+version, quantization, context length, hardware state, measurement method).

B200 × OCRia-1

Nov 3, 2026 · by @jerryjliu0

Context Quant Engine Decode Prefill
— — vllm 285.0 tok/s —
— — vllm 2800.0 tok/s —

OCRia-1 is one of the fastest models ever for doc parsing. We sped up OCRia-1 by 10x on a single B200 (285 tok/s -- 2800 tok/s). there are a ton of useful learnings in this extremely detailed blog post for anyone looking to optimize models! * we made use of established practices, like quantization and spec decoding (dflash2), to simultaneously reduce memory usage and increase tokens per second * we made 17 other…

Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, context length, hardware state, measurement method).

Also in this window

The 12 most recent posts are shown in full above; 10 posts from the same window are listed here in short form, each with its source post.

How to read this page

This digest is an automated transcription of public posts. Runs with a stated engine build, quantization, context length, hardware state and measurement method are published straight to the benchmark board with a link to their source; the rest stay here. These are author-reported numbers, not an independent DiggTop re-measurement: every row carries its provenance so you can check the original, and a number that turns out to be misreported is corrected or removed. Our own testing follows the methodology. Spotted an error? Reply on the source post or submit a corrected run at /submit.

Sources

FAQ

Are the numbers in this digest sourced?

They are transcribed from public X posts and the repositories those posts link to, with attribution and a source link on every entry. Rows that state the full methodology are published to the benchmark board automatically, so treat them as author-reported until a dispute is raised: the source link on every row is the check.

What does it take for a row to reach the sourced tier?

Engine and exact version, quantization, context length, hardware state and a described measurement method. Posts that state all five are published to the benchmark board as soon as the collector reads them, each row carrying its source link — nothing waits in a human review queue. Anything missing a condition stays in this digest.

Where does the data come from?

Public X posts about local-model inference measured on owned hardware, discovered by keyword search every two hours and linked back to the original thread. The digest is regenerated daily.

↑