X local-model bench digest — Oct 16, 2026
As of Oct 16, 2026 — 27 posts collected from X. 54 measured runs catalogued, 27 kept to the digest only. Latest source post Oct 16, 2026 10:42 UTC.
Every number below is what its author published, transcribed with the engine, quantization and context length they reported. Sources are linked on each entry. Only posts that state the full methodology (engine build, quantization, context length, hardware state, measurement method) reach the benchmark board; the rest stay here as leads.
RTX 3090 × GLM-5.3-Flash
Oct 16, 2026 · by @ItsmeAjayKV
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| 256K | — | llama.cpp | 22.5 tok/s | — |
| 256K | — | llama.cpp | — | 900.0 tok/s |
ちなみに、GLM-5.3-FlashがRTX 3090でもローカルで動くようになりました！ 今、私の環境で動かしています。 RTX 3090 + 64GB RAM + NVMe SSD Quant AJ-IQ2_XXS: https://t.co/9LRxs2TtHw Decode: 20 - 25 tok/s Prefill: 800 - 1000 tok/s Context: 256k Project-Mayaのおかげです！ありがとうございます！ 自分のPCで動くので、プライバシーも守れます。 電気代以外はお金もかかりません！
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, hardware state, measurement method).
RTX 5070 12GB + RTX 5060 Ti 16GB × Qwen3.8-Flash-Next
Oct 16, 2026 · by @murasametech
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| 4K | IQ2_XS | strata v0.1.43 | — | 750.0 tok/s |
| 14K | IQ2_XS | strata v0.1.43 | — | 1500.0 tok/s |
| 32K | IQ2_XS | strata v0.1.43 | — | 1750.0 tok/s |
2枚のGPU（RTX 5070 12GB＋RTX 5060 Ti 16GB）で、Strata v0.1.43 と Qwen3.8-Flash-Next IQ2_XS を「resident方式」で動かしてみました。 expertをRAMに常駐させる方式です。 長文を読ませたときの処理速度（prefill）は、4K tokで約750 tok/s、1.4万tokで約1,500 tok/s、3.2万tokで約1,750 tok/sでした。 以前IQ3_Sを同じ方式で動かしたときは、長文のたびにSSDから約88GBを読み直して、約150 tok/sまで落ちていました。（私の場合はRAM64GB, WSLなので条件が悪いです） 今回はSSDからの読み込みが1回あたり数十〜百数十MBで済み、その現象は起きませんでした。 ただしIQ3_Sの記録はStrataの版も量子化も違う参考値で、今回の計測も各1回です。…
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (hardware state, measurement method).
Unspecified platform × qwen
Oct 16, 2026 · by @sudoingX · third-party report
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | — | — | 50.1 tok/s | — |
i had to check the file size twice prismml dropped bonsai 2 at 2b, 4b and 9b today, ternary, built on qwen 3.5, and the 9b is a 2gb file, the qwen 3.5 9b it came from is 17.9gb the acceleration in open source right now is insane, bonsai 2 27b came out a month ago at 5.95gb and it's already at 4.8m downloads, i run it on a 3060 12gb at 50.1 tok/s with my kernel, this week alone i counted 11 releases from a 5M…
Digest-only: confidence 0.35 below 0.60; hardware.name missing; incomplete methodology (engine+version, quantization, context length, hardware state, measurement method).
RTX 3060 × Qwen3.8-35B-A3B
Oct 16, 2026 · by @Oluwaphilemon1 · third-party report
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| 262K | Q4_K_M | llama.cpp | 50.0 tok/s | — |
| 262K | Q4_K_M | llama.cpp | — | 550.0 tok/s |
Qwen3.8-35B-A3B is reportedly generating around 50 tokens/s on an RTX 3060 with just 12GB VRAM and 16GB system RAM. The same setup reportedly reaches approximately 550 tok/s in prompt processing, using a Q4_K_M quantization and a configured 262K context window. Here’s the setup: GPU: RTX 3060, 12GB VRAM System RAM: 16GB Quantization: Q4_K_M Generation: ~50 tok/s Prompt processing: ~550 tok/s Context: 262K tokens…
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, hardware state, measurement method).
RTX 3060 × Nemotron-3.5-Bolt-74B-A9B
Oct 16, 2026 · by @Oluwaphilemon1 · third-party report
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| 262144 | 2-bit GGUF | llama.cpp | 24.0 tok/s | — |
| 262144 | 2-bit GGUF | llama.cpp | — | 500.0 tok/s |
NVIDIA’s Nemotron-3.5-Bolt-74B-A9B is reportedly running on a single RTX 3060 with just 12GB VRAM and 16GB system RAM. The reported performance: • Decode: ~24 tokens/s • Prefill: ~500 tokens/s • Context window: 262K tokens • GPU: RTX 3060, 12GB VRAM • System RAM: 16GB NVIDIA’s smallest recommended setup reportedly calls for 32GB of RAM. This configuration uses half that amount while running the full context window…
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, hardware state, measurement method).
RTX 4090 × Qwen3.8-Flash-Next
Oct 16, 2026 · by @Oluwaphilemon1 · third-party report
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| 80K | Q4_K_XL | llama.cpp | 22.5 tok/s | 384.0 tok/s |
| 80K | Q4_K_XL | llama.cpp | 20.8 tok/s | 356.0 tok/s |
| 180K | Q4_K_XL | llama.cpp | 21.0 tok/s | 358.0 tok/s |
| 250K | Q4_K_XL | llama.cpp | 21.0 tok/s | 364.0 tok/s |
A 125B-parameter model running on a single RTX 4090 with 24GB VRAM, generating around 21 tokens/s with a 250K context window. That’s what a developer reports achieving with Qwen3.8-Flash-Next, a mixture-of-experts model with approximately 6B active parameters per token. No MTP. No DFlash2. No KV-cache quantization. The trick is combining GPU acceleration with system RAM through an experimental llama.cpp build. The…
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version).
Unspecified platform × Qwen3.8-27B
Oct 16, 2026 · by @jtdavies · third-party report
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | — | mlx 1.4 | — | 742.0 tok/s |
TensorFold with Qwen3.8-27B is crazy fast! 5.4x decode vs plane mlx-lm (from Apple) 1.4-1.7x decode vs oMLX (the runner up) Prefill sucks on all of them at 566-742 tok/s (so use the KV-cache) The TensorFold lead is lost to oMLX+MTP with Qwen3.8-Flash-Next though. All run on https://t.co/tpD9WjYi7m
Digest-only: confidence 0.35 below 0.60; hardware.name missing; incomplete methodology (quantization, context length, hardware state, measurement method).
Vera Rubin NVL72 × DeepSeek V4 Pro
Oct 16, 2026 · by @ArtificialAnlys
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | — | vllm | 906.0 tok/s | — |
| — | — | vllm | 408.0 tok/s | — |
| — | — | vllm | 255.0 tok/s | — |
Initial results for NVIDIA's Vera Rubin NVL72 are now live on AA-AgentPerf, with up to ~5.7x the total throughput per megawatt of its predecessor GB300 NVL72 on DeepSeek V4 Pro, ~5.4x on DeepSeek V4.1 Flash, ~3.0x on GLM-5.2 and ~2.8x on Kimi K3 Vera Rubin is NVIDIA's successor to Blackwell, and Vera Rubin NVL72 is its rack-scale system, with 72 Rubin GPUs and 36 Vera CPUs. Each Rubin GPU has the same 288 GB of…
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, quantization, context length).
DGX Spark x3 × Muse Spark Open
Oct 16, 2026 · by @WescheNex1q
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | 2-bit GGUF | llama.cpp | 7.0 tok/s | — |
Had to start of with the classic voxel pagoda Meta's Muse Spark Open is 816B parameters. We ran Unsloth's 2‑bit GGUF (226 GB) across 3 DGX Sparks with llama.cpp RPC. -Voxel pagoda, thinking at xhigh effort, no cap: -15.6K tokens · 37 min · 7 tok/s · ran first try It works, but it's cropped and sparse. At 2‑bit, a 5‑tier pagoda and a torii gate is what you get. Faster engines (vLLM, SGLang) can't fit it on 4 Sparks…
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, context length, hardware state, measurement method).
DGX Station GB300 × Nemotron 3.5 Super VL
Oct 16, 2026 · by @SJRonanMD
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| 1M | NVFP4 | vllm stock open-source | 539.0 tok/s | — |
| 1M | NVFP4 | vllm stock open-source | 6199.0 tok/s | — |
| 262K | NVFP4 | vllm stock open-source | 202.0 tok/s | — |
Nemotron 3.5 Super VL on one DGX Station GB300, stock open-source vLLM (NVFP4, MTP, 1M context): • 539 tok/s single user • 0.19 s to first token on a fresh 8K prompt • 6,199 tok/s aggregate at 128 concurrent • GSM8K 98% On the Station's RTX PRO 6000 it does 202 tok/s single-user at 262K context, +37% over Nemotron 3 Super, and adds vision.
Digest-only: incomplete methodology (engine+version, hardware state, measurement method).
DGX Spark × MiniCPM4.7-35b
Oct 16, 2026 · by @ciruai
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| 63870 | 8-bit expert weights with FP16 s | vllm | 40.5 tok/s | 1968.6 tok/s |
| 63870 | 8-bit expert weights with FP16 s | vllm | 122.8 tok/s | — |
A treat for those of you who like high precision over pure speed. I've created a new custom build with @OpenBMB's excellent MiniCPM4.7 35b model! Hadamard-rotated 8-bit expert weights, with 256-weight groups and FP16 scales. The remaining weights keeping their original precision, mostly BF16 I also included several optimizations to make it very good at concurrent loads. The headline TPS numbers might not be as…
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, hardware state, measurement method).
RTX 3090
Oct 15, 2026 · by @PeasantSmith · third-party report
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | EXL3 | — | 22.0 tok/s | — |
Project Maya v1.1.12 is out • EXL3 on one GPU: the CPU takes more of the experts - an RTX 3090 goes from ~15 to ~22 tok/s, 29 with DFlash2 • Several requests on a multi-GPU split can now be batched (opt-in): +27% on four RTX 4000 Ada • The VRAM tier keeps following your chat while DFlash2 drafts • AMD RDNA 2 (RX 6800/6900, Radeon PRO V620) - experimental https://t.co/ELQ2xdft4A
Digest-only: confidence 0.35 below 0.60; model.name missing; incomplete methodology (engine+version, context length, hardware state, measurement method).
Also in this window
The 12 most recent posts are shown in full above; 15 posts from the same window are listed here in short form, each with its source post.
- DGX Spark × Qwen3.8 Flash-Next — 166.0 tok/s decode (2 measurements) · @sojufx
- RTX PRO 6000 × Qwen3.8-27B-MLA — 505.0 tok/s decode (2 measurements) · @wei_wang
- M4 Pro × Qwen3.8-Flash-Next — 25.0 tok/s decode (1 measurement) · @yuanhao
- DGX Spark × Nemotron 3.5 Super — 57.0 tok/s decode (5 measurements) · @filicroval
- M3 Ultra 512GB × Muse Spark Open — 21.0 tok/s decode (1 measurement) · @rohanpaul_ai
- RTX 3060 × Bonsai 2-9B — 114.0 tok/s decode (2 measurements) · @DogukanUrker
- M5 Pro × Qwen — 700.0 tok/s decode (1 measurement) · @barronnotbaron
- DGX Spark x2 × DeepSeek-V4.1 — 141.0 tok/s decode (2 measurements) · @sudoingX
- Unspecified platform × Qwen — 20.0 tok/s decode (1 measurement) · @Mikadzyki_NFT
- RTX 3060 × Nemotron-3.5-Bolt-74B-A9B — 24.0 tok/s decode (2 measurements) · @DogukanUrker
- Unspecified platform — 10500.0 tok/s decode (1 measurement) · @somewheresy
- RTX PRO 6000 × Nemotron 3.5 Super VL — 243.9 tok/s decode (2 measurements) · @sgl_project
- NVIDIA Blackwell GPU × Nemotron 3.5 Super VL — 1860.0 tok/s decode (4 measurements) · @LambdaAPI
- RTX 5070 × Qwen3.8-Flash-Next — 94.0 tok/s decode (2 measurements) · @QwenAIFans
- DGX Spark × Qwen3.8-27B — 49.9 tok/s decode (1 measurement) · @0xBakeer
How to read this page
This digest is an automated transcription of public posts. Runs with a stated engine build, quantization, context length, hardware state and measurement method are published straight to the benchmark board with a link to their source; the rest stay here. These are author-reported numbers, not an independent DiggTop re-measurement: every row carries its provenance so you can check the original, and a number that turns out to be misreported is corrected or removed. Our own testing follows the methodology. Spotted an error? Reply on the source post or submit a corrected run at /submit.
Sources
- @ItsmeAjayKV — Oct 16, 2026
- @murasametech — Oct 16, 2026
- @sudoingX — Oct 16, 2026
- @Oluwaphilemon1 — Oct 16, 2026
- @Oluwaphilemon1 — Oct 16, 2026
- @Oluwaphilemon1 — Oct 16, 2026
- @jtdavies — Oct 16, 2026
- @ArtificialAnlys — Oct 16, 2026
- @WescheNex1q — Oct 16, 2026
- @SJRonanMD — Oct 16, 2026
- @ciruai — Oct 16, 2026
- @PeasantSmith — Oct 15, 2026
- @sojufx — Oct 15, 2026
- @wei_wang — Oct 15, 2026
- @yuanhao — Oct 15, 2026
- @filicroval — Oct 15, 2026
- @rohanpaul_ai — Oct 15, 2026
- @DogukanUrker — Oct 15, 2026
- @barronnotbaron — Oct 15, 2026
- @sudoingX — Oct 15, 2026
- @Mikadzyki_NFT — Oct 15, 2026
- @DogukanUrker — Oct 15, 2026
- @somewheresy — Oct 15, 2026
- @sgl_project — Oct 15, 2026
- @LambdaAPI — Oct 15, 2026
- @QwenAIFans — Oct 15, 2026
- @0xBakeer — Oct 15, 2026
FAQ
Are the numbers in this digest sourced?
They are transcribed from public X posts and the repositories those posts link to, with attribution and a source link on every entry. Rows that state the full methodology are published to the benchmark board automatically, so treat them as author-reported until a dispute is raised: the source link on every row is the check.
What does it take for a row to reach the sourced tier?
Engine and exact version, quantization, context length, hardware state and a described measurement method. Posts that state all five are published to the benchmark board as soon as the collector reads them, each row carrying its source link — nothing waits in a human review queue. Anything missing a condition stays in this digest.
Where does the data come from?
Public X posts about local-model inference measured on owned hardware, discovered by keyword search every two hours and linked back to the original thread. The digest is regenerated daily.