DiggTop
Menu
⌕

X local-model bench digest — Oct 30, 2026

As of Oct 30, 2026 — 37 posts collected from X. 109 measured runs catalogued, 2 published to the board, 35 kept to the digest only. Latest source post Oct 30, 2026 09:23 UTC.

Every number below is what its author published, transcribed with the engine, quantization and context length they reported. Sources are linked on each entry. Only posts that state the full methodology (engine build, quantization, context length, hardware state, measurement method) reach the benchmark board; the rest stay here as leads.

RTX 3090 x4 × GLM-5.3-Flash

Oct 30, 2026 · by @superalesha

Context Quant Engine Decode Prefill
— — — 58.0 tok/s —

Sam Altman says local inference is a terrible idea. Japanese companies did not get the memo. A consultant in Japan says he now gets requests to deploy local LLMs almost every day. His reason: GLM-5.3-Flash and DeepSeek V4.1 Flash 1017 hit near frontier quality. Both are Chinese. Both are open weights. Both run on hardware a company can own. That is what the cloud guys are scared of. Not that local is slow. That it…

Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, quantization, context length, hardware state, measurement method).

RTX 5060 Ti × Qwen3.8 Flash Next

Oct 30, 2026 · by @QwenAIFans · third-party report

Context Quant Engine Decode Prefill
— IQ3_S strata 65.3 tok/s 1810.0 tok/s

💥16GB 的 5060 Ti，本地塞进了 Qwen3.8 Flash Next Strata 跑 IQ3_S：解码约 65.3 tok/s，预填充约 1810。截图显存大约 15GB。 中端卡也能摸到这速度。你本机 Flash Next 能到多少？ 来源：@t28k2👇 https://t.co/LwBq2pherd

Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, context length, measurement method).

M5 Pro Mac mini 64GB × Le Chonk (Mistral Large 4)

Oct 30, 2026 · by @pratikg

Context Quant Engine Decode Prefill
— 1-bit (Unsloth) — 0.3 tok/s 3.0 tok/s

🟠 RESULTS: I ran a 1 trillion parameter model on my 64GB M5 Pro Mac mini. Le Chonk by @MistralAI (Mistral Large 4, 1T total, 52B active), using @UnslothAI’s 1-bit quant. That’s 233GB of weights on a machine with 64GB of memory, so it streamed straight off the SSD. I guessed 2-4 tok/s. I was wrong. - 0.35 tok/s decode, about 2.9 seconds per token - 3.0 tok/s prefill - ~57GB peak memory - 3 runs, all within 0.01 tok/s…

Digest-only: confidence 0.00 below 0.60; incomplete methodology (engine+version, quantization, context length, hardware state, measurement method).

NVIDIA B300 × deepseek-v4.1-flash-ultrafast

Oct 30, 2026 · by @rlaope

Context Quant Engine Decode Prefill
— — — 848.0 tok/s —
— — — 1000.0 tok/s —

Ahead of DeepSeek V5, we're running another discount round on deepseek-v4.1-flash-ultrafast. Fable-class work at 1/30 the price, 12x the speed. That's the line on our site, and it's still true. - 848 tok/s is the number we publish. Users are seeing 1,000+ tok/s sustained on long generations. - 1M context, up to 64K output - Served on our own NVIDIA B300s, live on OpenGateway - OpenAI and Anthropic compatible. Swap…

Digest-only: confidence 0.55 below 0.60; incomplete methodology (engine+version, quantization, context length, hardware state, measurement method).

B300 × DeepSeek V4.1 Flash

Oct 30, 2026 · by @rlaope

Context Quant Engine Decode Prefill
— — — 1000.0 tok/s —

i serve deepseek v4.1 flash ultrafast at 1000 tok/s on top of b300s. straight up: i slapped insane discounts on it just to win community favor and build brand mindshare. would love for you guys to push it hard. already burned hundreds of billions of tokens on my own b300 rigs, hope y'all enjoy running it just as much. web: https://t.co/IdiJr1VGzh #deepseek #og

Digest-only: confidence 0.40 below 0.60; incomplete methodology (engine+version, quantization, context length, hardware state, measurement method).

GX10 x4 × TensorFold DeepSeek V4.1 Flash 1017

Oct 30, 2026 · by @Q_Beaux

Context Quant Engine Decode Prefill
— FP8/MXFP4 — 35.6 tok/s —
— FP8/MXFP4 — 56.0 tok/s —

Running @MiaAI_lab TensorFold DeepSeek V4.1 Flash 1017 on 4× GX10s with the full official FP8/MXFP4 weights. Pure switch on the CRS812 was stable but limited me to ~35.6 tok/s. Each CX-7 port is PCIe Gen5 x4, so we were effectively on one rail (~110 Gb/s). The recipe’s published 65.8 came from a dual-rail switchless ring. I didn’t have a fourth direct cable, so we built a hybrid: 3× ordinary QSFP DACs: 1↔2, 2↔3, 3↔4…

Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, context length, measurement method).

M5 Max 36GB × Qwen3.8-27B

Oct 30, 2026 · by @Oluwaphilemon1 · third-party report

Context Quant Engine Decode Prefill
— 4-bit mlx 25.3 tok/s —
— 4-bit lmstudio 26.0 tok/s —
— 4-bit mlx 37.9 tok/s —
— 4-bit lmstudio 69.7 tok/s —
— 4-bit mlx 75.3 tok/s —
— 4-bit mlx 82.8 tok/s —

Same Mac. Same Qwen3.8-27B 4-bit weights. A different inference engine. And the reported generation speed jumps from around 25 tokens per second to almost 38. A new open-source project called Rapid-MLX is targeting local AI inference on Apple Silicon, with a particular focus on running coding agents directly on Macs. In its M5 Max 36GB benchmark, the reported single-stream decode speeds are: oMLX: 25.3 tok/s LM…

Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, context length, hardware state, measurement method).

M5 Max MacBook Pro × Qwen3.8-27B

Oct 30, 2026 · by @Oluwaphilemon1 · third-party report

Context Quant Engine Decode Prefill
— — inco splash 144.0 tok/s —

Qwen3.8-27B is reportedly reaching 144 tokens per second on an M5 Max MacBook Pro using a new open-source inference engine called Inco Splash. That’s a notable result for anyone interested in running capable models locally without relying on a cloud API. The project reports these performance comparisons: Up to 3× faster decoding than Ollama Up to 2× faster than oMLX Nearly 4× the throughput when an agent delegates…

Digest-only: confidence 0.55 below 0.60; incomplete methodology (engine+version, quantization, context length, hardware state, measurement method).

RTX 3060 × Qwen3.8-35B-A3B

Oct 30, 2026 · by @Oluwaphilemon1 · third-party report

Context Quant Engine Decode Prefill
262K Q4_K_M llama.cpp 50.0 tok/s 550.0 tok/s

Qwen3.8-35B-A3B is reportedly generating around 50 tokens/s on an RTX 3060 with just 12GB VRAM and 16GB system RAM. For a model in the 35B parameter class, that’s an interesting result, especially because the goal isn’t just running a chatbot. The developer wants to use it as a local coding agent. The reported performance: GPU: RTX 3060, 12GB VRAM System RAM: 16GB Generation: ~50 tokens/s Prompt processing: ~550…

Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, measurement method).

Intel Arc Pro B65 x2

Oct 30, 2026 · by @murasametech

Context Quant Engine Decode Prefill
— — strata 0.2.8 72.5 tok/s —
— — strata 0.2.8 81.5 tok/s —
— — strata 0.2.8 370.0 tok/s —
— — strata 0.2.8 20.0 tok/s —

Strata 0.2.8 リリース！ 🚀 Intel Arc B-series 正式対応＋高速化＆4×R9700並列強化、Mac DeepSeek安定化など多数の改善！ 主な変更点： 2×Intel Arc Pro B65：decode +17〜23%、prompt +7〜13%（Chat 71-74 tok/s、JSON 80-83 tok/s） 4×Radeon AI PRO R9700：32同時リクエストで総スループット 2.4倍（155→370 tok/s） Mac DeepSeek：メモリリーク修正（リクエスト毎1.3GBリークが解消、安定して18-22 tok/s） Tool call復元強化：正常認識率 92.9% → 98.3% 12GB以下RTXのrouting高速化をデフォルトOFFに戻し（長時間会話の繰り返し問題対策） DeepSeekの単語途中切れ修正 AMD batch window拡大（8→16…

Digest-only: confidence 0.55 below 0.60; model.name missing; incomplete methodology (quantization, context length, hardware state, measurement method).

RTX 4090 x2 × Qwen3.8-27B-INT8-W8A16-MTP

Oct 29, 2026 · by @Oluwaphilemon1 · third-party report

Context Quant Engine Decode Prefill
128K INT8-W8A16 vllm 300.0 tok/s —

Two RTX 4090 GPUs running Qwen3.8-27B at over 300 tokens/s aggregate throughput, with eight concurrent sessions. That’s more than 1 million tokens of combined generation capacity per hour for locally hosted AI agents, without relying on paid model subscriptions or external API providers. Here’s the setup: GPUs: 2 × NVIDIA RTX 4090 Model: Qwen3.8-27B-INT8-W8A16-MTP Inference engine: vLLM Concurrent sessions: 8…

Digest-only: confidence 0.00 below 0.60; incomplete methodology (engine+version, quantization, context length, hardware state, measurement method).

RTX 3090 × Qwen3.8-27B

Oct 29, 2026 · by @Oluwaphilemon1 · third-party report

Context Quant Engine Decode Prefill
— — vllm 127.0 tok/s —
— — vllm 1035.0 tok/s —

A single RTX 3090 can reportedly run Qwen3.8-27B at over 1,000 tokens/s aggregate throughput, serving 64 concurrent sessions. No multi-GPU setup. No expensive server hardware. Just one 24GB RTX 3090, capped at 250W. The project behind this is HyperQwen, an open-source inference setup built around vLLM. Its reported performance: • GPU: RTX 3090, 24GB VRAM • Power limit: 250W • Single-session generation: approximately…

Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, quantization, context length, measurement method).

Also in this window

The 12 most recent posts are shown in full above; 25 posts from the same window are listed here in short form, each with its source post.

How to read this page

This digest is an automated transcription of public posts. Runs with a stated engine build, quantization, context length, hardware state and measurement method are published straight to the benchmark board with a link to their source; the rest stay here. These are author-reported numbers, not an independent DiggTop re-measurement: every row carries its provenance so you can check the original, and a number that turns out to be misreported is corrected or removed. Our own testing follows the methodology. Spotted an error? Reply on the source post or submit a corrected run at /submit.

Sources

FAQ

Are the numbers in this digest sourced?

They are transcribed from public X posts and the repositories those posts link to, with attribution and a source link on every entry. Rows that state the full methodology are published to the benchmark board automatically, so treat them as author-reported until a dispute is raised: the source link on every row is the check.

What does it take for a row to reach the sourced tier?

Engine and exact version, quantization, context length, hardware state and a described measurement method. Posts that state all five are published to the benchmark board as soon as the collector reads them, each row carrying its source link — nothing waits in a human review queue. Anything missing a condition stays in this digest.

Where does the data come from?

Public X posts about local-model inference measured on owned hardware, discovered by keyword search every two hours and linked back to the original thread. The digest is regenerated daily.

↑