X local-model bench digest — Oct 12, 2026
As of Oct 12, 2026 — 40 posts collected from X. 109 measured runs catalogued, 2 published to the board, 38 kept to the digest only. Latest source post Oct 12, 2026 12:13 UTC.
Every number below is what its author published, transcribed with the engine, quantization and context length they reported. Sources are linked on each entry. Only posts that state the full methodology (engine build, quantization, context length, hardware state, measurement method) reach the benchmark board; the rest stay here as leads.
RTX 5090 × GLM-5.3-Flash
Oct 12, 2026 · by @wei_wang
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | — | project maya | 30.0 tok/s | — |
| — | EXL3 4bit | tensorfold | 62.2 tok/s | — |
GLM-5.3-Flash 有 3,210 亿参数，现在一张 RTX 5090 加 96GB 内存就能跑到 30 tok/s 以上。做法是把 MoE 的专家按冷热分到显存、内存和 SSD 三层，常用的放显卡上，不常用的放远一点。 一、为什么专家可以拆开放 按 GLM-5.3-Flash 的 config：45 层，前 3 层是普通 FFN，后面 42 层每层 288 个路由专家，每个 token 只选 8 个，外加 1 个共享专家。 一个专家是三个 4096×2048 的矩阵，约 2,500 万参数。42 层加起来，路由专家约 3,040 亿，占了全部参数的九成以上。 每个 token 每层只用 288 个里的 8 个，不到 3%。GPU 不需要同时装下所有专家，只要用到的那几个能及时拿到。 二、三层存储，速度差几十倍 1. RTX 5090 显存 32GB，带宽 1,792 GB/s 2. 内存到显卡走 PCIe 5.0…
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, context length, measurement method).
M5 Max × Qwen-Image
Oct 12, 2026 · by @wquguru · third-party report
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | — | mlx 2.1 | 22.0 tok/s | — |
| — | — | mlx 2.1 | 92.0 tok/s | — |
本地部署模型离不开一些加速技巧，往浅里说，可以换 Turbo 之类的模型，往深里则可以做引擎优化。系统性整理了一下相关的案例： 1. macOS 本地部署 Qwen-Image 2.1，Turbo + MLX 跑 VAE + Q8 + 前缀 KV 缓存四招叠加，速度提升近 5 倍：https://t.co/u4bvFzsqqT 2. Apple Silicon 优先 MLX：M5 Max 上用 MFLUX 跑 Qwen-Image-2.1，1600×672 每步约 1.8 秒，40 步不到 80 秒：https://t.co/N5tmnlYm99 3. 扩散模型同样能做前缀 KV 缓存：Qwen-Image-2.1 在 Diffusers 上复用提示词前缀，每步只重算图像 token：https://t.co/vQGtR1x2f8 4. Uzu 在 Apple Silicon 上深度优化投机解码，Qwen3.5-9B 从…
Digest-only: confidence 0.35 below 0.60; incomplete methodology (quantization, context length, hardware state, measurement method).
RTX 4090 × Qwen3.8-Flash-Next
Oct 12, 2026 · by @MinLiBuilds · third-party report
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | — | — | 120.0 tok/s | — |
🔥 赚麻了，本地每年产出 $56000 token。 一张魔改 RTX 4090 48GB，跑 Qwen3.8-Flash-Next + Strata，单路生成约 120 tok/s。 按 24 小时持续输出、实测输入输出比例和 95% 缓存命中率估算，每天可处理近 3 亿 Token。 😲 对标 GPT-6.1 Sol API 价格，理论等值约 $156/天，$4,700/月。 当然，软件能力可能不一样。 最近也开始让 Muse 24 小时跑任务了，看看能到什么程度🚀
Digest-only: confidence 0.35 below 0.60; incomplete methodology (engine+version, quantization, context length, hardware state, measurement method).
Tesla V100 32GB x2 × GLM-5.3-Flash
Oct 12, 2026 · by @UtaAoya
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | — | project maya v1.1.0 | 22.9 tok/s | — |
| 32768 | — | project maya v1.1.0 | — | 552.9 tok/s |
Local LLM - V100環境 中古約35万円環境※でGLM-5.3-FlashがMaya v1.1.0でこれくらい動いたよ！☺️ Strata革命以降『主戦場はGPUのパワー勝負から、PC全体最適』へ。 なのに『X投稿には“ハードウェア構成が1部しか書いてない“から再現が難しい』ので、私の環境を詳しく書きました。騒音と消費電力(150Wで耐えられる音量に)、メモリチャネルも参考にしてください。 ※35万円構成は256GBから128GB分を除外した金額。見れば分りますがDDR4全振りでSSD性能捨ててますw チャッピー解説 中古Tesla V100 32GB×2＋DDR4で、GLM-5.3-Flashに挑戦！🚀 約35万円構成のサーバーでProject Maya v1.1.0を検証。 ✅ DDR4を256GB→128GB（2133→2400 MT/s）に変更(マザーボードとCPUの限界のため抜きました泣) ✅…
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (quantization, measurement method).
RX 9070 XT x2 × Qwen3.8-Flash-Next
Oct 12, 2026 · by @doplxyz
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | IQ3_S | strata | 63.5 tok/s | 153.0 tok/s |
| — | IQ3_S | strata | 72.9 tok/s | 134.9 tok/s |
| — | IQ2_XS | strata | 73.1 tok/s | 177.4 tok/s |
| — | IQ2_XS | strata | 86.3 tok/s | 184.3 tok/s |
【Strata起動高速化と固定化オプション検証まとめ】 ■ 背景と課題 Strata（R9700 32GB + RX 9070 XT 16GB環境）において、起動時の自動判定オーバーヘッドを是正するためパラメータ固定化を実施。初期構成（Coder IQ1_M 23.4GB・単一GPU）は約16秒で起動していたが、現行のIQ3_S（46.8GB・2GPU）では重み読込だけで約32秒を要し、スプリット探索や重み二重展開が遅延を招いていた。 ■ 適用した固定化・最適化オプション 1.
--layer-split固定（IQ3_S: 42 / IQ2_XS: 38） 46通りのスプリット探索を完全スキップ。 2.--trim-stage-weights（STRATA_STAGE_TRIM=1相当）…
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, context length).
M4 Mac Mini + Chestnut eGPU × Qwen3.8-Next-Flash
Oct 12, 2026 · by @0xSero
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| 200K | EXL3 3 bits | exl3 | 22.0 tok/s | — |
| 200K | EXL3 3 bits | exl3 | — | 430.0 tok/s |
Qwen3.8-Next-Flash is at 23 tok/s in exl3-3 bits M4 Mac Mini NVMe engram + 47% of the experts 16 GB RAM holds OS + some 23% experts GPU holds 30% of the experts Chestnut eGPU by Tinygrad 22 tok/s decode 430 tok/s prefill Its 200k kv cache Vision enabled Cost 2250$ + tax https://t.co/YF06i9gij9
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, hardware state, measurement method).
V100 16GB × Qwen3.8-Flash-Next
Oct 12, 2026 · by @llm_frontier_jp
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| 32768 | Q2_0 | — | 54.3 tok/s | — |
Strata × Qwen3.8-Flash-Nextを実測。 V100 16GB×1でDecode 54.3 tok/s（Q2_0・MTP4）。 16GB／32GB、Q2_0／IQ3_Sの結果を4枚にまとめました。 入力32,768／出力1,024 tokens、各設定2回の中央値。 ※CPU・RAMも異なる構成比較。16GB・IQ3_Sの条件5は参考値。 https://t.co/qf7NM4KRUj
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, hardware state).
Unspecified platform
Oct 12, 2026 · by @jun1228909 · third-party report
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | IQ3_S | — | 150.0 tok/s | — |
> IQ3_S is comparable and much faster at 150 tok/s え？まじ？👀 一週間前にXXSの推論エフォートnoneで喜んでた数字より大きいんだけど まさかまた速くなってる？😂 https://x.com/jun1228909/status/2108046964277530715?s=46 そろそろstrata検証バッチでも組むか？😂
Digest-only: confidence 0.35 below 0.60; hardware.name missing; model.name missing; incomplete methodology (engine+version, context length, hardware state, measurement method).
RTX 3090 + M4 Mac Mini 16GB × Qwen3.8-next-flash-exl3-3 bits
Oct 12, 2026 · by @0xSero
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| 200K | EXL3 3 bits | tinygrad | 17.5 tok/s | — |
| 200K | EXL3 3 bits | tinygrad | — | 422.0 tok/s |
Ready for Qwen 4? Here are the stats for my rtx 3090 + m4 mac mini 16 GB build, for 2800$ this is amazing Qwen3.8-next-flash-exl3-3 bits - vision - 200k kv cache - 17.5 tok/s decode - 422 tok/s prefill Using tinygrad's chesnut eGPU, via usbc, shockingly good https://t.co/5Qzd6IemUH
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (engine+version, hardware state, measurement method).
Unspecified platform × Qwen3.8
Oct 12, 2026 · by @uzuki425 · third-party report
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | — | — | 78.8 tok/s | — |
| — | — | — | 53.9 tok/s | — |
Strata＋Qwen3.8−nf めちゃ速度が向上。1.35から切り替えてチューニングをしっかりやると+46%向上 --remote-expert-opt と helper の expert 数 auto の組み合わせが 78.8 tok/s で一番速かったです。v0.1.43 のデフォルトより +27%、53.9 tok/s と比べると +46%向上
Digest-only: confidence 0.35 below 0.60; hardware.name missing; incomplete methodology (engine+version, quantization, context length, hardware state, measurement method).
Unspecified platform × DeepSeek-V4.1-Flash
Oct 12, 2026 · by @ItsmeAjayKV
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| — | — | — | 175.4 tok/s | — |
DeepSeek-V4.1-Flashで、今まで見た中で一番遅い速度です。 平均：175.41 tok/s 最低：146.7 tok/s 最高：234.6 tok/s それでも、ほとんどの他のモデルよりずっと速いです。 https://t.co/IV3DSNj3Af
Digest-only: hardware.name missing; incomplete methodology (engine+version, quantization, context length, hardware state, measurement method).
RTX 5070 x1 + RTX 5060 Ti × Qwen3.8-Flash-Next
Oct 12, 2026 · by @murasametech
| Context | Quant | Engine | Decode | Prefill |
|---|---|---|---|---|
| 4096 | IQ2_XS | strata v0.1.43 | — | 750.0 tok/s |
| 14K | IQ2_XS | strata v0.1.43 | — | 1500.0 tok/s |
| 32K | IQ2_XS | strata v0.1.43 | — | 1750.0 tok/s |
| — | IQ3_S | strata v0.1.43 | — | 150.0 tok/s |
RAM 64GBの自宅PCで、MoEモデルのexpertをRAMに常駐させて動かす方式を試しました。 GPUはRTX 5070 12GB＋RTX 5060 Ti 16GB、モデルはQwen3.8-Flash-NextのIQ2_XS（約2bit）、推論エンジンはStrata v0.1.43です。 長文を入れてもSSDからの読み込みは1回あたり数十〜百数十MBで済み、Prefillは4K tokensで約750、1.4万で約1,500、3.2万で約1,750 tok/sでした。 同じ方式でIQ3_S（約3bit）を動かしたときは、長文のたびに約88GBを読み直し、Prefillが約150 tok/sまで落ちていました。 版も違うので同条件の比較ではありませんが、expertがRAMに収まるかどうかで挙動が大きく変わると考えられます。計測は各1回です。
Digest-only: no run with engine+version+quant+context+decode tps; incomplete methodology (hardware state, measurement method).
Also in this window
The 12 most recent posts are shown in full above; 28 posts from the same window are listed here in short form, each with its source post.
- M1 Max × littlebit-qwen3-4b — 74.0 tok/s decode (2 measurements) · @LittleBit_llm
- Radeon R9700 AI Pro x4 × Qwen 3.8 Next Flash — 6880.0 tok/s decode (3 measurements) · @richgel999
- Unspecified platform × GLM-5.3 — 493.0 tok/s decode (3 measurements) · @NFT_Chen
- RTX 4090 × Qwen3.8-Flash-Next — 205.2 tok/s decode (2 measurements) · @servasyy_ai
- RTX 5090 × UD-Q4_K_M — 150.0 tok/s decode (2 measurements) · @KyleHessling1
- DGX Spark × GLM 5.3 Flash — 62.0 tok/s decode (3 measurements) · @WescheNex1q
- RTX PRO 6000 Blackwell × GLM-5.3 — 125.0 tok/s decode (1 measurement) · @wei_wang
- DGX Spark x4 × GLM 5.3 Flash — 133.0 tok/s decode (6 measurements) · @Tech2Wild
- DGX Spark x4 × GLM-5.3-Flash — 176.0 tok/s decode (12 measurements) · @Tech2Wild
- DGX Spark × RED-SNOW-5.3-FLASH — 78.1 tok/s decode (7 measurements) · @yume_arasaki
- M1 MacBook × Qwen3.8-27B — 5.5 tok/s decode (3 measurements) · @Oluwaphilemon1
- DGX Spark x8 × GLM-5.3 — 23.0 tok/s decode (1 measurement) · @wei_wang
- RTX 3060 × Qwen3.8-35B-A3B — 50.0 tok/s decode (2 measurements) · @Oluwaphilemon1
- RTX 3090 × Qwen3.8-Flash-Next — 87.0 tok/s decode (3 measurements) · @superalesha
- Strix Halo × Qwen3.8 Flash-Next HC-Q8 — 31.4 tok/s decode (4 measurements) · @no_stp_on_snek
- Unspecified platform × Deepseek v4.1 at — 1000.0 tok/s decode (2 measurements) · @softpoo
- RTX 5060 Ti — 64.5 tok/s decode (2 measurements) · @sudoingX
- DGX Spark x4 × Deepseek V4.1 — 130.0 tok/s decode (2 measurements) · @softpoo
- RTX 3090 x4 × GLM-5.3 Flash — 101.5 tok/s decode (2 measurements) · @superalesha
- RTX 3090 × GLM-5.3-Flash — 23.0 tok/s decode (3 measurements) · @ItsmeAjayKV
- DGX Spark x2 × GLM-5.3-Flash — 85.0 tok/s decode (4 measurements) · @LocalsOnlyAI
- Radeon RX 6000 — 50.0 tok/s decode (1 measurement) · @sudoingX
- DGX Spark x2 × DeepSeek-V4.1-Flash — 114.6 tok/s decode (6 measurements) · @adrien_jeanneau
- Apple Silicon × Qwen3.5 9B — 104.5 tok/s decode (2 measurements) · @akshay_pachaar
- Radeon AI PRO R9700 × Qwen 3.8 Flash-Next Base — 67.5 tok/s decode (2 measurements) · @doplxyz
- RTX 5090 × GLM-5.3-Flash — 14.2 tok/s decode (2 measurements) · @crowqnt
- RTX 3090 × DeepSeek V4.1 Flash — 87.0 tok/s decode (2 measurements) · @superalesha
- DGX Spark × DeepSeek V4.1 Flash — 32.0 tok/s decode (1 measurement) · @jayleaton
How to read this page
This digest is an automated transcription of public posts. Runs with a stated engine build, quantization, context length, hardware state and measurement method are published straight to the benchmark board with a link to their source; the rest stay here. These are author-reported numbers, not an independent DiggTop re-measurement: every row carries its provenance so you can check the original, and a number that turns out to be misreported is corrected or removed. Our own testing follows the methodology. Spotted an error? Reply on the source post or submit a corrected run at /submit.
Sources
- @wei_wang — Oct 12, 2026
- @wquguru — Oct 12, 2026
- @MinLiBuilds — Oct 12, 2026
- @UtaAoya — Oct 12, 2026
- @doplxyz — Oct 12, 2026
- @0xSero — Oct 12, 2026
- @llm_frontier_jp — Oct 12, 2026
- @jun1228909 — Oct 12, 2026
- @0xSero — Oct 12, 2026
- @uzuki425 — Oct 12, 2026
- @ItsmeAjayKV — Oct 12, 2026
- @murasametech — Oct 12, 2026
- @LittleBit_llm — Oct 12, 2026
- @richgel999 — Oct 12, 2026
- @NFT_Chen — Oct 12, 2026
- @servasyy_ai — Oct 12, 2026
- @KyleHessling1 — Oct 12, 2026
- @WescheNex1q — Oct 12, 2026
- @wei_wang — Oct 12, 2026
- @Tech2Wild — Oct 12, 2026
- @Tech2Wild — Oct 12, 2026
- @yume_arasaki — Oct 12, 2026
- @Oluwaphilemon1 — Oct 12, 2026
- @wei_wang — Oct 12, 2026
- @Oluwaphilemon1 — Oct 12, 2026
- @superalesha — Oct 11, 2026
- @no_stp_on_snek — Oct 11, 2026
- @softpoo — Oct 11, 2026
- @sudoingX — Oct 11, 2026
- @softpoo — Oct 11, 2026
- @superalesha — Oct 11, 2026
- @ItsmeAjayKV — Oct 11, 2026
- @LocalsOnlyAI — Oct 11, 2026
- @sudoingX — Oct 11, 2026
- @adrien_jeanneau — Oct 11, 2026
- @akshay_pachaar — Oct 11, 2026
- @doplxyz — Oct 11, 2026
- @crowqnt — Oct 11, 2026
- @superalesha — Oct 11, 2026
- @jayleaton — Oct 11, 2026
FAQ
Are the numbers in this digest sourced?
They are transcribed from public X posts and the repositories those posts link to, with attribution and a source link on every entry. Rows that state the full methodology are published to the benchmark board automatically, so treat them as author-reported until a dispute is raised: the source link on every row is the check.
What does it take for a row to reach the sourced tier?
Engine and exact version, quantization, context length, hardware state and a described measurement method. Posts that state all five are published to the benchmark board as soon as the collector reads them, each row carrying its source link — nothing waits in a human review queue. Anything missing a condition stays in this digest.
Where does the data come from?
Public X posts about local-model inference measured on owned hardware, discovered by keyword search every two hours and linked back to the original thread. The digest is regenerated daily.