NVIDIA-Nemotron-3.5-Lightning-30B-A3B
9recorded runs
446.7best tok/s
6verified
Fastest GPUs for this model (best tok/s)
- RTX 5090446.7
- RTX 3090353.7
- CMP 170HX217.9
- RTX 3090 x2197.0
- Intel Arc Pro B70186.6
| GPU | Model | Quant | Backend | Ctx | tok/s | prompt tok/s | VRAM GB | Source |
|---|---|---|---|---|---|---|---|---|
| RTX 5090 · 32 GB | NVIDIA-Nemotron-3.5-Lightning-30B-A3B | NVFP4 | vllm 0.27.1 | 65536 | 446.7 ✓ | — | — | src |
| RTX 3090 · 24 GB | NVIDIA-Nemotron-3.5-Lightning-30B-A3B | NVFP4 | llama.cpp 030ebb558 + PR 26623 @ 000d8ecd4 + l… | 8192 | 353.7 ✓ | 1951.8 | 21.4 | src |
| RTX 3090 x2 · 24 GB | NVIDIA-Nemotron-3.5-Lightning-30B-A3B | NVFP4 | vllm 0.27.1 | 131072 | 197.0 ✓ | 6719.0 | 19.5 | src |
| Intel Arc Pro B70 · 32 GB | NVIDIA-Nemotron-3.5-Lightning-30B-A3B | GPTQ-INT4-G64-sym-local+DFlash-B | vllm v0.26.1rc1.dev668+g3ee2df303 (XPU) | 16384 | 186.6 ✓ | 7160.0 | — | src |
| CMP 170HX · 64 GB | NVIDIA-Nemotron-3.5-Lightning-30B-A3B | Q8_0 | llama.cpp 0.3.0-dev+b10705 | 67592 | 165.1 ✓ | 1524.7 | — | src |
| Intel Arc Pro B70 · 32 GB | NVIDIA-Nemotron-3.5-Lightning-30B-A3B | Unsloth-Dynamic-Q4_K_M | llama.cpp 9fee29e9435f; oneAPI 2026.1.1 | 8192 | 72.2 ✓ | — | — | src |
| CMP 170HX · 64 GB | NVIDIA-Nemotron-3.5-Lightning-30B-A3B | W4A16 | vllm 0.28.0 | 67592 | 217.9 … | 8392.9 | — | src |
| RX 7900 XT x2 · 20 GB | NVIDIA-Nemotron-3.5-Lightning-30B-A3B | NVFP4 | llama.cpp unknown | 2048 | 116.0 … | 1191.7 | — | src |
| RTX 4070 · 12 GB | NVIDIA-Nemotron-3.5-Lightning-30B-A3B | IQ4_XS | llama.cpp unknown | 262144 | 35.0 … | 398.2 | 11.3 | src |