Llama-3.1-8B-Instruct
1recorded run
158.1best tok/s
1verified
Fastest GPUs for this model (best tok/s)
| GPU | Model | Quant | Backend | Ctx | tok/s | prompt tok/s | VRAM GB | Source |
|---|---|---|---|---|---|---|---|---|
| RTX PRO 6000 Blackwell · 96 GB | Llama-3.1-8B-Instruct | FP8 | vllm 0.26.0 | 32768 | 158.1 ✓ | — | — | src |