phi-4
1recorded run
34.2best tok/s
1verified
Fastest GPUs for this model (best tok/s)
- RTX 3060 x234.2
| GPU | Model | Quant | Backend | Ctx | tok/s | prompt tok/s | VRAM GB | Source |
|---|---|---|---|---|---|---|---|---|
| RTX 3060 x2 · 12 GB | phi-4 | Q4_K_M | llama.cpp b10443 | 16384 | 34.2 ✓ | 889.5 | 9.8 | src |
| GPU | Model | Quant | Backend | Ctx | tok/s | prompt tok/s | VRAM GB | Source |
|---|---|---|---|---|---|---|---|---|
| RTX 3060 x2 · 12 GB | phi-4 | Q4_K_M | llama.cpp b10443 | 16384 | 34.2 ✓ | 889.5 | 9.8 | src |