Carnice-9b
1recorded run
56.1best tok/s
1verified
Fastest GPUs for this model (best tok/s)
- RTX 3060 x256.1
| GPU | Model | Quant | Backend | Ctx | tok/s | prompt tok/s | VRAM GB | Source |
|---|---|---|---|---|---|---|---|---|
| RTX 3060 x2 · 12 GB | Carnice-9b | Q4_K_M | llama.cpp 2d97363 | 512 | 56.1 ✓ | 1621.8 | 6.7 | src |