NVIDIA-Nemotron-3-Nano-4B

Nemotron family · 4.0B params

3recorded runs

237.8best tok/s

1verified

Fastest GPUs for this model (best tok/s)

GPU	Model	Quant	Backend	Ctx	tok/s	prompt tok/s	VRAM GB	Source
RTX 5070 · 12 GB	NVIDIA-Nemotron-3-Nano-4B	Q4_K_M	llama.cpp b10449 (0d9ceae1e38291035605613ab41a…	32768	185.8 ✓	5377.5	4.2	src
RTX 5090 · 32 GB	NVIDIA-Nemotron-3-Nano-4B	Q8_0	lmstudio unknown	2048	237.8 …	1173.5	—	src
RTX 3090 · 24 GB	NVIDIA-Nemotron-3-Nano-4B	Q4_K_M	llama.cpp unknown	512	171.3 …	5608.8	0.0	src