Quick pick: For Ollama, VRAM is everything. A used or new RTX 3060 12GB is the cheapest card that fits 8B to 14B models well. A 16GB card (RTX 5060 Ti 16GB or RTX 4060 Ti 16GB) fits 14B to 20B models. A used RTX 3090 24GB is still the budget king for 30B class models.
Specs below are NVIDIA reference specs. Board partner cards vary slightly in clocks and size. We skip prices because they move daily, especially on used cards.
Why we lean NVIDIA here
Ollama supports NVIDIA (CUDA) and many AMD Radeon cards (ROCm), but NVIDIA cards are the least hassle on Windows and Linux and have the most community troubleshooting. If you already own a supported Radeon card, use it. If you are buying for AI, NVIDIA is the safer bet.
VRAM vs model size
- 8GB: 7B to 8B models only, with modest context.
- 12GB: 8B with long context, 14B (qwen3:14b is about 9.3GB).
- 16GB: 14B comfortably, gpt-oss:20b (about 14GB).
- 24GB: qwen3:30b and 32B models (about 19GB to 20GB).
Full chart: how much RAM and VRAM for Ollama.
The picks
NVIDIA GeForce RTX 3060 12GB: cheapest good option
- 12GB GDDR6, 192-bit bus, 360GB/s bandwidth
- 170W board power
- Widely available used and new
Why it works: 12GB for the least money. Runs 8B models fully on GPU with room for long context and fits 14B 4-bit models.
Downsides: Older and slower than newer cards. Make sure you get the 12GB version: an 8GB RTX 3060 also exists and is much worse for AI.
Skip it if: you want 20B+ models fully on GPU.
NVIDIA GeForce RTX 5060 Ti 16GB: best new budget card
- 16GB GDDR7, 128-bit bus, 448GB/s bandwidth
- 180W board power
- Current generation with warranty
Why it works: 16GB of fast memory on a new, efficient card. Fits gpt-oss:20b and 14B models with headroom.
Downsides: Also sold in an 8GB version; only the 16GB model makes sense for local AI. Narrow 128-bit bus.
Skip it if: you can find a clean used RTX 3090 for similar money and have the PSU and case space for it.
NVIDIA GeForce RTX 4060 Ti 16GB: if you find it cheaper
- 16GB GDDR6, 128-bit bus, 288GB/s bandwidth
- 165W board power
Why it works: Same 16GB capacity as the 5060 Ti and low power draw. Worth it only when it is clearly cheaper.
Downsides: Lower bandwidth than the 5060 Ti, so token speed is lower. Previous generation.
Skip it if: the 5060 Ti 16GB costs about the same.
NVIDIA GeForce RTX 3090 24GB (used): budget king for 30B
- 24GB GDDR6X, 384-bit bus, 936GB/s bandwidth
- 350W board power, large triple slot cards
- Mostly a used or refurbished purchase now
Why it works: 24GB and very high bandwidth. Runs qwen3:30b and 32B models fully on GPU at good speed.
Downsides: Power hungry and hot. Plan on a quality 750W+ PSU and check case clearance. Used cards may have been mined on; buy from sellers with returns.
Skip it if: your PSU is under 750W or you only run 8B models.
Before you buy
- Check your power supply wattage and PCIe power connectors.
- Measure your case; many cards are over 300mm long.
- Prefer more VRAM over a newer chip at the same price for AI use.
No tower to put it in? Look at mini PCs for local AI or our budget AI PC under $1,000 paths. Then set up the software with our Ollama install guide.
FAQ
What is the best budget GPU for Ollama?
The RTX 3060 12GB is the cheapest card with enough VRAM for 8B to 14B models. If you can spend more, a 16GB RTX 5060 Ti or a used 24GB RTX 3090 runs bigger models.
Is 8GB of VRAM enough for Ollama?
For 7B to 8B models, yes. For 14B and up, the model will spill into system RAM and slow down.
Is the RTX 3090 still worth it for local AI?
Yes, if you have a 750W+ power supply and room in your case. Its 24GB of VRAM is the cheapest way to run 30B class models fully on GPU.
Does Ollama work with AMD GPUs?
Yes, Ollama supports many AMD Radeon cards through ROCm. NVIDIA cards are still the least hassle, especially on Windows.