Buying guide · updated 2026-10-17
What hardware do you need to run AI locally?
Short answer: memory decides everything. A 16 GB graphics card runs the popular 7B-14B models fast; 24 GB opens up 32B models; a Mac's unified memory counts as GPU memory, so a 24 GB Mac mini is the easiest way in. Everything below is for 4-bit models, the default in Ollama and LM Studio.
How much memory each model needs
Rule of thumb: about 0.6 GB per billion parameters at 4-bit, plus about 1 GB for the context. A model that does not fit in GPU memory still runs in Ollama and LM Studio, but the part that spills into system RAM runs many times slower.
| Model size | Examples | Memory needed | Runs well on |
|---|---|---|---|
| 3B-4B | Llama 3.2 3B, Gemma 3 4B, Qwen3 4B | ~3 GB | Any recent laptop, 8 GB Mac |
| 7B-8B | Llama 3.1 8B, Qwen3 8B, Mistral 7B | ~5-6 GB | 8 GB GPU minimum, 12-16 GB comfortable; 16 GB Mac |
| 12B-14B | Gemma 3 12B, Qwen3 14B, Phi-4 | ~9-10 GB | 16 GB GPU; 24 GB Mac |
| 24B-32B | Mistral Small 24B, Gemma 3 27B, Qwen3 32B | ~16-20 GB | 24 GB GPU; Mac with 32-48 GB |
| 70B | Llama 3.3 70B | ~40-43 GB | Two 24 GB GPUs; Mac with 64 GB+; 128 GB mini PC |
Picks by budget
Each pick is the cheapest way to reach one memory step. Prices move every week, so we link to the current listings instead of quoting one.
Best value GPU
NVIDIA GeForce RTX 5060 Ti 16GB
16 GB VRAM · runs 7B-14B models fully on the GPU
The cheapest current card with 16 GB of video memory. Make sure it is the 16 GB version, not the 8 GB one.
24 GB on a budget
NVIDIA GeForce RTX 3090 24GB
24 GB VRAM · runs 24B-32B models at 4-bit
The classic local-AI card: 24 GB for less than a new 24 GB+ card. Mostly sold renewed or used, so check the seller and the return window. Needs a 750 W+ power supply.
No compromises
NVIDIA GeForce RTX 5090 32GB
32 GB VRAM · 32B models with long context, Flux at high quality
The most memory on a single consumer card. Expensive and power hungry (needs a 1000 W supply); only worth it if you run models every day.
As an Amazon Associate, AltAI earns from qualifying purchases. Links open an Amazon search for the named product; prices and stock are on Amazon.
Mac, mini PC or graphics card?
A graphics card (NVIDIA) is the fastest per dollar and the best supported: every tool, including Stable Diffusion and fine-tuning, assumes CUDA first. You need a desktop PC with room and power for it.
A Mac uses one pool of memory for everything, so 24-48 GB of unified memory works like a big graphics card. Slower than an NVIDIA card of the same size, but silent, efficient and zero setup: install Ollama or LM Studio and go. Memory cannot be upgraded, so buy enough.
A Ryzen AI Max mini PC works like a Mac with Windows or Linux and up to 128 GB, the cheapest way to load a 70B model. It is the slowest of the three per model size.
Easiest setup
Apple Mac mini with 24 GB or more
Unified memory doubles as GPU memory · quiet · low power
On Apple silicon the GPU uses the main memory, so a 24 GB Mac mini runs 14B models comfortably. Buy the memory up front: it cannot be upgraded later. 16 GB models are fine only for 8B and smaller.
Laptop for 32B models
MacBook Pro with 48 GB or more
48 GB+ unified memory · runs 24B-32B models on battery
The only laptops that run 32B models without a separate GPU. Pick by memory, not by chip: 48 GB is the step that matters.
Biggest models per dollar
Mini PC with AMD Ryzen AI Max+ 395 and 128 GB
Up to ~96 GB usable as GPU memory · runs 70B models at 4-bit
Shared memory like a Mac, at a lower price per GB. 70B models run, but slowly (a few words per second); best for people who want big models more than fast ones. Check the model has 128 GB, not 64 GB.
As an Amazon Associate, AltAI earns from qualifying purchases. Links open an Amazon search for the named product; prices and stock are on Amazon.
Image generation (Stable Diffusion, Flux)
Image models need NVIDIA more than language models do: most guides, extensions and speed-ups are written for CUDA. Macs work in tools like Draw Things and ComfyUI, more slowly.
| Model | GPU memory | Runs well on |
|---|---|---|
| Stable Diffusion 1.5 | ~4-6 GB | Almost any NVIDIA card from the last 6 years; 16 GB Mac |
| SDXL | ~8 GB (12 GB comfortable) | 12-16 GB GPU; 24 GB Mac |
| Flux.1 dev | ~12-16 GB with fp8 / GGUF, 24 GB for full quality | 16 GB GPU minimum, 24-32 GB ideal |
Already have a PC? The cheapest upgrade
If your GPU is small, more system RAM lets you run models that do not fit on the card, at reduced speed. Going from 16 or 32 GB to 64 GB is often enough to try 24B-32B models before spending on a new GPU.
Cheapest upgrade
64 GB DDR5 RAM kit (2 × 32 GB)
Lets Ollama and LM Studio spill a too-big model into system memory
When a model does not fit in the GPU, Ollama and LM Studio run the rest from system RAM: slower, but it runs. Check your motherboard first: DDR4 boards and laptops need a different kit.
As an Amazon Associate, AltAI earns from qualifying purchases. Links open an Amazon search for the named product; prices and stock are on Amazon.
Books for running and building on local models
Book
AI Engineering — Chip Huyen
O'Reilly · building real applications on top of models
The practical book on evaluating models, prompting, RAG and fine-tuning, once you have a model running.
Book
Build a Large Language Model (From Scratch) — Sebastian Raschka
Manning · code you can run on your own machine
Explains what the model files you download in Ollama actually are, by building a small one step by step.
Book
Hands-On Large Language Models — Jay Alammar & Maarten Grootendorst
O'Reilly · visual explanations, open models
The most visual introduction to how language models work, built on the open models you can run locally.
As an Amazon Associate, AltAI earns from qualifying purchases. Links open an Amazon search for the named product; prices and stock are on Amazon.
How we picked
We start from the memory a model needs, then name the least expensive current hardware that reaches each step, and say what each pick is bad at. Memory figures are approximations for 4-bit GGUF models at a 4K-8K context; real use varies with context length and the tool. Brands do not pay for placement. Software that runs these models: Ollama alternatives, LM Studio alternatives, Ollama vs LM Studio, Stable Diffusion alternatives.
As an Amazon Associate, AltAI earns from qualifying purchases. Links open an Amazon search for the named product; prices and stock are on Amazon.