Cohere
by Cohere Inc.
Cohere offers Command, Embed, and Rerank models optimized for enterprise search, RAG, and text generation. Strong on data privacy with private cloud deployment. Competing with OpenAI at the API tier.
Commercial LLM API vs open-source hub.
Our pick
It depends on the job
Both are free to start, so pick by what you need most:
by Cohere Inc.
Cohere offers Command, Embed, and Rerank models optimized for enterprise search, RAG, and text generation. Strong on data privacy with private cloud deployment. Competing with OpenAI at the API tier.
by Hugging Face
The model and dataset hub for the entire open-source ML community. Free to host. Spaces, Inference Endpoints, AutoTrain, and datasets.
| Feature | Cohere | Hugging Face |
|---|---|---|
| Vendor | Cohere Inc. | Hugging Face |
| Category | ML Platforms | ML Platforms |
| Free tier | Yes | Yes |
| Starting price | Free | Free |
| Strengths | private cloud deploy, RAG-optimized Embed + Rerank, enterprise SLAs | largest model catalog, free hosting, Spaces for demos |
| Weaknesses | smaller ecosystem than OpenAI, no consumer product | Inference Endpoint cold starts, can be overwhelming |
The one number that matters is memory: the model has to fit in your GPU's VRAM (or a Mac's unified memory). Speed comes second. Figures are for the 4-bit models Ollama and LM Studio download by default.
| Model size | Examples | Memory needed | Runs well on |
|---|---|---|---|
| 3B-4B | Llama 3.2 3B, Gemma 3 4B, Qwen3 4B | ~3 GB | Any recent laptop, 8 GB Mac |
| 7B-8B | Llama 3.1 8B, Qwen3 8B, Mistral 7B | ~5-6 GB | 8 GB GPU minimum, 12-16 GB comfortable; 16 GB Mac |
| 12B-14B | Gemma 3 12B, Qwen3 14B, Phi-4 | ~9-10 GB | 16 GB GPU; 24 GB Mac |
| 24B-32B | Mistral Small 24B, Gemma 3 27B, Qwen3 32B | ~16-20 GB | 24 GB GPU; Mac with 32-48 GB |
| 70B | Llama 3.3 70B | ~40-43 GB | Two 24 GB GPUs; Mac with 64 GB+; 128 GB mini PC |
Best value GPU
NVIDIA GeForce RTX 5060 Ti 16GB
16 GB VRAM · runs 7B-14B models fully on the GPU
The cheapest current card with 16 GB of video memory. Make sure it is the 16 GB version, not the 8 GB one.
24 GB on a budget
NVIDIA GeForce RTX 3090 24GB
24 GB VRAM · runs 24B-32B models at 4-bit
The classic local-AI card: 24 GB for less than a new 24 GB+ card. Mostly sold renewed or used, so check the seller and the return window. Needs a 750 W+ power supply.
Easiest setup
Apple Mac mini with 24 GB or more
Unified memory doubles as GPU memory · quiet · low power
On Apple silicon the GPU uses the main memory, so a 24 GB Mac mini runs 14B models comfortably. Buy the memory up front: it cannot be upgraded later. 16 GB models are fine only for 8B and smaller.
As an Amazon Associate, AltAI earns from qualifying purchases. Links open an Amazon search for the named product; prices and stock are on Amazon.