Running language models on a personal computer with Ollama

Why run the model on your own computer?

• Complete privacy: No data leaves your computer.
• No internet: Works offline after downloading the model.
• No access and subscription limits: No registration or monthly payment required.
• Connectable to your own apps: Via a local API.

Of course, we must be realistic: models running on an ordinary laptop do not match the most powerful commercial models; however, they are very useful for summarization, answering questions, assisting in coding, and working with private documents.

What hardware is required?

RAM (or GPU VRAM)	Suitable Model Size	Experience
8 GB	1 to 4 billion parameter models	Simple and fast, medium quality
16 GB	7 to 8 billion parameter models	Good balance of quality and speed
32 GB or more	14 to 32 billion parameter models	High quality, slower on CPU

A graphics card (especially NVIDIA with high memory) or Macs with M-series chips multiply the speed, but small models can run without them. Ollama automatically downloads quantized versions of models, which take up much less memory.

Step 1: Install Ollama

• Windows and Mac: Download and run the installer from the official website ollama.com.
• Linux: Run in the terminal:

curl -fsSL https://ollama.com/install.sh | sh

To ensure installation:

ollama --version

Step 2: Download and run the first model

ollama run qwen3:8b

This command downloads the model (first time only) and opens a text chat in the terminal. To exit, type /bye.

Some suggested models to start with:

Model	Command	Good for
Qwen3 8B	ollama run qwen3:8b	General purpose, acceptable Persian
Llama 3.2 3B	ollama run llama3.2:3b	Weak systems, fast response
DeepSeek-R1 8B	ollama run deepseek-r1:8b	Reasoning, math
Gemma 3	ollama run gemma3	Google's open model, multilingual

The full list of models is in the Ollama Library. You can see each model with ollama list and remove it with ollama rm model-name.

Step 3: ChatGPT-like Graphical Interface

If working with the terminal is not comfortable for you:

• LM Studio: Full desktop app with a graphical interface to search, download, and chat with models (no Ollama required).
• Open WebUI: Professional web interface that connects to Ollama; has document upload and multi-user capabilities. Ideal for setting up an internal company assistant.

Step 4: Connect to your own apps

After installation, Ollama provides an API at http://localhost:11434:

curl http://localhost:11434/api/generate -d '{
  "model": "qwen3:8b",
  "prompt": "Give three tips for writing an official email",
  "stream": false
}'

Ollama also has an OpenAI-compatible API on the /v1 path; therefore, many apps and libraries written for ChatGPT will connect to your local model by changing the server address.

Practical tips

1. For Persian, larger models yield much better results; if you have enough memory, try the 14-billion version and up.
2. Read each model's license, especially if you have commercial use. See license comparisons in Open-Source Model Comparison.
3. Low speed? Try a smaller model or a more quantized version and close other heavy applications.
4. To query your own documents, use the RAG technique; Open WebUI does this without programming.