Run your first local AI in 10 minutes

Everything in this guide is free, open-source and runs offline after setup. By the end you’ll have a private assistant that answers on your machine, even with the Wi-Fi off.

What you need

• Any modern computer (Windows, macOS or Linux). 8GB RAM is enough to start; 16GB is comfortable.
• About 10 minutes and 3GB of disk space.

No GPU required for this tutorial. If you have one, great — it gets faster.

1. Install Ollama

Ollama is the easiest on-ramp to local models. Grab the installer for your system from ollama.com and run it. On macOS and Linux you can also use a one-liner:

curl -fsSL https://ollama.com/install.sh | sh

Verify it’s alive:

ollama --version

2. Pull your first model

Models are a few gigabytes each. Start with a 4B model — it runs well even without a GPU:

ollama run qwen3:4b

First run downloads the model (~2.6GB). When the prompt appears, just talk to it. Ask it to summarize your week, draft an email, or explain a concept.

Exit with /bye or Ctrl+D.

3. Make it yours

Two upgrades that take five minutes each:

A nicer interface. Open WebUI gives you a ChatGPT-style experience with chat history, document uploads and multiple models:

docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui ghcr.io/open-webui/open-webui:main

Then visit http://localhost:3000.

A better model. When you’re ready, step up to an 8B model (needs ~6GB VRAM or patience on CPU):

ollama run qwen3:8b

4. What’s actually happening

Ollama runs llama.cpp under the hood. Your prompt stays in RAM, generation happens on your CPU/GPU, and nothing leaves the machine. You can watch network traffic if you’re the suspicious type — we encourage it.

Where to next

• Check the model directory for what fits your hardware.
• Run the ROI calculator and see when your setup pays for itself.
• Ready for images? Read your first local images with ComfyUI.