How to Use Ollama for Private AI-Assisted Development

To use Ollama for private AI-assisted development, install Ollama, pull a coding model like qwen3-coder or qwen2.5-coder, and point your editor or agent at the local API on http://localhost:11434. Everything runs on your own machine, so your code never leaves it. Most guides stop at local autocomplete and chat, but Ollama can also drive full AI coding agents that plan, write, and review code locally, which is where private development gets genuinely useful in 2026.

What Does Private AI-Assisted Development With Ollama Actually Mean?

Ollama runs open-weight language models on your hardware and exposes them through a local HTTP server. The server listens on localhost:11434 and mirrors the OpenAI chat completions API at /v1/chat/completions, so any tool that speaks that format can talk to Ollama by changing the base URL. No token leaves your network, no prompt hits a vendor's logs, and the model keeps working with your Wi-Fi off.

That local API is the whole reason private development works. Your editor sends a completion request to http://localhost:11434/v1/chat/completions, the model responds, and the loop closes without a round trip to a hosted provider. The setup you build for autocomplete is the same endpoint an agent will use later, so nothing you learn here goes to waste.

What Hardware Do You Need to Run Ollama Locally?

Model size drives your memory requirement more than anything else. A rough guide that matches Ollama's own documented guidance:

• 1.5B–3B models: around 4GB of RAM, fine for lightweight autocomplete.

• 7B–8B models: around 8GB of RAM or VRAM, workable on a modern laptop.

• 13B models: around 16GB, comfortable on an Apple Silicon Mac or a mid-range GPU.

• 32B–33B models: around 32GB, where a dedicated GPU or a high-memory Mac starts to matter.

Apple Silicon is a good fit because its unified memory lets the GPU address the same pool as the CPU, so a 32GB Mac can hold a larger model than a discrete 8GB GPU. One caveat worth knowing: mixture-of-experts models like qwen3-coder:30b only activate a few billion parameters per token, but you still need memory for the full model on disk and in memory, roughly 19GB at a 4-bit quantization, not just the active-parameter count. If a model runs slowly or fails to load, drop to a smaller parameter count or a more aggressively quantized build before touching anything else.

Which Ollama Models Are Best for Coding?

The practical defaults have shifted since the last generation of guides, so it's worth being specific about what's current in 2026:

• qwen3-coder:30b: the current top pick for local coding. A 30B-parameter mixture-of-experts model with only 3.3B active per token, a native 256K context window, and roughly a 19GB download at Q4 quantization. Best quality-per-GB on a 24–32GB GPU or a 32GB Apple Silicon Mac.

• qwen2.5-coder: the previous generation's leader (the 32B variant still holds up well on multi-language code repair) and still a fast, capable default when you want something lighter — qwen2.5-coder:7b fits comfortably in 8GB.

• deepseek-coder-v2:16b: the budget pick. A mixture-of-experts coder at roughly 8.9GB with a 160K context window, the best option once you're limited to a 12–16GB GPU.

• devstral-small-2:24b: worth calling out on its own, since it's built specifically for agentic coding rather than autocomplete — multi-file edits, shell commands, running tests. At 24B parameters with a 384K context window, it scores 65.8% on SWE-bench Verified, a real-world benchmark of resolving actual GitHub issues — mid-pack next to frontier hosted models, but strong for something you can run entirely on your own hardware. This is the one to reach for once you get to the agent section below. (Requires Ollama 0.13.3 or later.)

Older picks like codellama and starcoder2 still run and still work for basic autocomplete on very constrained hardware, but they're a generation behind the models above and not where you should start in 2026.

Pull one with a single command:

Start with the largest model your memory allows, since capability climbs with parameter count. You can keep several models pulled and switch between them per task.

How Do You Wire Ollama Into Your Editor?

The common starting point is Ollama plus the Continue extension in VS Code. Continue's current configuration format is YAML (config.yaml), not the older config.json format you'll still see in some tutorials. After installing Continue, open the command palette and run "Continue: Open Config File," then add your local model:

That gives you inline autocomplete and a chat sidebar, both served from localhost. For a fully air-gapped machine, pull every model you need while you still have a connection, then disconnect. Ollama keeps serving from local weights, and Continue keeps working against localhost:11434 with no internet at all.

This covers what most tutorials treat as the finish line. It is a solid private setup for completions and questions. It is also single-session and single-agent, which becomes the ceiling once you want the model to do real work.

How Do You Go From Local Autocomplete to Private AI Agents?

Autocomplete suggests the next few lines. An agent takes a task, writes a plan, edits files across the repo, runs commands, and checks its own output — the distinction covered in more depth in AI IDE vs. Agentic Development Environment. Doing that privately means the agent's model runs through Ollama on your machine, and the code it touches stays local the entire time.

The friction shows up fast. A single agent editing your working branch is hard to supervise, and running two agents at once means they fight over the same files. This is the problem Unstoppable Code is built to solve: a desktop development environment (macOS primary) that runs AI coding agents against a local Ollama model, on-device, with the isolation and review that agent work needs.

Two mechanics make private agent work practical. First, every agent runs in its own isolated git worktree, so parallel agents never collide and your working branch stays clean — the operational side of running several of these at once is covered in How to Manage Parallel Agents Without Losing Track. Second, you approve an agent's plan before it writes any code, and the run leaves a tracked audit trail you can inspect afterward; How to Review an AI Agent's Plan Before It Writes Code covers what to actually look for in that plan, beyond just approving on reflex. For repeatable work, a YAML pipeline engine (styled after GitHub Actions but built for agents) adds retries, quality gates, human-in-the-loop pauses, and durable state that resumes where it stopped.

Ollama Plus Continue Versus a Private Agent Environment

Both keep your code on your machine. They solve different problems.

Does it run fully on-device? Ollama + Continue: yes. Unstoppable Code + Ollama: yes.

Do you get inline autocomplete and chat? Ollama + Continue: yes. Unstoppable Code + Ollama: yes, plus supervised agents on top.

Can you run parallel agents without collisions? Ollama + Continue: no — it's a single session. Unstoppable Code + Ollama: yes, one isolated git worktree per agent.

Is there plan review before code is written? Ollama + Continue: no, nothing built in. Unstoppable Code + Ollama: yes — approve, then execute, with an audit trail.

Is multi-step automation built in? Ollama + Continue: no, it's manual. Unstoppable Code + Ollama: yes, a YAML pipeline engine with retries and quality gates.

Can you switch models per task? Ollama + Continue: yes, but it's a config edit each time. Unstoppable Code + Ollama: yes — Claude, Codex, or local Ollama, switchable per task without touching a config file.

The Continue path is the right tool when you want private completions inside an editor you already use. The agent environment is the right tool when you want the model to carry a task end to end while you stay in control of the plan and the branch.

When Does Fully Private Development Make Sense?

Keep everything local when the code itself is the constraint: regulated codebases, client work under NDA, security-sensitive internal tools, or anything you simply do not want leaving your hardware. How to Use AI Agents Safely on a Production Codebase goes deeper on what "leaves your hardware" actually means in practice — which paths are opt-in, which are provider-driven, and what a compliance review will actually ask you. The on-device path means the model, the agent, and the repository all sit on your machine, with no prompt or file crossing the network.

The trade-off is capability, though it's narrower than it used to be. Local models built specifically for agentic coding, like Devstral Small 2, now score competitively on real-world benchmarks like SWE-bench Verified — but the strongest hosted models still lead on open-ended reasoning across the board. Pick the largest, most task-appropriate model your hardware supports, and reserve local-only mode for the work that actually requires it. Unstoppable Code lets you keep that choice per task, running Ollama on-device for sensitive code and switching to a hosted model elsewhere in the same workspace.

Getting Started

Install Ollama, pull qwen3-coder:30b if your hardware supports it (or qwen2.5-coder:7b if it doesn't), and confirm the API answers on localhost:11434. Wire it into Continue for private autocomplete, then move to a supervised agent setup — with devstral-small-2:24b if you want a model built for that specifically — when you want the model to plan and ship real changes without your code leaving your machine. You can try the agent workflow at code.unstoppabledomains.com.