Every time you use ChatGPT or Claude, what you type travels to another company's servers. Often that doesn't matter, but with a client's code, work documents or personal data, it's a different story. The alternative is to run the agent on your own computer, with the model, the tools and the data under your control.

In 2026 this is already feasible on an upper-mid-range computer. In this article I explain what exactly “local” means, which is less obvious than it seems, what hardware you need, which programs to use, which models work well as agents and when it's worth it compared with the cloud.

What “local agent” means (and what it doesn't)

An agent has two parts: the model, which does the reasoning, and the agent program, which runs the loop of thinking, using a tool and checking the result. If this idea is new to you, I explain it in what AI agents are. For an agent to be truly local, both parts have to run on your computer.

There's a catch here. Many agent tools install on your computer but send every request to a model in the cloud. Even Ollama, the most popular program for running local models, now offers models that run in its own cloud (Ollama, n.d.-a). When the model does run on your machine, its documentation is clear: the company doesn't see what you type or your data (Ollama, n.d.-c). If you want to make sure nothing leaves your computer, you can switch off the cloud part with the variable OLLAMA_NO_CLOUD=1 (Ollama, n.d.-c).

What computer you need

What matters most is memory: the whole model has to fit into the graphics card's memory (VRAM) or, more slowly, into system RAM. LM Studio, one of the most widely used programs, recommends at least 16 GB of RAM and, on Windows and Linux, at least 4 GB of dedicated video memory; on a Mac it requires an Apple Silicon processor (LM Studio, n.d.-a).

To give you an idea, this is the memory Google lists just for loading the weights of its open Gemma 4 models (Google, 2026a). “Q4” means compressed to 4 bits, which is the usual way to run them on a home machine:

Two details are worth keeping in mind. First, those figures cover only the model: Google notes that they don't include the extra memory needed by the software and the context, that is, the text the model keeps in view while it works (Google, 2026a). Second, with MoE models such as the 26B A4B, only about 4 billion parameters are active at a time, but all 26 billion have to be loaded into memory (Google, 2026a).

And without a graphics card? That's possible too: llama.cpp can run on the processor alone or split the work between the processor and the graphics card (ggml-org, n.d.-a). It's slower, but it works.

The most common mistake: running short on context

Agents need a lot of context, because they keep piling up what they read, the tools they use and the results they get. Ollama sets the context based on the available video memory: with less than 24 GB it allocates only 4,000 tokens by default, whereas for agents and coding tools it recommends at least 64,000 (Ollama, n.d.-b).

If your local agent ignores its tools or loses the thread halfway through a task, this is the first thing to check. To increase the context, just start Ollama with OLLAMA_CONTEXT_LENGTH=64000 ollama serve (Ollama, n.d.-b).

Programs for running the model

• Ollama: the simplest option. It's free, open source (MIT license) and available for Mac, Windows and Linux. You download a model with a single command and it's ready to go on your computer (Ollama, n.d.-e). It lets the model use tools, even several at once, and run complete agent loops (Ollama, n.d.-d).
• llama.cpp: the engine running under the hood of many of these programs. It's more technical, but it also gives you the most control. It runs on the processor and on almost any graphics card, and it supports tool use as well (ggml-org, n.d.-a, n.d.-b).
• LM Studio: the option with a graphical interface, no terminal needed. It marks models that support tool use natively with a hammer icon (LM Studio, n.d.-b).

All three offer an OpenAI-compatible interface, so many programs built for ChatGPT can work with your local model just by changing the server address (LM Studio, n.d.-b; OpenCode, n.d.).

Agent programs that work with local models

• Coding agents: Ollama connects directly to Claude Code, Codex, OpenCode and other tools through its ollama launch command (Ollama, n.d.-e). OpenCode, which is open source, works with Ollama, LM Studio or llama.cpp (OpenCode, n.d.). I compare the most widely used coding agents in Claude Code vs Codex vs Cursor.
• Building your own agents: smolagents, from Hugging Face, works with local models and offers two kinds of agent: one that acts by writing Python code and another that calls tools (Hugging Face, n.d.). Ollama's list of integrations also includes crewAI and Mozilla's any-agent (Ollama, n.d.-e).
• Connecting tools: MCP is an open standard for connecting AI applications to external systems; its creators compare it to a USB-C port for artificial intelligence (Model Context Protocol, n.d.).

Which models to use

For an agent, it isn't enough for the model to be smart: it also has to know how to use tools properly. These are some open models that document that ability:

• gpt-oss-20b (OpenAI): runs with 16 GB of memory and supports function calling, web browsing and Python execution. Apache 2.0 license (OpenAI, n.d.).
• Gemma 4 (Google): available from E2B up to 31B, with built-in tool use and an Apache 2.0 license (Google, 2026b).
• Qwen (Alibaba): Qwen3-8B stands out precisely for tool use (Qwen Team, n.d.-c), while Qwen3.5-9B and Qwen3.6-35B-A3B are designed for agents and coding, also under the Apache 2.0 license (Qwen Team, n.d.-a, n.d.-b).

To compare how well different models, open and closed, use tools, there's a public ranking from the University of California, Berkeley: the Berkeley Function Calling Leaderboard (Patil et al., 2026). It's worth checking before you choose, because it's updated frequently.

Local vs. cloud: when it's worth it

• In favor of local: privacy, no monthly fees or usage limits, and it works without an internet connection.
• Against: it requires a good computer and some setup, and the models that fit on a home machine still lag behind the best cloud models on long, complex tasks.

A report published by the European Data Protection Board (EDPB) sums it up this way: hosting the model yourself can offer more privacy, but it demands substantial computing resources and technical expertise (Barbera, 2025, p. 27).

If what you need is the most capable agent possible and you don't mind using the cloud, take a look at what OpenAI, Google, Microsoft and Anthropic offer.

Privacy and the GDPR

In Germany, the conference of data protection authorities (Datenschutzkonferenz [DSK]) considers technically closed systems preferable from a data protection standpoint. The reason is that with cloud systems, data leaves the environment the user controls and is often transferred to countries outside the EU (Datenschutzkonferenz [DSK], 2024, pp. 5–6).

There are two important caveats. The guidance is aimed at companies and organizations, not individuals, and by “closed system” it means a controlled environment, not necessarily the computer in your home. Moreover, using a local model doesn't exempt you from GDPR obligations if you process other people's data; what it does achieve is keeping that data from being sent to a third party.

Frequently asked questions

Can you run an AI agent locally?

Yes, as long as both the model and the agent program run on your computer. With Ollama, for example, you can switch off the cloud part with OLLAMA_NO_CLOUD=1 (Ollama, n.d.-c). Keep in mind, though, that tools that need the internet, such as web search, won't work offline.

What is the best local AI agent?

It depends on what you need it for. For coding, OpenCode or Claude Code connected to a local model (OpenCode, n.d.; Ollama, n.d.-e); for building your own agents, smolagents (Hugging Face, n.d.). For the model, pick one that handles tools well, such as gpt-oss-20b, Gemma 4 or Qwen (OpenAI, n.d.; Google, 2026b; Qwen Team, n.d.-c).

What is a local agent in AI?

An agent whose model and program both run on your own computer, without sending what you type to another company's servers (Ollama, n.d.-c).

Is there a free local AI agent available?

Yes. Ollama, llama.cpp, OpenCode and smolagents are free and open source, and models such as gpt-oss-20b, Gemma 4 and Qwen are released under the Apache 2.0 license (Ollama, n.d.-e; ggml-org, n.d.-a; Hugging Face, n.d.; OpenAI, n.d.; Google, 2026b).

How much memory do I need?

It depends on the model. Compressed to 4 bits, a 12B model needs about 6.7 GB and a 31B model about 17.5 GB, for the model alone (Google, 2026a). LM Studio recommends at least 16 GB of RAM (LM Studio, n.d.-a).

Why does my local agent ignore its tools?

Often it's because it lacks context: with less than 24 GB of video memory, Ollama allocates only 4,000 tokens by default, whereas for agents it recommends 64,000 (Ollama, n.d.-b).

Keep reading

• →What are AI agents? Explained with examples
• →AI coding agents: Claude Code vs Codex vs Cursor
• →AI agents from OpenAI, Google, Microsoft and Anthropic, compared
• →AI agents built a secret message board — then broke into Hugging Face
• →All blog articles