How to Use Local LLM in VS Code

Learning how to use local LLM in VS Code means three steps: install Ollama, pull a coding-friendly model, then connect it with either the official Ollama VS Code extension (Chat model picker) or Continue (~/.continue/config.yaml). Local models need no cloud sign-in for chat; your prompts stay on the machine that runs Ollama.

That is the whole path. The rest of this guide covers model picks for coding, VRAM and Mac hardware, tab autocomplete, and the limits Microsoft still documents for offline Copilot-style features.

How to use local LLM in VS Code (two paths)

Pick one primary setup. You can install both, but start with one so debugging stays simple.

Microsoft’s AI language models docs state that bring-your-own / local models work without a GitHub account and without a Copilot plan for chat, including offline Ollama use. The same page notes that semantic search, inline suggestions, and embeddings-dependent features are not available through BYOK alone. The built-in Ollama provider is deprecated; prefer the Marketplace extension from the Ollama publisher.

Local LLM setup: install Ollama first

1. Install Ollama for your OS from the official site, then confirm the service answers:

   curl http://127.0.0.1:11434

   You should see a short “Ollama is running” style response.
2. Pull at least one model with an exact tag (Continue docs warn that ollama run starts a chat you do not need for an IDE):

   ollama pull qwen2.5-coder:7b
   ollama pull qwen2.5-coder:1.5b
   ollama list

3. Leave Ollama running while VS Code is open. Remote hosts work if you set apiBase / host env vars, but the default local URL is enough for a laptop.

For runner trade-offs (Ollama vs llama.cpp vs LM Studio), see Ollama vs llama.cpp vs LM Studio. For which open models to keep on disk, see open source LLM models to run locally.

Path A: official Ollama extension in VS Code Chat

Per Ollama’s VS Code integration docs:

1. Use Visual Studio Code 1.127 or newer (earlier builds do not cancel provider requests reliably).
2. Install the Ollama extension from the VS Code Marketplace (publisher: Ollama).
3. Start Ollama and ensure ollama list shows your models.
4. Open Chat in VS Code, open the model picker under the chat input, and choose a model in the Ollama section.

The extension discovers models at http://127.0.0.1:11434 by default. Local models do not require ollama signin; that command is for Ollama’s cloud tags only.

Context length gotcha: VS Code may display a model’s maximum context even when Ollama allocated a smaller window at runtime. Ollama’s docs recommend setting context length to at least 64k in Ollama Settings for local Chat work, then reloading the VS Code window and resending the prompt.

If models are missing: run Ollama: Refresh Models, then Ollama: Diagnose Models, and read the Ollama output channel. Confirm the daemon is up and that tags match ollama list.

Path B: Continue.dev for chat, edits, and tab complete

Continue’s Ollama guide and provider reference use config.yaml (the older config.json format is deprecated).

1. Install the Continue extension from the Marketplace.
2. Create or edit ~/.continue/config.yaml (Windows: under your user Continue config path).
3. Point chat at a larger coder model and autocomplete at a tiny one:

name: Local Config
version: 0.0.1
schema: v1

models:
  - name: Qwen Coder Chat
    provider: ollama
    model: qwen2.5-coder:7b
    apiBase: http://localhost:11434
    roles:
      - chat
      - edit
      - apply
  - name: Fast autocomplete
    provider: ollama
    model: qwen2.5-coder:1.5b
    roles:
      - autocomplete

4. Reload VS Code. Enable tab autocomplete in Continue’s status UI, and keep "editor.inlineSuggest.enabled": true in VS Code settings.
5. Disable competing completion providers if ghost text never appears; Continue’s autocomplete troubleshooting calls that out explicitly.

If Continue reports “Model requires more system memory”, lower defaultCompletionOptions.contextLength (their Ollama provider page shows an example at 2048). Tag mismatches return 404 model not found until you ollama pull the exact ID shown in ollama list.

For tool / agent workflows, set capabilities: [tool_use] when auto-detection fails. Not every “tools” marketing claim works in practice; if Agent mode still refuses, switch to a model with confirmed tool calling.

Local LLM for coding: model picks

Use Qwen coder tags when you want a dedicated coding stack; use Gemma or Llama instruct builds when you want a general assistant that still handles code. Unsloth GGUF builds are useful when you prefer llama.cpp-side quants before importing them into Ollama.

Exact download sizes change by tag; always check the Ollama library page for the tag you pull. For Gemma 4 specifically, see how to run Gemma 4 locally.

Local LLM hardware and VRAM

Hardware decides whether Chat feels snappy or thrashing. Match weight size plus KV cache to free GPU VRAM or Mac unified memory — start with how much VRAM you need for an LLM.

Rough planning bands for a coding laptop:

• About 8 GB: tiny autocomplete (1.5B-class) plus a small chat model; keep context short.
• About 16 GB: comfortable 7B–12B chat at 4-bit with moderate context.
• About 24 GB: larger coder / reasoning tags; raise context carefully.
• Mac unified memory: treat total RAM like VRAM, leave headroom for the IDE and browser; see MLX vs GGUF on Mac if you prefer Apple-native runners outside VS Code.

Quant choice matters as much as parameter count. When you dig into GGUF bits, use GGUF quantization explained.

Local LLM for Mac

On Apple Silicon, Ollama still works as the VS Code backend for both the official extension and Continue. Install Ollama’s Mac app, pull the same tags, and point extensions at 127.0.0.1:11434. If you also use LM Studio or MLX for other apps, keep one process bound to the port VS Code expects so model discovery does not race.

VS Code itself does not need a special Mac-only extension for the Ollama Chat path — the same Marketplace package applies. What changes is memory pressure: large context plus Xcode, Docker, or browsers can push unified memory into swap and make autocomplete stutter.

Local LLM use cases inside the editor

Once a model is selected, typical local workflows are:

• Explain a selected function without pasting it into a web chat.
• Generate tests or docstrings with the file open as context (Continue @ / VS Code Chat attachments, depending on the UI).
• Apply multi-file edits when the model supports tools and you enable agent-style modes.
• Stay offline on airplanes or air-gapped machines (chat works; Copilot cloud features do not).

Keep secrets out of prompts the same way you would with a cloud model. Local hosting removes network egress to a vendor, but logs, shared GPUs, and synced workspaces can still leak text.

FAQ

Do I need GitHub Copilot to use a local LLM in VS Code?

No for chat with BYOK / the Ollama extension. Microsoft documents that local models work without a GitHub account and without a Copilot plan. You still need Copilot services for features tied to their cloud: inline suggestions, semantic search, and embeddings-based helpers.

Is Continue better than the official Ollama extension?

They solve overlapping but different jobs. The Ollama extension is the shortest path into VS Code Chat. Continue shines when you want local tab autocomplete and role-based chat/edit models in one YAML file. Many developers run Ollama once and pick the IDE layer they prefer.

Which model should I pull first for local LLM for coding?

Start with a small coder autocomplete model (qwen2.5-coder:1.5b if you use Continue) and a mid-size chat coder (qwen2.5-coder:7b or a current Qwen3 / Gemma instruct tag that fits your VRAM). Confirm tools if you need agent mode.

Why don’t my Ollama models show up in the VS Code picker?

Ollama is probably stopped, the tag is wrong, or the extension needs a refresh. Run ollama list, then Ollama: Refresh Models and Ollama: Diagnose Models. For Continue, match model: exactly to ollama list and verify curl http://localhost:11434.

Can I use llama.cpp instead of Ollama behind VS Code?

Yes in principle if you expose an OpenAI-compatible endpoint and wire a custom provider, but the documented happy path for VS Code Chat and Continue is Ollama’s HTTP API on port 11434. llama.cpp remains excellent as a runner; Ollama is just the glue most VS Code guides assume.

How do I keep context long enough for a big repo?

Raise Ollama’s context length (Ollama’s VS Code docs suggest at least 64k for Chat), lower Continue’s contextLength only when you hit memory errors, and prefer models whose cards actually support the window you set. Long context burns VRAM fast — see the VRAM guide before pushing 128k on a 16 GB laptop.