Self-Host Tabby AI Code Completion with Docker and VS Code (2026 Guide)
You can get GitHub Copilot-style inline code suggestions without sending every keystroke to a hosted coding service. The trick is to separate autocomplete from agentic coding. A coding agent plans changes and edits files; an autocomplete server predicts what belongs at the cursor. Tabby is built for the second job: you host the server, choose a completion model, and connect your editor to it.
This guide sets up a self-hosted Tabby server with Docker and an NVIDIA GPU, connects Visual Studio Code, verifies the completion API, and covers the practical fixes that matter when the extension connects but suggestions do not appear. An Apple Silicon route is included too. The commands follow Tabby's official documentation checked on 21 November 2026; they are documented setup examples, not claims of a benchmark run.
What you are building
The architecture is intentionally simple:
- Tabby server: hosts the completion model and exposes an HTTP API.
- IDE extension: sends the code around your cursor to your Tabby endpoint and displays returned suggestions.
- Persistent data directory: stores Tabby's application data and configuration outside the container.
Tabby supports separate completion, chat, and embedding models. Completion is primarily a fill-in-the-middle (FIM) task: the editor sends the text before and after the cursor, and the model predicts the missing section. Chat is optional; you can run autocomplete without enabling the chat panel.
This is not a full autonomous coding agent. It does not replace tools such as Aider or OpenHands for multi-file tasks, shell commands, or repository-wide changes. If you want that workflow, see our guides to Aider with local models and OpenHands with Ollama.
Prerequisites
- A Linux x86-64 machine for the Docker/NVIDIA path.
- Docker Engine and permission to run containers.
- An NVIDIA GPU and a working NVIDIA driver. Install the NVIDIA Container Toolkit so Docker can pass the GPU into the container.
- Visual Studio Code on the computer where you want completions.
- Enough disk space for the models Tabby downloads. Model size and memory needs vary; check the official model registry before choosing a larger model.
Hardware guidance: Tabby's model registry recommends roughly NVIDIA T4, 10-series or 20-series GPUs, or Apple Silicon such as M1 for its 1B–3B completion models. Its guidance for 7B–13B completion models is a stronger class such as V100/A100 or NVIDIA 30/40-series GPUs. Treat these as recommendations, not guaranteed minimums or a promise of a particular tokens-per-second result. A smaller model is usually the sensible first test.
Step 1 — Confirm Docker can access the GPU
Install the NVIDIA driver for your distribution, then install and configure the NVIDIA Container Toolkit using NVIDIA's current instructions. Restart Docker if the installation guide asks you to.
nvidia-smi
docker version
nvidia-smi should show the GPU and driver. This only confirms the host driver is visible; it does not by itself prove that Docker GPU passthrough is configured. If Docker cannot create a GPU-enabled container later, return to the Container Toolkit installation guide before changing Tabby settings.
Step 2 — Start the Tabby server
Tabby's official Docker quick start uses the registry.tabbyml.com/tabbyml/tabby image, a persistent ~/.tabby directory mounted at /data, and a completion model plus an optional chat model. The command below follows that layout but binds the port to loopback, so the service is reachable from the same machine rather than being exposed to every network interface by default.
mkdir -p "$HOME/.tabby"
docker run -d \
--name tabby \
--restart unless-stopped \
--gpus all \
-p 127.0.0.1:8080:8080 \
-v "$HOME/.tabby:/data" \
registry.tabbyml.com/tabbyml/tabby \
serve \
--model StarCoder-1B \
--chat-model Qwen2-1.5B-Instruct \
--device cuda
The example enables both completion and chat. If you only need inline autocomplete, you can omit --chat-model Qwen2-1.5B-Instruct; the chat panel then will not be enabled. Tabby documents these model IDs in its Docker quick start and model registry. The server may need to download model files on its first start, so allow time and disk space.
Check the container and its logs:
docker ps --filter name=tabby
docker logs -f tabby
Look for Tabby starting successfully and listening on port 8080. Then open http://localhost:8080 in a browser on the server machine.
Why the volume matters: $HOME/.tabby:/data keeps Tabby's data outside the disposable container layer. Recreating the container should not erase the data stored in that mounted directory. Do not remove the directory when upgrading or troubleshooting unless you intentionally want to reset the instance.
Step 3 — Create the administrator account
On the first visit, register an account. Tabby's documentation states that the first registered account becomes the administrator/owner. After signing in, use the homepage to find the endpoint and the credentials needed by IDE extensions.
- Open
http://localhost:8080. - Complete the first-account registration.
- Sign in and locate the endpoint and personal access token shown for connecting an editor.
- Store the token in a password manager. Do not paste it into a source file, commit it to Git, or share it in screenshots.
If this is a shared server, do the initial registration before allowing other people to reach the instance. Review the user and group settings before inviting teammates.
Step 4 — Install the Tabby extension in VS Code
In VS Code, open the Command Palette or Quick Open and run the extension install command:
ext install TabbyML.vscode-tabby
You can also find the extension through the Visual Studio Marketplace. Once installed:
- Open the Command Palette with
Ctrl+Shift+Pon Windows/Linux orCmd+Shift+Pon macOS. - Run Tabby: Connect to Server....
- Enter
http://localhost:8080if VS Code and Tabby are on the same computer. - When prompted, provide the personal access token from the Tabby homepage.
- Check the Tabby status indicator in the editor. It should show that the server is connected.
For a machine-readable configuration, Tabby's agent reads ~/.tabby-client/agent/config.toml after the extension has been launched. A minimal configuration looks like this:
[server]
endpoint = "http://localhost:8080"
token = "YOUR_TABBY_TOKEN"
Replace the placeholder locally; never commit a real token. Also note that settings configured in the IDE take precedence over the config file. If you change the file and nothing happens, check whether the VS Code extension settings already contain an endpoint or token.
Step 5 — Verify the completion API before judging the extension
Tabby exposes a completion endpoint at POST /v1/completions. You can inspect the interactive API page at http://localhost:8080/swagger-ui/ or send a small request from a terminal:
curl -sS -X POST \
'http://localhost:8080/v1/completions' \
-H 'accept: application/json' \
-H 'Content-Type: application/json' \
-d '{
"language": "python",
"segments": {
"prefix": "def add(a, b):\n ",
"suffix": "\n"
}
}'
The API should return a JSON response containing a completion choice when the model is loaded and the request is accepted. If your deployment requires authentication for this endpoint, add the bearer token header used by your instance:
-H "Authorization: Bearer $TABBY_TOKEN"
Do not put the token directly into shell history on a shared machine. You can export it for the current shell session or use the Swagger UI after signing in.
This test separates two failure classes. If the API request fails, investigate the server, model, network, or authentication. If the API works but VS Code does not show suggestions, investigate the extension endpoint, token, trigger mode, and editor context.
Step 6 — Choose a model that fits the machine
Tabby maintains a registry of completion, chat, and embedding models. The roles are not interchangeable:
- Completion model: predicts code around the cursor. Prioritize low latency and FIM compatibility.
- Chat model: handles conversational questions and explanations in the IDE. It is optional for autocomplete-only use.
- Embedding model: supports semantic representations used by context-aware features. It is not the model that writes the inline completion.
Start with the small model from the quick-start command and confirm the whole path works. Then compare another model from the model registry using the same project, language, and prompt situations. Model licenses differ, so check the license for the exact model you deploy; the server being open source does not make every model's weights license identical.
Tabby also supports HTTP model connectors, including Ollama. This is useful if you already operate an Ollama service and want it to provide the models instead of having Tabby launch its own local model subprocesses. For example, the official Ollama connector documentation shows this completion configuration:
[model.completion.http]
kind = "ollama/completion"
model_name = "codellama:7b"
api_endpoint = "http://localhost:11434"
prompt_template = "<PRE> {prefix} <SUF>{suffix} <MID>"
That prompt template is the CodeLlama-style example from Tabby's documentation, not a universal template for every coding model. Use the template expected by your chosen model. If Tabby runs directly on the same host as Ollama, the documented localhost endpoint can work. If Tabby runs inside Docker, localhost means the Tabby container itself, not the host. Configure a reachable host address or a Docker host-gateway route instead; do not assume the container can reach the host's Ollama port automatically. See the official Ollama connector guide before changing the model backend.
Step 7 — Use Tabby in a real coding workflow
- Open a project you can safely experiment in.
- Create a small function or begin a familiar pattern, then pause after the opening lines.
- Wait for the inline suggestion. Accept only the part that makes sense; otherwise dismiss it and keep typing.
- Try a second language or a different file type supported by your editor.
- Use the Tabby chat panel only if you started the server with a chat model.
- Run the project's formatter, linter, and tests after accepting generated code.
Autocomplete quality is highly dependent on the language, model, context, and latency. A suggestion that looks plausible can still use the wrong API, omit an edge case, or introduce a security bug. Treat it as a fast draft, not a code reviewer.
Optional — Run Tabby on Apple Silicon
If you are using an Apple M-series Mac rather than an NVIDIA Linux host, Tabby's official quick start documents a Homebrew installation using Apple's Metal acceleration:
brew install tabbyml/tabby/tabby
tabby serve \
--device metal \
--model StarCoder-1B \
--chat-model Qwen2-1.5B-Instruct
Open http://localhost:8080 and follow the same account and VS Code connection steps. Tabby's documentation cautions that M1/M2 compute is more appropriate for individual use; for a shared team instance it recommends considering a CUDA- or ROCm-capable host. Check the current installation guide for platform-specific changes before installing.
Optional — Put a shared Tabby server behind HTTPS
The loopback-bound Docker command above is for same-machine use. For a team, do not simply publish port 8080 to the public internet. Put Tabby behind a reverse proxy with TLS, restrict who can reach it, and use individual accounts/tokens. Tabby's reverse-proxy guide specifically calls out WebSocket support because the Answer Engine uses WebSockets for streaming.
An Nginx location based on Tabby's documented example looks like this:
location / {
proxy_pass http://127.0.0.1:8080;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
Configure the public hostname and TLS certificate in the surrounding Nginx server block, and set Tabby's External URL to the real HTTPS address. Keep the upstream port private. For a small private team, a VPN or private overlay network can reduce exposure further, but still use authentication and firewall rules.
Troubleshooting: fixes for the common failure points
1. Docker says it cannot select a GPU device
Confirm nvidia-smi works on the host, then check the NVIDIA Container Toolkit installation and Docker runtime configuration. Restart Docker after changing the runtime. Do not change the model or Tabby configuration until a basic GPU-enabled container can start.
2. The browser cannot open localhost:8080
Run docker ps and docker logs tabby. Check that the container is running and that port 8080 is mapped. If another process already uses the port, choose a different host port, for example -p 127.0.0.1:8081:8080, then use http://localhost:8081 in the browser and editor.
3. VS Code says Disconnected
Verify the endpoint from the computer where the extension is running. If the server is on another machine, localhost is the wrong address; use the server's reachable private hostname or HTTPS URL. Check the token and remember that IDE settings override the agent config file.
4. The server connects but no suggestions appear
Test /v1/completions directly. If that works, check the extension's trigger mode and try a supported language in a normal source file. Tabby's default mode waits briefly after you stop typing; manual mode requires the configured keyboard shortcut. The model may also decide that the current context does not need a suggestion.
5. Requests time out or suggestions arrive too slowly
Check the server logs and GPU utilization. A large model on a CPU can be slow. Try a smaller completion model, reduce competing workloads, or use a supported acceleration backend. Tabby's troubleshooting guide documents CUDA, ROCm, and Metal device options; use the one that matches the host rather than copying a GPU flag from another platform.
6. Chat is missing but autocomplete works
That is expected if the server was started without --chat-model. Add a supported chat model and restart the server if you want the chat panel. Chat and completion are separate model roles.
7. Ollama works on the host but Tabby in Docker cannot reach it
Inside a container, localhost:11434 points back to that container. Configure Ollama to listen on an address reachable from the Tabby container, then use the host's reachable address in Tabby's model configuration. On Docker Desktop, host.docker.internal is commonly available; on Linux, a host-gateway mapping may be needed. Restrict Ollama's listener to trusted interfaces and never expose it openly just to make the connection work.
Security and privacy checklist
- Keep the service bound to loopback for personal use. For remote access, use a VPN/private network or a reverse proxy with HTTPS and firewall restrictions.
- Use individual Tabby accounts and tokens. Rotate a token if it appears in a log, screenshot, shell history, or repository.
- Do not assume “self-hosted” means code never leaves the developer's computer. The editor sends relevant context to the Tabby endpoint. If that endpoint is on another server, code travels to that server.
- If you configure an external model provider through an HTTP connector, review that provider's data handling terms. The local Tabby UI does not make a remote model call local.
- Review model licenses before using weights in a commercial product or company workflow.
- Keep the persistent data directory backed up and restrict filesystem permissions on the host.
- Tabby's extension documentation describes aggregated usage collection and provides a setting to disable it. Review the current extension configuration guide if you want to opt out.
- Review generated code, run tests, and scan changes for secrets and unsafe patterns before merging.
FAQ
Is Tabby a replacement for Cursor, Aider, or OpenHands?
Not one-for-one. Tabby is strongest as a self-hosted completion service with IDE integrations and additional chat/context features. Aider and OpenHands are more directly aimed at agentic repository work. You can use the tools for different jobs.
Can I use Tabby with Ollama?
Yes. Tabby's official model HTTP API documentation includes Ollama connectors for completion, chat, and embeddings. The completion prompt template must match the selected model, and Docker networking must be configured correctly if the services are in separate containers.
Does Tabby need an NVIDIA GPU?
No. Tabby documents CPU-only Linux operation and a Metal path for Apple Silicon, as well as CUDA and Vulkan options. Performance depends on the model and hardware; the NVIDIA Docker path is not the only supported installation route.
Can I use it with VS Code on a different computer?
Yes, if the editor can reach the server endpoint and you configure the correct URL and token. Use a private network or HTTPS reverse proxy; do not expose an unauthenticated inference port to the internet.
Is every model in the registry under the same license?
No. Check the license listed for the exact completion, chat, or embedding model you choose. Tabby software licensing and model-weight licensing are separate questions.
Official references
- Tabby Docker installation
- Tabby Homebrew installation for Apple Silicon
- Registering the first Tabby account
- Connecting IDE/editor extensions
- Tabby model registry and hardware guidance
- Tabby Ollama model connector
- Tabby extension troubleshooting
- Deploying Tabby behind a reverse proxy
- Tabby official releases
- NVIDIA Container Toolkit installation
Related GyanAangan guides
- Continue + Ollama in VS Code: local chat, edits, and autocomplete — compare another route for bringing local models into the editor.
- Aider with Ollama and LM Studio — use a terminal-based pair programmer when you need file edits and Git-aware workflows.
- Benchmark local GGUF models properly — build a more repeatable way to compare local inference performance.
Bottom line: Tabby gives you a dedicated, self-hosted path for inline code completion. Start with a small model, keep the endpoint private, verify the API before debugging the editor, and treat every accepted suggestion like code from a junior teammate: useful, but still subject to review and tests.
