Open WebUI + vLLM Integration Guide 2026: API URL, Docker, Model Discovery & Verification

If you already run vLLM for local inference, connecting it to Open WebUI is one of the cleanest ways to turn a model server into a comfortable daily chat interface. The important detail is that Open WebUI does not need a special vLLM connector: vLLM exposes an OpenAI-compatible API, and Open WebUI can consume OpenAI-compatible providers.

This guide walks through the complete path from a running vLLM server to a working model inside Open WebUI, including Docker networking, model discovery, API URLs, authentication, Apple Silicon and Linux considerations, tool calling, timeouts, and a simple end-to-end verification flow.

What the Open WebUI + vLLM setup actually looks like

There are two separate services in the common setup:

vLLM's OpenAI-compatible server normally exposes the API under /v1. Open WebUI then sends chat requests to endpoints such as /v1/chat/completions and can query /v1/models to discover models.

This separation matters when troubleshooting. If the model works with curl but does not appear in Open WebUI, the model server may be fine and the problem may instead be the Open WebUI connection URL, Docker networking, authentication, or model-list discovery.

Prerequisites

• A working vLLM installation or container.
• A model supported by your vLLM build and hardware.
• Open WebUI installed locally, in Docker, or on another machine.
• Network access from the Open WebUI process to the vLLM API.
• Enough system RAM or GPU memory to actually load the selected model.

Before touching Open WebUI, make vLLM work by itself. This gives you a known-good baseline.

Step 1: Start a vLLM OpenAI-compatible server

A basic server can be started with:

vllm serve Qwen/Qwen2.5-1.5B-Instruct

The exact model is only an example. Choose a model appropriate for your hardware and task. Current vLLM documentation uses the vllm serve command for launching its OpenAI-compatible server and supports changing the host and port.

For a server that should accept connections from another machine or a Docker container, you may need:

vllm serve Qwen/Qwen2.5-1.5B-Instruct \
  --host 0.0.0.0 \
  --port 8000

Binding to 0.0.0.0 makes the service listen on available network interfaces. It does not mean you should expose port 8000 directly to the public internet. If the server is remote, prefer a private network, VPN, firewall rule, or reverse proxy rather than an unrestricted public endpoint.

Give the model a predictable served name

If you want Open WebUI and other clients to see a shorter model ID, you can configure a served name:

vllm serve Qwen/Qwen2.5-1.5B-Instruct \
  --host 0.0.0.0 \
  --port 8000 \
  --served-model-name qwen-local

Then use qwen-local as the model identifier when testing the API. This is useful when the underlying Hugging Face repository name is long or when you want a stable client-facing name.

Step 2: Verify vLLM before connecting Open WebUI

Do not start debugging Open WebUI until the vLLM endpoint responds.

First check the model list:

curl http://localhost:8000/v1/models

You should receive JSON containing a data array with the served model identifier. If this request fails, Open WebUI cannot fix the problem. Check the vLLM terminal output, model loading state, port, host binding, firewall, and GPU memory.

Then test a chat completion:

curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen-local",
    "messages": [
      {
        "role": "user",
        "content": "Reply with exactly: vLLM is working"
      }
    ],
    "temperature": 0
  }'

If you did not set --served-model-name, replace qwen-local with the model ID returned by /v1/models.

A successful response should contain an assistant message. This two-command check is extremely useful because it separates model-server problems from frontend integration problems.

Step 3: Add vLLM to Open WebUI

In Open WebUI, open Settings → Admin → Connections. Find the OpenAI-compatible API connections section and choose Add Connection.

For a normal host installation where both applications can reach each other through localhost, use:

Save the connection and allow Open WebUI to query the server. If /v1/models is available, the model should normally become selectable in the model picker.

Open WebUI's current provider documentation specifically lists vLLM as an OpenAI-compatible provider and uses http://localhost:8000/v1 as the normal vLLM URL.

Docker networking: the localhost mistake

This is probably the most common connection error when Open WebUI runs in Docker.

Suppose vLLM runs directly on your computer:

Your computer
├── vLLM → localhost:8000
└── Docker
    └── Open WebUI

Inside the Open WebUI container, localhost refers to the container itself, not your host operating system. Therefore this URL can fail:

http://localhost:8000/v1

On Docker Desktop, a common solution is:

http://host.docker.internal:8000/v1

Open WebUI's documentation recommends host.docker.internal when Open WebUI is running in Docker and the model server is running on the host.

When vLLM is another container

If both services are containers on the same Docker network, do not use localhost or necessarily host.docker.internal. Use the vLLM container or Compose service name.

http://vllm:8000/v1

For example, if your Compose service is named vllm, Docker's internal DNS can resolve that name for Open WebUI when both services share a network.

Remote vLLM server

If vLLM runs on a different Linux server, workstation, or GPU machine, the Open WebUI connection should point to that machine's reachable address:

http://192.168.1.50:8000/v1

For a remote production-style setup, HTTPS behind a reverse proxy is preferable. Do not assume that adding an API key alone makes an exposed vLLM service secure. Current vLLM documentation explicitly notes that its API-key authentication does not protect every endpoint on the HTTP server, so network-level controls and a properly configured reverse proxy remain important.

Why your model is not appearing in Open WebUI

Start with the API rather than repeatedly changing Open WebUI settings.

1. Check the model endpoint

curl http://YOUR-VLLM-HOST:8000/v1/models

If the command fails, fix network access first.

2. Check the model identifier

The identifier Open WebUI receives must correspond to what vLLM actually serves. If you use --served-model-name, use that stable name.

3. Check the Open WebUI container's network path

When Open WebUI runs in Docker, test connectivity from inside the container rather than from your laptop shell. A server that responds from the host does not automatically prove that the container can reach it.

4. Use the Model IDs allowlist when necessary

Open WebUI's OpenAI-compatible provider documentation notes that if a provider does not expose a usable /models endpoint, you can manually specify model IDs. vLLM normally exposes model discovery, so manual configuration should not be your first fix, but it is useful for unusual proxies or restricted deployments.

Apple Silicon: what changes?

The Open WebUI side of the integration does not fundamentally change on a Mac. The important difference is the vLLM backend and hardware support.

If you are running vLLM through a Metal-oriented setup, validate that the particular vLLM build, model architecture and backend are supported before spending time on Open WebUI. Open WebUI only sees an HTTP API; it cannot make an unsupported model backend work.

For an Apple Silicon workflow, the clean debugging order is:

1. Start the model server.
2. Verify /v1/models.
3. Verify one chat completion with curl.
4. Connect Open WebUI.
5. Only then troubleshoot UI-specific behavior.

For a broader Apple Silicon serving discussion, see our vLLM on Apple Silicon guide.

Linux and NVIDIA GPU servers

Linux with a supported NVIDIA GPU is a common environment for vLLM because vLLM is designed around high-performance model serving and batching. The Open WebUI connection remains the same:

http://SERVER-IP:8000/v1

The practical difference is usually network topology. If Open WebUI is on the same server but in Docker, use the appropriate Docker network path. If it is on another machine, expose only the required interface and protect the service with firewall rules, private networking, or a reverse proxy.

Tool calling: connecting is not the same as being agent-ready

A model appearing in Open WebUI proves that model discovery works. It does not prove that tool calling will work correctly.

Tool use depends on the model, its chat template, vLLM's supported tool-calling configuration, and Open WebUI's request handling. vLLM supports OpenAI-style chat completions and tool-related parameters, but model-specific support still matters.

If ordinary chat works but tools fail, test these layers independently:

1. Confirm ordinary /v1/chat/completions works.
2. Confirm the selected model is intended for tool calling.
3. Check the vLLM tool-calling documentation for the model's required parser or configuration.
4. Check the request and response in logs.
5. Only then investigate Open WebUI tool settings.

Do not interpret a raw JSON-looking response as automatically proving that vLLM is broken. A model can generate JSON-like text without producing a structured tool call, and a frontend can also mishandle an otherwise valid tool-call response.

Timeouts and slow model loading

A GPU model may take significantly longer to initialize than a normal cloud API. Open WebUI's current vLLM integration documentation notes that its model-list client timeout can be increased with:

AIOHTTP_CLIENT_TIMEOUT_MODEL_LIST=30

This is useful when the vLLM server is reachable but slow during startup or model initialization.

Do not increase every timeout blindly. First determine whether the delay is model loading, network latency, a dead endpoint, or GPU memory pressure.

Common errors and practical fixes

Privacy and security considerations

One major reason to combine local vLLM with Open WebUI is keeping inference on infrastructure you control. But local does not automatically mean private.

• Protect remote vLLM endpoints with a firewall or private network.
• Use HTTPS when traffic crosses an untrusted network.
• Do not commit API keys to Git repositories or Docker Compose files shared publicly.
• Remember that Open WebUI can have access to conversations, files, tools and external connections depending on your configuration.
• If using web search, MCP, external tools or cloud providers alongside vLLM, some data may leave your local environment.

If you expose vLLM directly to the internet, treat it as an application service rather than a harmless local port. Current vLLM documentation specifically warns that API-key protection does not cover every endpoint.

How to verify the complete setup

Use this sequence instead of relying only on the Open WebUI interface:

1. Server check: curl http://HOST:8000/v1/models.
2. Inference check: send one /v1/chat/completions request.
3. Open WebUI check: add the OpenAI-compatible connection using the correct network address.
4. Discovery check: confirm the vLLM model appears in the model selector.
5. Chat check: send a short deterministic prompt.
6. Long-context check: test a realistic prompt rather than immediately loading a huge document.
7. Tool check: only if the selected model and vLLM configuration support tool calling.

If step 1 fails, investigate vLLM. If step 1 works but step 2 fails, investigate model serving. If steps 1 and 2 work but Open WebUI cannot discover the model, investigate the connection URL and network path. This layered approach saves a lot of trial and error.

Open WebUI + vLLM vs Open WebUI + Ollama

If you mainly want a simple personal local chatbot, Ollama can be easier. If you already operate vLLM or want a dedicated inference server, Open WebUI is a useful frontend without requiring you to replace the serving layer.

Useful GyanAangan follow-up guides

• How to Connect Open WebUI to Ollama in Docker
• Open WebUI Tool Calling & Context Length Troubleshooting
• Open WebUI MCP Setup Guide
• vLLM on Apple Silicon in 2026
• OpenCode + Ollama in 2026

FAQ

What URL should I use for vLLM in Open WebUI?

For the normal vLLM OpenAI-compatible server, use http://localhost:8000/v1 when Open WebUI can reach the host directly. If Open WebUI runs in Docker and vLLM runs on the host, use http://host.docker.internal:8000/v1 on supported Docker Desktop environments. For another machine, use that server's reachable hostname or IP followed by /v1.

Why does vLLM work with curl but not Open WebUI?

The most common reason is that curl is running from a different network namespace. A curl command on the host can reach localhost:8000 while an Open WebUI container cannot. Test the address from the same environment where Open WebUI runs.

Why is my vLLM model not showing in Open WebUI?

Check /v1/models, verify the served model name, and confirm that the Open WebUI connection points to the correct /v1 URL. If your provider does not expose model discovery, Open WebUI supports manually specifying model IDs.

Do I need an API key?

No, not for a basic local vLLM server unless you configure one. If you expose the service beyond a trusted local network, authentication should be part of a broader security design rather than your only protection.

Can Open WebUI use vLLM for RAG?

Yes, but RAG has more requirements than ordinary chat. Open WebUI's provider compatibility documentation lists embeddings as an optional endpoint for providers used for RAG. You may therefore need a separate embedding provider depending on your architecture.

Can I use tool calling with vLLM?

It can work when the model and vLLM configuration support structured tool calling. If ordinary chat works but tools do not, inspect model-specific tool-call support and vLLM parser configuration before blaming the Open WebUI connection itself.

Final checklist

• vLLM starts without model-loading errors.
• /v1/models returns the expected model.
• /v1/chat/completions produces a normal response.
• Open WebUI uses the correct network address.
• Docker deployments do not incorrectly use container-local localhost.
• The model ID matches the vLLM served name.
• Slow startup has an appropriate Open WebUI timeout.
• Remote deployments are protected by network controls and HTTPS where appropriate.
• Tool calling is tested separately from ordinary chat.

The key idea is simple: treat vLLM and Open WebUI as two independent layers. Prove the vLLM API works first, connect it using the correct network path, verify model discovery, and only then investigate advanced features such as RAG or tool calling. That approach makes the setup much easier to diagnose and gives you a reusable OpenAI-compatible local inference endpoint for other clients too.

Official sources

• Open WebUI: vLLM provider connection
• Open WebUI: OpenAI-compatible providers
• vLLM: OpenAI-Compatible Server
• vLLM: Quickstart