Kimi K3.1 Pro Open Weights Arrive: vLLM Day-0 Support, Hardware Reality and Hosted APIs (November 2026)

Updated: November 5, 2026. A frontier-scale model is much more useful to infrastructure teams when its weights and its serving path arrive together. Moonshot AI has now published the Kimi K3.1 Pro weights on Hugging Face, and the vLLM team published day-zero serving support on November 4. On November 5, Together AI also listed the model on its serverless inference platform.

This is a meaningful change from the October launch: developers are no longer limited to Moonshot’s hosted products and API. They can inspect the checkpoint, evaluate a self-hosted deployment, or use another hosted inference provider. But “open weights” does not mean “runs on a workstation.” The published model is a 2.8-trillion-parameter mixture-of-experts system, and the current vLLM recipe describes an eight-GPU deployment baseline for its MXFP4 checkpoint.

In short: Kimi K3.1 Pro is now downloadable under Moonshot’s custom license; vLLM has a hardware-specific serving path; and hosted inference is available if you do not operate a large accelerator cluster. The important decisions are licensing, runtime version, GPU capacity, and whether your workload actually benefits from a million-token context.

What changed between November 4 and November 5?

• October 24: Moonshot introduced Kimi K3.1 Pro and K3.1 Pro Highspeed, describing Pro as a post-trained version of Kimi K3 and promising the full weights by November 4. The original announcement is available in the Kimi K3.1 Pro technical blog.
• November 4: The official Moonshot Hugging Face repository became available with the model card, configuration, weights and a dedicated license. The vLLM engineering team also published day-zero support.
• November 5: Together AI added moonshotai/Kimi-K3.1-Pro to serverless inference. Its official changelog lists a 1,048,576-token context, NVFP4 serving quantization, text and image input, function calling, structured outputs, and per-million-token pricing.

An independent report from Automater News also records the November 4 weights release and vLLM support. That corroborates the release timeline; it is not an independent reproduction of the performance figures discussed below.

What is Kimi K3.1 Pro?

Moonshot describes Kimi K3.1 Pro as an open-weight, natively multimodal agent model and a post-trained successor to Kimi K3. The official model card documents the following architecture:

The important architecture change from Kimi K3 is in the 24 full-attention layers. K3.1 Pro adds a Gated DeepSeek Sparse Attention indexer that selects up to 2,048 previous tokens for the main attention operation. The other 69 layers retain Kimi Delta Attention. This is intended to reduce the amount of key-value data that must be read during long-context decoding.

Do not confuse 104B active parameters with a 104B model download. The model still contains 2.8T total parameters. Mixture-of-experts routing reduces the experts used for an individual token; it does not make the complete checkpoint disappear from the deployment. The model card also identifies the vision encoder and the MXFP4/MXFP8 format. See the model card and configuration for the exact details.

Why the vLLM day-zero release matters

Supporting a model with a new attention layout is more than adding its name to a model registry. The runtime has to understand its attention kernels, cache state, parallelism strategy, model-specific request format and tool-call parsing. The vLLM implementation addresses those pieces rather than treating K3.1 Pro as an ordinary dense transformer.

The vLLM team describes three kinds of state that must be managed together: the MLA latent, sparse-attention indexer keys, and Kimi Delta Attention state. Its packed KV-cache layout places them in a block-oriented structure so that the runtime can manage the different page sizes and transfer blocks between prefill and decode workers.

The release also includes token-sharded prefill, decode context parallelism, fused decode paths and a Kimi-specific reasoning/tool parser. The published support matrix spans NVIDIA GPUs, AMD GPUs, Google TPUs and Alibaba T-Head Zhenwu PPUs. The actual tested configurations are hardware-specific; this is not a claim that every accelerator in those families can run the model with identical performance.

What the published performance numbers mean

vLLM reports up to 1.69× faster time to first token for a 340K-token prompt when comparing token-sharded with head-sharded prefill on eight GB300 GPUs. Its engineering post also reports AgentX results of up to 375.9 output tokens per second per user and 37.8K total tokens per second per GPU on GB300 at different points on the latency/throughput frontier.

These are vLLM-reported measurements from specific hardware, software and benchmark configurations. They are not independent tests, do not describe a typical workstation, and should not be used as a direct forecast for another GPU cluster. Read the methodology and configuration in the vLLM engineering report before comparing them with another provider’s results.

The hardware reality: an eight-GPU baseline, not a laptop install

The current vLLM Kimi K3.1 Pro recipe, updated November 5, identifies vLLM 0.34.0 or later and gives an MXFP4 deployment baseline of TP8: eight GPUs across two GB300 trays or one B300 node, or one MI355X node. Its default NVIDIA example uses two nodes with tensor parallelism across eight GPUs. The recipe also documents separate configurations for Zhenwu PPU systems.

The checkpoint uses MXFP4 for routed-expert weights while attention, shared experts and the vision tower remain BF16. The recipe describes a very large artifact footprint, with routed-expert weights alone around 1.56 TB on disk. Actual usable context and concurrency depend on GPU memory, KV-cache format, parallelism, batch settings and the serving configuration.

Version warning: as checked on November 5, the vLLM GitHub releases page lists v0.33.0 as the latest tagged stable release, while the Kimi recipe calls for v0.34.0+. Do not assume that a generic pip install vllm on the stable release is sufficient. Start from the dedicated Kimi image and exact hardware recipe, then pin the image digest and model revision for a repeatable deployment. See the vLLM release list and the current Kimi recipe.

How to evaluate a self-hosted deployment

This is an infrastructure evaluation path for teams that already have a supported multi-GPU system. It is not a copy-and-paste setup for a MacBook, gaming PC or small VPS.

1. Confirm the target hardware. Check the current vLLM recipe for the accelerator, node count, interconnect and memory configuration you actually have. The documented baseline is TP8 for the MXFP4 checkpoint. Do not infer that eight unrelated GPUs on a slow network will behave like the validated node or tray configuration.
2. Read the model license before downloading. The repository includes a separate Kimi K3.1-Pro License. Check whether your planned use is internal inference, a commercial API, a hosted product or a derivative model.
3. Use the current serving recipe. Follow the dedicated vLLM deployment recipe. Its NVIDIA example uses the Kimi-specific container image, fast safetensors loading, tensor parallelism and the Kimi tool/reasoning parsers. The recipe provides the node-rank and master-address settings needed for multi-node deployment.
4. Start with the documented serving flags. The vLLM engineering post shows the model-specific options below. Use the matching image and topology from the recipe rather than running this command on an arbitrary machine.

vllm serve moonshotai/Kimi-K3.1-Pro \
  --tensor-parallel-size 8 \
  --trust-remote-code \
  --load-format fastsafetensors \
  --enable-auto-tool-choice \
  --tool-call-parser kimi_k3 \
  --reasoning-parser kimi_k3

5. Send a small local smoke test. Once the server reports ready, test the OpenAI-compatible chat endpoint with a short prompt before trying a large context or agent workflow.

curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "moonshotai/Kimi-K3.1-Pro",
    "messages": [
      {"role": "user", "content": "Explain sparse attention in two sentences."}
    ],
    "max_tokens": 256
  }'

A successful response confirms that the server can load the model and answer a basic request. It does not prove that million-token prompts, high concurrency, image input or tool execution will work within your memory and latency budget. Measure those separately.

Hosted alternative: Together AI serverless

If you do not have a multi-GPU cluster, Together AI’s November 5 changelog lists Kimi K3.1 Pro on serverless inference under the model ID moonshotai/Kimi-K3.1-Pro. The catalog lists a 1,048,576-token context, NVFP4 serving quantization, text and image input, function calling and structured outputs. The listed price is $2 per million input tokens, $10 per million output tokens and $0.20 per million cached input tokens at the time of this update.

The provider’s NVFP4 serving label is not the same description as Moonshot’s published MXFP4/MXFP8 checkpoint format. Treat the catalog entry as Together’s serving configuration, not proof that the raw downloadable checkpoint has been converted into the same format.

Call the hosted model with the OpenAI Python SDK

Together documents an OpenAI-compatible endpoint at https://api.together.ai/v1. The following is a basic integration example; it does not require you to download the model weights.

python -m venv .venv
source .venv/bin/activate
pip install openai

export TOGETHER_API_KEY="YOUR_API_KEY"

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["TOGETHER_API_KEY"],
    base_url="https://api.together.ai/v1",
)

response = client.chat.completions.create(
    model="moonshotai/Kimi-K3.1-Pro",
    messages=[
        {
            "role": "user",
            "content": "Review this API design and list its three biggest risks."
        }
    ],
    max_tokens=512,
)

print(response.choices[0].message.content)

For image requests, Together documents the standard chat-completions content format with a text block and an image_url block. Its catalog currently lists K3.1 Pro as image-capable. For tool use, pass the tool schema through tools, inspect the returned tool_calls, validate the arguments in your application, and execute only the operations your own authorization layer permits.

Important multi-turn detail: Moonshot’s model card says to pass the complete assistant message back into the next request, including reasoning_content and tool_calls, rather than rebuilding the message from its visible text alone. Together’s compatibility documentation also warns that reasoning field names vary by model and should be preserved when continuing a conversation. Test this behavior in your actual SDK and agent framework.

License, security and privacy checks

• It is open-weight under a custom license. The license permits use, modification, distribution and commercial activity subject to its terms, but it is not a generic permissive license with no extra conditions. For example, a Model-as-a-Service operator whose aggregate revenue exceeds US$20 million over a consecutive 12-month period must enter a separate agreement before commercial use. Products above 100 million monthly active users or US$20 million in monthly revenue have a model-name display condition. The license also defines exceptions for internal use and access through official products or certified inference partners. Read the full text and obtain legal advice for your business model.
• Review remote code. The documented vLLM command uses --trust-remote-code. Pin the repository revision, inspect custom model code, use a controlled container, and avoid granting the model server unnecessary host permissions.
• Protect the inference endpoint. Keep the server on a private network, put authentication and rate limits in front of it, and do not expose an unauthenticated vLLM port to the public internet. Separate model-serving credentials from application users.
• Check hosted-provider retention settings. Together says prompt and response storage is on by default and may be used for product improvements; model-training use is opt-in and off by default. Zero data retention is not enabled by default. Organization administrators can enable it by turning off prompt storage, but usage metadata and account data are still retained. Review the provider’s privacy and security documentation and ZDR policy before sending sensitive code or customer information.
• Do not trust tool calls automatically. A model can propose a tool call; your application remains responsible for schema validation, authorization, audit logging, confirmation for destructive actions and tenant isolation.

Who should care—and who should wait?

• Inference and platform teams: the most direct beneficiaries. The weights and day-zero serving path let teams evaluate a large model on infrastructure they control, including supported non-NVIDIA accelerators.
• Research teams: can inspect the architecture, test sparse attention, evaluate long-context workloads and experiment with fine-tuning under the custom license.
• Application developers without GPU clusters: can evaluate the model through Together or Moonshot APIs without operating the full checkpoint.
• Desktop local-AI users: should wait for a genuinely practical, verified quantized desktop path. The official deployment material reviewed here points to vLLM and large accelerator configurations; it does not establish a simple LM Studio or Ollama setup for ordinary consumer hardware.

Limitations and what remains unproven

• A one-million-token model context is a model capability, not a promise that every server can hold one million tokens for every concurrent request. KV-cache capacity and runtime settings determine practical limits.
• The vLLM benchmark figures are published by the serving team and depend on specific hardware, batch sizes, context lengths and benchmark harnesses. They are not independent performance guarantees.
• The current vLLM recipe is version- and hardware-specific. Its v0.34.0+ requirement differs from the v0.33.0 stable tag shown on the release page on November 5.
• The public checkpoint’s custom license needs review before commercial redistribution or offering model inference as a service.
• Hosted API prices, model availability, rate limits and privacy settings can change. Recheck the provider’s live catalog and account settings before production use.
• No hands-on deployment or benchmark run is claimed in this article. The setup guidance is based on the official model card, vLLM recipe and provider documentation.

Frequently asked questions

Can I run Kimi K3.1 Pro on a MacBook or a single consumer GPU?

That is not the deployment path documented by the current vLLM recipe. The MXFP4 baseline is TP8 on an eight-GPU configuration such as a B300 node or two GB300 trays. Use a hosted API if you do not have suitable infrastructure.

Does 104B active parameters mean I only need memory for 104B parameters?

No. The model has 2.8T total parameters. The active count describes how many parameters are selected for a token’s computation in the MoE architecture. The deployment still has to make the routed experts available, along with attention, shared experts, vision components and runtime memory.

Is Kimi K3.1 Pro open source?

Moonshot calls it open-weight. The repository uses a custom Kimi K3.1-Pro License with additional conditions. Do not assume it is interchangeable with an OSI-approved open-source license; read the actual license before commercial use.

Can I use the model through an OpenAI-compatible API?

Yes. vLLM exposes an OpenAI-compatible local chat-completions endpoint. Together also documents an OpenAI-compatible hosted endpoint, and Moonshot provides its own compatible API. Model IDs and supported parameters differ by provider.

Does the model support images and tools?

The official model card lists text and image input. The vLLM launch command enables Kimi-specific tool-call parsing, and Together’s serverless catalog lists image input, function calling and structured outputs. Verify each capability with the exact provider and runtime you plan to use.

What is the simplest way to try it without buying GPUs?

Use a hosted API. As of November 5, Together AI lists Kimi K3.1 Pro on serverless at $2 per million input tokens, $10 per million output tokens and $0.20 per million cached input tokens. Check the live catalog before relying on those prices.

Related GyanAangan reading

• Kimi K3.1 Pro Limits Cyber Risk Before Open-Weights Release — the earlier article covers Moonshot’s cyber-risk reduction claims and what developers should verify. This update focuses on the weights, license and serving ecosystem.
• DeepSeek Open-Sources DeepInfer — a related look at the engineering behind large-scale open-model inference.
• Mistral Large 4 Releases Open Weights — compare another frontier-scale open-weight release, its license and deployment requirements.

Primary sources and further reading

1. Moonshot AI: Introducing Kimi K3.1 Pro — original October 24 announcement and release commitment.
2. Moonshot AI official Hugging Face model card — architecture, modalities, checkpoint format, model usage and deployment references.
3. Kimi K3.1-Pro License — the binding license terms and commercial-use conditions.
4. vLLM: Day-0 Support for Kimi K3.1 Pro — serving architecture, hardware coverage, launch flags and team-reported performance.
5. vLLM Kimi K3.1 Pro deployment recipe — current version requirement, hardware presets and multi-node commands.
6. vLLM releases — tagged runtime versions and release notes.
7. Together AI official changelog — November 5 serverless model availability and pricing.
8. Together AI serverless model catalog — current model identifier, context, modalities, features and pricing.
9. Together AI OpenAI compatibility — API base URL, SDK setup and compatibility limitations.
10. Together AI privacy and security and Zero data retention — retention defaults and organization controls.
11. Automater News independent coverage — corroborating release timeline and serving availability, not an independent benchmark reproduction.

Editorial note: This article separates Moonshot’s model-card claims, vLLM’s engineering measurements and Together AI’s provider-specific catalog details. All version, hardware and pricing references are snapshots checked on November 5, 2026. Recheck the linked primary documentation before deploying or budgeting a production service.