Run Hugging Face Models Fully Offline: Download, Verify and Load Qwen Locally (2026)

Updated: October 30, 2026.

A model that works on your laptop can fail the moment a server loses internet access. It may try to resolve a model name, check for a newer revision, fetch tokenizer files, or download a missing weight shard. The fix is not simply to copy one weights file: you need a complete, reproducible model bundle and an application that is explicitly configured to load only local files.

This tutorial shows how to prepare a small instruction-tuned model from Hugging Face, pin its repository revision, download only the files needed by Transformers, verify the bundle, and run inference with Hub access disabled. It is useful for private deployments, disconnected servers, CI environments, and machines where outbound access is restricted.

What you will build: a local Python inference script that loads Qwen2.5-0.5B-Instruct from a directory on disk. The model repository identifies this checkpoint as a 0.5-billion-parameter instruction model under the Apache-2.0 license. Review the official model card and license before adopting it.

Who this guide is for

• Developers packaging Hugging Face models for a server with no internet access.
• Teams that need repeatable model versions across development, staging, and production.
• People debugging unexpected downloads or Hub requests during model startup.
• Engineers preparing an offline workstation or transferring model files to an isolated environment.

This is a Transformers workflow, not an Ollama or LM Studio import guide. If your goal is to expose a common API across different model runtimes, see the GyanAangan guide to building a local LLM gateway with LiteLLM.

Prerequisites

• Python 3.11 or newer is a sensible starting point for the current Transformers documentation.
• A terminal and permission to install Python packages.
• Internet access for the preparation phase only.
• Enough disk space for the model weights, tokenizer, configuration, Python environment, and a temporary copy if you transfer the bundle.
• A CPU or supported accelerator. A 0.5B-parameter model is comparatively small, but actual memory use depends on dtype, runtime overhead, and the operating system.

Install PyTorch appropriate for your machine. NVIDIA GPU users should follow the official PyTorch installation selector rather than guessing a CUDA wheel. The example below also includes a CPU fallback and detects Apple MPS when available.

1. Create an isolated Python environment

Make a project folder and virtual environment so the model tooling does not interfere with other applications.

mkdir hf-offline-demo
cd hf-offline-demo
python -m venv .venv

Activate it on macOS or Linux:

source .venv/bin/activate

On Windows PowerShell:

.venv\Scripts\Activate.ps1

Install the Hub CLI and Transformers with their PyTorch integration:

python -m pip install --upgrade pip
python -m pip install --upgrade huggingface_hub "transformers[torch]"

Check that the CLI is available and inspect the installed versions:

hf --help
hf version
python -c "import transformers, huggingface_hub, torch; print('transformers', transformers.__version__); print('huggingface_hub', huggingface_hub.__version__); print('torch', torch.__version__)"

If the hf command is not found, confirm that the virtual environment is active and reinstall huggingface_hub inside it. The current CLI is documented in the official Hugging Face Hub CLI guide.

2. Pin the model revision before downloading

Using a moving branch such as main makes future downloads capable of resolving to a different repository state. For repeatable deployments, record a full commit hash and use it consistently. This example uses the repository revision c89bee90d9f811437d9735454613c35b4a3c4dc8, visible on the model repository's revision-specific file page when this guide was prepared.

Set the values in your shell so you can reuse them:

MODEL_ID="Qwen/Qwen2.5-0.5B-Instruct"
REVISION="c89bee90d9f811437d9735454613c35b4a3c4dc8"
MODEL_DIR="./models/qwen2.5-0.5b-instruct"

On PowerShell, define equivalent variables with $MODEL_ID, $REVISION, and $MODEL_DIR. The important part is the full revision hash, not the variable syntax.

Before downloading any model, inspect its model card, license, files, and intended use. A model being publicly downloadable does not mean every use case is appropriate or that the model is accurate for your application.

3. Download only the files needed for Transformers

The repository contains more than one artifact format. Downloading every file can waste time and disk space if your application only loads the Transformers checkpoint. The Hub CLI supports include filters; the command below requests configuration and tokenizer files, the SafeTensors checkpoint, and the license/model-card files. Check the file list if you choose a different model, because its tokenizer or weight layout may differ.

hf download Qwen/Qwen2.5-0.5B-Instruct \
  --revision c89bee90d9f811437d9735454613c35b4a3c4dc8 \
  --include "*.json" \
  --include "*.safetensors" \
  --include "*.txt" \
  --include "*.model" \
  --include "LICENSE" \
  --include "README.md" \
  --local-dir ./models/qwen2.5-0.5b-instruct

Let the command finish successfully before disconnecting the machine. The downloaded directory should contain the model configuration, tokenizer files, a SafeTensors weight file, and the license/model card. Do not assume that one .safetensors file is enough for a different model; sharded checkpoints may require several weight files and an index JSON.

The official download guide documents revision pinning, include/exclude filters, cache downloads, and local-directory downloads.

4. Check the downloaded bundle

First, inspect the directory and ensure the key files exist:

find ./models/qwen2.5-0.5b-instruct -maxdepth 2 -type f -print

Look for files such as config.json, tokenizer.json, tokenizer_config.json, vocabulary/merge files, and model.safetensors. Exact filenames vary by model. If the model uses multiple weight shards, keep all required shards and the index file.

For a full, unfiltered repository download, the Hub CLI also provides hf cache verify to compare cached files with Hub checksums. See the CLI documentation. If you intentionally filtered out files, missing optional files are expected; do not interpret that as a complete-repository verification.

For a filtered bundle that you will copy to another machine, create a checksum manifest so you can detect accidental corruption during transfer. On Linux, from the project root:

find ./models/qwen2.5-0.5b-instruct -type f -print0 | sort -z | xargs -0 sha256sum > model-files.sha256

After transferring the folder and manifest to a Linux destination, run the same check from the same project root:

sha256sum -c model-files.sha256

All entries should report OK. This manifest checks that files have not changed since you created it; it does not independently prove that the original download was trustworthy. For stronger supply-chain controls, preserve the model ID, full revision, source URL, license, and approved checksums in your deployment record.

5. Write a local-only inference script

Create run_local.py. It loads the model from the local directory rather than from a Hub model ID. Both tokenizer and model loading use local_files_only=True, and remote model code is not trusted.

from pathlib import Path

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

MODEL_DIR = Path("models/qwen2.5-0.5b-instruct")

if not MODEL_DIR.is_dir():
    raise SystemExit(f"Model directory does not exist: {MODEL_DIR.resolve()}")

if torch.cuda.is_available():
    device = "cuda"
    dtype = torch.float16
elif hasattr(torch.backends, "mps") and torch.backends.mps.is_available():
    device = "mps"
    dtype = torch.float16
else:
    device = "cpu"
    dtype = torch.float32

tokenizer = AutoTokenizer.from_pretrained(
    MODEL_DIR,
    local_files_only=True,
    trust_remote_code=False,
)

model = AutoModelForCausalLM.from_pretrained(
    MODEL_DIR,
    local_files_only=True,
    trust_remote_code=False,
    dtype=dtype,
)
model.to(device)
model.eval()

messages = [
    {
        "role": "user",
        "content": "Explain why an application should validate model output before using it.",
    }
]

inputs = tokenizer.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_dict=True,
    return_tensors="pt",
).to(device)

with torch.inference_mode():
    output_ids = model.generate(
        **inputs,
        max_new_tokens=100,
        do_sample=False,
    )

new_tokens = output_ids[0, inputs["input_ids"].shape[-1]:]
print(tokenizer.decode(new_tokens, skip_special_tokens=True).strip())

The code uses the model's chat template rather than manually inventing role tokens. The Qwen model card documents this pattern in its Transformers example. A small model is useful for validating the workflow, but its answers may be shallow or wrong; do not treat fluent text as proof of correctness.

If your installed Transformers version does not accept the dtype argument, consult that version's model-loading API documentation and use the supported dtype parameter for that release. Record working package versions after testing them in your own environment.

6. Disable Hugging Face Hub requests

Set the offline flag in the process environment before Python imports the Hugging Face libraries. On macOS or Linux:

export HF_HUB_OFFLINE=1
export HF_HUB_DISABLE_TELEMETRY=1
python run_local.py

On Windows PowerShell:

$env:HF_HUB_OFFLINE = "1"
$env:HF_HUB_DISABLE_TELEMETRY = "1"
python .\run_local.py

HF_HUB_OFFLINE=1 tells huggingface_hub not to make HTTP calls to the Hub; missing cached files should produce an error instead of triggering a download. local_files_only=True adds a local-only requirement to the model-loading calls. The telemetry flag disables telemetry for supported Hugging Face libraries, not every package that might be installed.

Run the script once while online with these settings and confirm it succeeds. Then test on a machine or container with outbound networking disabled. The official environment-variable reference describes offline mode and notes that the environment variables are read when the library is imported.

Important distinction: the Hub offline flag is not a firewall. It does not prevent unrelated code from opening sockets. If your requirement is strict network isolation, enforce it at the container, host, or network layer as well.

7. Move the bundle to an air-gapped machine

1. Copy the model directory, the inference script, and a locked dependency list to the destination through your approved transfer process.
2. Verify the checksum manifest after the transfer.
3. Install the same tested Python packages from a trusted local wheelhouse or approved package mirror. Do not expect pip install to work without a package source.
4. Set HF_HUB_OFFLINE=1 and keep local_files_only=True in the code.
5. Run the script with outbound networking blocked and confirm the model starts without attempting to download files.

Capture the tested environment before transfer:

python -m pip freeze > requirements.lock.txt

A frozen package list is useful for reproduction, but it is not itself a complete offline installer. If the destination has no package access, download the matching wheels and all transitive dependencies in advance, then install them from that wheelhouse. Keep platform, Python, and accelerator differences in mind: a wheel set prepared for one OS or architecture may not work on another.

8. Troubleshooting

It still tries to download a file

Confirm that the script loads from the local directory, not the repository name. Check that the environment variable is set in the same shell that starts Python and that it is set before the process imports Hugging Face libraries. A missing tokenizer file or weight shard must be added to the bundle during the preparation phase.

It says a file is missing in offline mode

Reconnect the preparation machine, inspect the model repository's file list, and download the missing required artifact at the same pinned revision. Do not silently switch to main halfway through a deployment; that can create a mixed-version bundle.

Out of memory or extremely slow generation

Reduce max_new_tokens, close other memory-heavy applications, and confirm the model size and dtype fit the device. CPU inference may work but be slow. Do not assume that a 0.5B model will have the same latency on every CPU or that a GPU is automatically being used; print the selected device during diagnostics if needed.

Tokenizer or chat-template errors

Recheck that tokenizer files came from the same repository revision as the weights. Use the official model card's template example and avoid mixing files from different revisions.

Different output after a package update

Pin the model revision and lock the tested Python dependencies. The same model weights can behave differently across framework versions, device backends, precision settings, and generation parameters. Treat package upgrades as a change to validate, not a harmless housekeeping step.

9. Security and privacy checklist

• Pin revisions: use a full commit hash instead of relying on a moving branch.
• Review licenses: confirm the model's license and use restrictions before redistribution or commercial deployment.
• Prefer SafeTensors: use model formats intended for tensor data and avoid loading untrusted pickle-based checkpoints. Read the SafeTensors project documentation.
• Do not enable remote code casually: keep trust_remote_code=False unless you have reviewed and explicitly approved the repository code.
• Keep credentials out of the bundle: never copy access tokens into scripts, logs, Docker images, or the model directory.
• Do not overstate privacy: local inference keeps the prompt path local in this example, but other application code, telemetry, plugins, or network-enabled dependencies may still transmit data. Enforce network controls and audit the whole application.
• Validate outputs: model responses are untrusted data. Validate structure and business rules before sending them to another system or using them to trigger actions.

10. Trade-offs: when this workflow is useful

If you need to train or adapt a model rather than simply run it offline, see the separate GyanAangan QLoRA fine-tuning guide. Training artifacts, adapters, and inference bundles have different packaging requirements.

Frequently asked questions

Does HF_HUB_OFFLINE=1 make the entire computer offline?

No. It disables Hub HTTP calls made through the Hugging Face Hub library. Use host or container network controls if the process must have no outbound connectivity at all.

Can I download the model on one computer and run it on another?

Yes, if you copy every required model/tokenizer file and install compatible runtime dependencies on the destination. Verify the transfer and test the bundle on the target operating system and hardware.

Why pin a commit if the model is public?

Public repositories can change. Pinning a commit makes the intended file set explicit and helps reproduce a deployment later.

Can I use the same directory with Ollama or LM Studio?

Not necessarily. This tutorial downloads a Transformers/SafeTensors checkpoint. Ollama and many LM Studio workflows use different model formats, often GGUF, and may require a separate compatible artifact.

Is local inference automatically safe for confidential data?

No. It reduces dependence on a hosted inference endpoint, but you still need to audit application networking, logs, telemetry, file permissions, and downstream integrations.

Official references

• Hugging Face Hub: Download files and pin revisions
• Hugging Face Hub CLI reference
• Hugging Face Hub environment variables and offline mode
• Qwen2.5-0.5B-Instruct model card
• Pinned model repository revision
• Transformers model-loading API
• SafeTensors repository
• PyTorch installation selector

Final checklist

• Record the model ID, full revision hash, license, and tested package versions.
• Download the complete set of files needed by the selected model architecture.
• Verify the bundle and transfer checksums before moving it to an isolated environment.
• Load from a local path with local_files_only=True and keep HF_HUB_OFFLINE=1 enabled.
• Test startup and inference with network access blocked; do not claim offline readiness until the target environment passes that check.

The core lesson is simple: offline model deployment is a packaging and reproducibility problem, not just an environment-variable trick. Pin the source, carry the complete dependency set, and test with networking actually disabled.