Trace Local Ollama Apps with OpenTelemetry and Phoenix: A Python Deep-Dive (2026)

Updated: October 27, 2026.

Your local LLM can return a convincing answer while still being painfully slow, using the wrong model, or failing halfway through a streamed response. Without traces, those failures look like a black box. With a small amount of instrumentation, you can inspect the model call, timing, token counts, errors, and the surrounding application workflow in one timeline—without sending prompts to a third-party observability service.

This guide builds a local observability loop for Python applications using Ollama, the OpenInference Ollama instrumentor, OpenTelemetry, and Arize Phoenix. The example is deliberately small enough to understand, but the same pattern can be extended to a local coding assistant, a Django endpoint, or a tool-using agent.

Who this guide is for

This is for Python developers who already run a model locally and want to answer practical questions: Which step is slow? Did the model call fail? How many tokens did Ollama report? Did a streaming request finish? Is the problem in the model call or in the application around it?

It is not a model benchmark, and tracing alone cannot tell you whether an answer is factually correct. It gives you evidence about execution so you can debug and evaluate the system more methodically.

What you will build

A local Ollama model and a Python script that calls it.
A Phoenix UI that receives traces over the OpenTelemetry Protocol (OTLP).
A parent span for the application workflow and an instrumented child span for the Ollama chat call.
Privacy-conscious defaults that hide prompt and response text from recorded spans while retaining useful execution metadata.

The OpenInference Ollama instrumentor documents support for chat calls through ollama.chat, ollama.Client.chat, and ollama.AsyncClient.chat, including streaming and tool-call metadata. It does not currently instrument every Ollama client method: generate and embedding methods are listed as unsupported. Check the current integration documentation before designing around a specific method.

Prerequisites

Python 3.10 or another version supported by the packages you install.
Ollama installed and able to run a model on your machine.
Enough RAM/VRAM for the selected model; model requirements vary, so start with a model that already works on your hardware.
A terminal and basic familiarity with virtual environments.

We will use llama3.2 as the example model name because it appears in the integration's official quickstart. You can substitute a model you have already downloaded and can run. The exact model tag and hardware fit should be checked against the Ollama model library.

Step 1: Confirm Ollama works before adding tracing

Open a terminal and check that the CLI is available:

ollama --version
ollama list

If you have not downloaded the example model, pull it:

ollama pull llama3.2

Then make a simple request:

ollama run llama3.2 "Explain why tracing is useful for debugging software in two sentences."

Expected result: Ollama returns a response. The exact wording and generation time will vary. If the command fails, solve the Ollama/model problem first; an instrumentation layer cannot repair a model that is unavailable or does not fit in memory.

On desktop installations, Ollama may already be running as a background service. On a headless setup, start its server using the method documented for your operating system (commonly ollama serve). By default, the local API listens on http://localhost:11434. Do not expose that endpoint to an untrusted network just to make the demo work.

Step 2: Create an isolated Python environment

Create a project folder and virtual environment:

mkdir ollama-tracing-demo
cd ollama-tracing-demo
python -m venv .venv

Activate it on macOS or Linux:

source .venv/bin/activate

On Windows PowerShell:

.venv\Scripts\Activate.ps1

Install the Ollama client, the Ollama instrumentor, Phoenix, and the OpenTelemetry SDK/exporter:

python -m pip install --upgrade pip
python -m pip install "ollama>=0.4.0" openinference-instrumentation-ollama arize-phoenix opentelemetry-sdk opentelemetry-exporter-otlp

The version floor ollama >= 0.4.0 is specified by the current OpenInference integration documentation. This command intentionally does not pin a specific package version; for a reproducible deployment, resolve and lock the versions that you have validated in your own environment.

Step 3: Start Phoenix locally

In a second terminal, activate the same virtual environment and run:

phoenix serve

Open http://localhost:6006 in your browser. The local Phoenix server acts as the trace collector and UI. The integration's documented local OTLP/HTTP endpoint is:

http://127.0.0.1:6006/v1/traces

For this tutorial, both the application and Phoenix are on the same machine. That keeps the trace path local. If you later switch to a hosted or remote collector, re-evaluate what data is exported, who can access it, and how long it is retained.

Step 4: Add instrumentation and a workflow span

Create a file named app.py with the following code. It instruments the Ollama Python client, creates a parent span for the workflow, and records a few non-sensitive attributes that help group requests. Prompt and response text are hidden from spans as a safer starting point.

from openinference.instrumentation import TraceConfig
from openinference.instrumentation.ollama import OllamaInstrumentor
from opentelemetry import trace
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import SimpleSpanProcessor

PHOENIX_ENDPOINT = "http://127.0.0.1:6006/v1/traces"
MODEL = "llama3.2"

# Configure an OpenTelemetry provider and send spans to local Phoenix.
provider = TracerProvider()
provider.add_span_processor(
    SimpleSpanProcessor(OTLPSpanExporter(endpoint=PHOENIX_ENDPOINT))
)
trace.set_tracer_provider(provider)

# Instrument before calling ollama.chat.
# Hiding input/output text preserves timing and model metadata while
# reducing the chance that prompts or answers are stored in traces.
OllamaInstrumentor().instrument(
    tracer_provider=provider,
    config=TraceConfig(hide_inputs=True, hide_outputs=True),
)

tracer = trace.get_tracer("gyanaangan.ollama-tracing-demo")


def main():
    import ollama

    with tracer.start_as_current_span("ticket-triage-workflow") as span:
        span.set_attribute("app.workflow", "ticket-triage")
        span.set_attribute("app.environment", "local")

        response = ollama.chat(
            model=MODEL,
            messages=[
                {
                    "role": "system",
                    "content": (
                        "Classify a support ticket briefly. "
                        "Do not invent details not present in the ticket."
                    ),
                },
                {
                    "role": "user",
                    "content": (
                        "Ticket: The desktop app closes when I upload a large PDF. "
                        "Return a likely category and one safe next troubleshooting step."
                    ),
                },
            ],
        )

        print(response.message.content)


if __name__ == "__main__":
    main()

Two details matter. First, the instrumentor must be enabled before the Ollama chat call. Avoid importing a function reference with from ollama import chat before instrumentation; the official integration guide warns that a previously captured function reference can bypass the patch. Second, the parent span is ordinary OpenTelemetry instrumentation, while the Ollama instrumentor adds a model-call span. Nested spans make it easier to distinguish application overhead from model execution.

SimpleSpanProcessor is easy to understand in a small local demo because it exports spans as they end. For a busy production service, prefer a batch processor and configure shutdown/flush behavior, sampling, timeouts, and exporter failure handling deliberately. Do not treat this sample as a production deployment template.

Step 5: Run the example and inspect the trace

With Ollama and Phoenix running, execute:

python app.py

The terminal should print the model's generated response. The exact answer is not prescribed and has not been presented here as a tested benchmark. Switch to the Phoenix UI, open the traces view, and look for a trace in the local project. You should be able to inspect the workflow span and the child span emitted for the Ollama chat call.

Useful things to inspect include:

Duration: Compare the parent workflow duration with the model-call span. If the parent is much longer, investigate code outside the LLM request.
Model name: Confirm which model tag was requested. A request can fail before generating an answer, so model metadata on an error span is useful.
Token counts: The instrumentor maps Ollama's reported prompt and completion counts to OpenInference token-count attributes when those counts are present. Treat them as reported usage metadata, not a universal measure of latency or quality.
Error status: Exceptions are recorded on spans. Inspect the error and correlate it with local server logs.
Parent/child relationship: Confirm that the model call appears under the workflow span. If it does not, check instrumentation order and whether the client call used the supported API.

Because this example hides inputs and outputs, you should not expect to read the full prompt or answer in the trace. That is intentional. For a throwaway test with synthetic data, you can selectively enable content capture, but make that a conscious privacy decision rather than a default.

Step 6: Add request grouping without logging personal data

When you move beyond a single script, traces need enough context to answer operational questions. Add stable, low-sensitivity identifiers such as a generated request ID, environment name, workflow name, or non-identifying experiment label. Avoid using an email address, customer name, raw ticket body, access token, or other personal data as a span attribute.

OpenInference supports context attributes such as session IDs, user IDs, metadata, and tags through its documented helpers. Before adding any identifier, decide whether it is necessary, whether it can be pseudonymous, and how long it should remain available. High-cardinality attributes can also make dashboards and storage more expensive.

Step 7: Make the trace useful for a real debugging session

Tracing becomes valuable when you use it to test a hypothesis. Try these controlled experiments with non-sensitive prompts:

Cold versus warm call: Run the same short request twice and compare the durations. Record the conditions and do not assume one result proves a general performance claim.
Model selection: Run the same small test set against two model tags that your machine can support. Compare latency, failures, and output quality separately; faster does not automatically mean better.
Long prompt: Use a synthetic longer prompt and inspect the reported prompt-token count and duration. Keep the payload small enough for your hardware.
Failure path: Temporarily request a model tag that is not installed, then inspect whether the request produces an error span. Restore a valid model name afterward.
Streaming: Adapt the call to use stream=True and consume the iterator completely. The instrumentor documents streaming support and notes that the span finishes when the stream is exhausted, fails, or is abandoned.

Change one variable at a time, save the test inputs, and compare several runs if you need a meaningful measurement. This guide does not claim that these experiments were executed in a live environment.

Architecture: what each component does

Ollama: Runs the model and exposes its local API.
Ollama Python client: Sends chat requests from the application.
OpenInference instrumentor: Hooks supported client calls and translates their execution into AI-oriented span attributes.
OpenTelemetry SDK/exporter: Creates, processes, and exports trace data using an interoperable telemetry protocol.
Phoenix: Receives OTLP traces and provides a UI for inspecting individual runs.

This separation is useful: you can change the tracing backend without redesigning your model prompts, and you can add manual spans around database access, routing, retrieval, or tool execution. OpenInference is designed to complement OpenTelemetry with conventions for AI operations; it does not replace application-level logging, metrics, or quality evaluation.

Trade-offs and limitations

Instrumentation coverage is specific: The current Ollama instrumentor covers chat calls. It does not currently instrument generate or embedding methods, so do not assume every Ollama operation will appear automatically.
Tracing is not evaluation: A successful span only says that the operation executed; it does not prove the answer is correct, safe, or useful.
Telemetry has overhead: Span creation, serialization, and export consume resources. Measure the impact in your own workload and use sampling when appropriate.
Simple export is for learning: A synchronous/simple processor is straightforward for a local script, but production systems need batching, lifecycle management, and resilience planning.
Model behavior varies: Latency and output can change with model tag, quantization, context size, hardware, concurrency, and prompt length. Do not generalize from one run.
UI and package APIs evolve: Pin tested dependencies for repeatable deployments and consult the current official documentation when upgrading.

Troubleshooting

No trace appears in Phoenix

Confirm Phoenix is running and http://localhost:6006 opens.
Check that the exporter endpoint is exactly http://127.0.0.1:6006/v1/traces for the local setup.
Make sure OllamaInstrumentor().instrument(...) executes before the chat call.
Call the supported API as ollama.chat(...) or use a client instance; avoid a pre-imported function reference.
Check terminal errors and package versions. Restart the script after changing instrumentation configuration.

The model request fails

Run ollama list and verify that the exact model tag exists.
Run a direct ollama run test before troubleshooting telemetry.
Check that the Ollama server is listening locally and that the model fits available memory.

Token counts or stream output are missing

Token counts depend on usage fields returned by the Ollama response; they may not be present in every failure path.
For streaming, consume the iterator to completion. An abandoned stream may end early.
Confirm that the method is one the instrumentor supports. Embedding and generate calls are not covered by the documented chat instrumentor.

Custom workflow span is not visible

Ensure the global tracer provider is set before calling trace.get_tracer().
Check that the custom span wraps the instrumented call and that both use the same provider/context.
Restart the Python process after changing the provider setup.

Security and privacy checklist

Keep local endpoints private. Bind Ollama and Phoenix to loopback for a local-only workflow unless you have intentionally secured a remote deployment.
Hide sensitive content. This example uses TraceConfig(hide_inputs=True, hide_outputs=True). OpenInference also documents flags for hiding tool definitions and invocation parameters. Confirm the resulting trace attributes in your installed version.
Minimize metadata. Do not add raw prompts, documents, credentials, customer identifiers, or secrets to custom span attributes.
Review retention and access. A local UI is not a substitute for operating-system access controls, disk encryption, backups, or a retention policy.
Check every export destination. If you point the exporter at a remote Phoenix instance or another OTLP collector, traces leave the local machine. Apply authentication, TLS, network restrictions, and data governance appropriate to that environment.
Do not confuse redaction with complete privacy. Hiding input/output fields reduces exposure, but metadata and error details can still reveal sensitive context if you add it yourself.

For the complete configuration options, see the OpenInference configuration specification. For general instrumentation concepts, consult the official OpenTelemetry Python documentation.

How this fits into a larger local AI stack

Once a single model call is visible, extend the same pattern to the surrounding application: create spans for retrieval, prompt construction, policy checks, and tool execution. For related implementation ideas, see GyanAangan's local LLM gateway with LiteLLM and local AI code-review hook with Ollama and Git. If your application uses retrieval, the separate guide to production RAG evaluation and observability covers retrieval and answer-quality concerns that a basic trace alone cannot solve.

FAQ

Does this send prompts to Arize's cloud?

Not in the local configuration shown here: the exporter points to 127.0.0.1:6006 on the same machine. If you change the endpoint or use a remote deployment, data may leave the machine. Verify your actual network and deployment configuration.

Can I trace Ollama tool calls and streaming?

The current integration documentation describes chat streaming and tool-call metadata support. Consume streams fully, and verify the supported model/client behavior in your environment. The instrumentor does not automatically make tool execution safe; validate and authorize tools independently.

Will this work with every Ollama model?

The instrumentor targets the Ollama Python client's supported chat methods, not a specific model family. Individual models may differ in tool-call behavior, resource requirements, and response quality. Test the exact model tag and use case you intend to deploy.

Can traces tell me which model is best?

They can help compare operational signals such as latency, errors, and reported token counts. To choose a model, also evaluate task quality, consistency, safety, memory use, and the constraints of your workload using a representative test set.

Why are prompts not visible in the example?

The script hides inputs and outputs to make privacy-conscious instrumentation the default. If you need content-level debugging, use synthetic or approved test data and change the masking policy deliberately. Do not disable redaction in production just to make traces easier to read.

Official references

Caveat: This article follows the cited official documentation, but the sample has not been executed as part of this publication run. Validate package compatibility, network settings, and trace masking in your own environment before using the pattern with real user data.