Cloud GPU vs LLM API in 2026: Pricing, Cost per Million Tokens & Break-Even Calculator

Last checked: 14 October 2026. GPU availability and prices can change quickly. This guide uses published provider prices available on the date above and separates them from illustrative cost calculations. I have not benchmarked these cloud GPUs for this article; the throughput numbers in the worked examples are assumptions you should replace with measurements from your model and workload.

If you have been running models locally with Ollama, LM Studio or llama.cpp, there is a point where your laptop or desktop is no longer enough. Perhaps you need to serve a larger model, let several people use the same model, run evaluations overnight, or deploy an AI feature for a client. The next decision is often whether to rent a cloud GPU and self-host the model or pay for a managed LLM API.

These options solve different problems. A rented GPU gives you control over the model, runtime and serving stack, but you also take responsibility for capacity planning, security, deployment and uptime. A managed API removes much of that infrastructure work and bills by token or request, but the model, limits, pricing and data-handling terms are controlled by the provider.

This article shows how to compare the costs of RunPod, Lambda and AWS-style GPU infrastructure with token-priced APIs, how to estimate a break-even point, and how to deploy a test endpoint with vLLM without pretending that one GPU price tells you the whole story.

Quick answer: should you rent a GPU or use an LLM API?

1. What does a cloud GPU actually cost in October 2026?

Start with the provider's pricing page, but record the exact product type. A dedicated GPU Pod, a serverless inference worker, a multi-GPU cluster and a reserved enterprise deployment are not interchangeable. Their prices can differ even when the GPU name looks similar.

RunPod: Pods and Serverless are different products

RunPod's published pricing page, updated on 8 October 2026, lists examples including:

Source: RunPod official GPU pricing. The listed rate is not the complete monthly bill. Storage, idle volumes, network requirements, serverless worker settings and the exact cloud tier can change the total. RunPod also has Community Cloud options whose availability and rates can differ from Secure Cloud.

A key advantage of an on-demand Pod is that you can choose a GPU, deploy your own container and terminate the instance when the job is complete. But remember that your model download, container setup, health checks and warm-up take time too. If you repeatedly destroy and recreate a machine, include that setup time in the workload design.

Lambda: GPU instances and clusters

Lambda's published AI cloud pricing lists an NVIDIA H100 SXM instance at $3.99 per GPU-hour for the one-GPU configuration shown, with different rates for multi-GPU configurations and longer cluster commitments. It also lists A100 and other GPU options. Cluster offerings have their own minimum durations and procurement requirements, so do not apply a single-GPU instance rate to a cluster without checking the offer.

Source: Lambda official GPU cloud pricing. The page notes that applicable taxes may be added. Confirm availability, minimum commitment, storage, region and the exact configuration before budgeting.

At the published headline rate, a RunPod Secure Cloud H100 SXM and Lambda's listed one-GPU H100 SXM instance are both $3.99 per GPU-hour on the date checked. That does not prove they deliver identical throughput, reliability, networking or support. It simply means that headline hourly GPU price alone may not settle the decision. Measure the same model, context length, concurrency and request mix on the actual candidate machines.

AWS: compare the full instance, not just the GPU label

AWS offers GPU instances such as the G5 family with NVIDIA A10G GPUs, each with 24 GB of GPU memory. Instance sizes differ in vCPU, system RAM, storage and network bandwidth. G5 is not the same hardware class as an H100, so comparing its hourly price directly with an H100 price without accounting for performance and model fit can mislead.

Sources: AWS EC2 G5 instance specifications and AWS Pricing Calculator. AWS pricing depends on region, instance size, purchase option and other services. Use the calculator for the exact region and configuration you intend to run, and include storage, snapshots, data transfer and any load balancer or monitoring costs.

AWS can be attractive when your application already relies on IAM, VPCs, CloudWatch, private networking, existing contracts or other AWS services. A lower GPU-only price on a specialist provider does not automatically beat the total cost of a cloud architecture you already operate. Conversely, if you only need a single GPU to test an open model, the broader AWS service ecosystem may be more than you need.

2. Why hourly price is not the same as cost per million tokens

Suppose one GPU costs $1.09 per hour and another costs $3.99 per hour. It is tempting to divide each by one million and call that the cost of inference. That is wrong: a GPU produces a workload-dependent number of tokens per second. Throughput changes with the model, quantization, input length, output length, batch size, concurrency, runtime, GPU memory and latency target.

For a simple first-pass estimate, define:

• H = GPU price in dollars per hour.
• T = measured generated output tokens per second at your target configuration.
• U = fraction of paid time spent doing useful inference, from 0 to 1.

An approximate output-token infrastructure cost per one million generated tokens is:

cost_per_1M_output_tokens = H * 1,000,000 / (T * 3,600 * U)

This is an infrastructure estimate, not a benchmark. It assumes that the measured throughput represents your real traffic and that the productive fraction is reasonably stable. It does not include storage, CPU/RAM overhead, engineering time, taxes, monitoring, support, data transfer or the cost of failed requests. It also does not account for input-token processing; measure prefill/input throughput separately.

3. Worked example: how utilisation changes the answer

Consider a hypothetical workload with a GPU price of $1.09 per hour. Assume, purely for illustration, that your chosen model and serving configuration generate 80 output tokens per second while serving your target request mix. This 80-token figure is not a claim about L40S performance; it is a placeholder to demonstrate the arithmetic.

At 80 output tokens per second, the theoretical output volume during one hour of continuous productive generation is:

80 tokens/second * 3,600 seconds = 288,000 output tokens/hour

At 100% productive utilisation, the approximate cost per million generated output tokens is:

$1.09 * 1,000,000 / 288,000 = $3.78 per 1M output tokens

Now suppose only 30% of paid time is productive because traffic is sporadic, requests arrive in bursts, or the instance is left running while idle. The approximate cost becomes:

$3.78 / 0.30 = $12.61 per 1M output tokens

Using the same hypothetical 80-token-per-second throughput at $3.99 per hour, the corresponding estimates are about $13.85 per million output tokens at 100% productive utilisation and $46.18 at 30% utilisation. Again, these figures are illustrative arithmetic, not measured GPU performance or a claim about a specific model.

This is why there is no universal break-even answer. A GPU can look very cheap when it is busy almost all the time and expensive when it sits idle. A smaller GPU serving a smaller model can also outperform a larger GPU on cost for a particular task. Benchmark the model you actually intend to use.

4. Compare with a token-priced API—but keep the models separate

Managed APIs charge for input and output tokens separately for many text models. For example, the official OpenAI API pricing page lists GPT-5.4 at $2.50 per million input tokens and $15 per million output tokens for standard usage, with separate cached-input and other pricing conditions. The same page lists other models at different rates. Check the live pricing table before calculating a production budget.

Source: OpenAI API pricing.

For a request mix of one million uncached input tokens and one million output tokens, using those published GPT-5.4 rates as an example:

Input cost  = 1 * $2.50 = $2.50
Output cost = 1 * $15.00 = $15.00
Total       = $17.50 per 1M input + 1M output tokens

This API total is not directly comparable with the illustrative GPU output-only estimate above. It includes input and output pricing for a particular managed model, while the GPU calculation estimated output generation only and used an assumed throughput. The self-hosted open model may have different quality, tool-use behaviour, context support and latency. A fair comparison must include both input and output tokens, use a representative request mix, and compare models that are acceptable for the same task.

A better comparison worksheet

5. Calculate monthly cost before deploying

For a GPU instance that stays running, the simplest baseline is:

monthly_compute_cost = hourly_rate * billable_hours_per_month

Using a 30-day month (720 hours) for illustration, a $1.09/hour instance costs $784.80 in compute if it runs continuously, while a $3.99/hour instance costs $2,872.80. These calculations exclude storage, data transfer, taxes and any extras. A calendar month with 31 days has 744 hours; a provider's actual billing model and downtime handling can change the final amount.

For a job that uses the GPU for only 20 hours in a month, the corresponding compute arithmetic is $21.80 at $1.09/hour or $79.80 at $3.99/hour—assuming billing really stops when the instance is terminated and there are no continuing storage or service charges. If you keep a persistent volume, snapshot or serverless worker configuration, verify whether it continues to incur charges.

For production, estimate at least three scenarios:

• Low traffic: your normal traffic with substantial idle periods.
• Expected traffic: the average load you plan to support.
• Peak traffic: a burst that must meet your latency and availability target.

Include a safety margin for peak capacity, warm-up, model reloads, failed requests, health checks and deployments. A GPU sized only for average traffic may queue requests or miss latency targets during a burst.

6. Deploy a small vLLM test before committing to a GPU

If you want to self-host a compatible model, vLLM provides an OpenAI-compatible server and an official Docker image. The exact model must fit the GPU's memory and be supported by your selected vLLM release. The following is a minimal example based on the official vLLM Docker documentation; it uses a small model for testing rather than assuming that every GPU can serve a large production model.

Official guide: vLLM Docker deployment.

export HF_TOKEN="your_hugging_face_token"

docker run --rm --runtime nvidia --gpus all \
  -v "$HOME/.cache/huggingface:/root/.cache/huggingface" \
  -e HF_TOKEN="$HF_TOKEN" \
  -p 127.0.0.1:8000:8000 \
  --ipc=host \
  vllm/vllm-openai:latest \
  --model Qwen/Qwen3-0.6B

This command binds the server to localhost, which is safer for a quick test on a machine you control. Cloud environments may require adjustments to the NVIDIA runtime, container image, firewall and model authentication. For a production deployment, pin a tested image version rather than using latest, and follow the model's license and access requirements.

Check that the endpoint responds before sending real traffic:

curl http://127.0.0.1:8000/v1/models

Then run a representative load test with the actual prompt lengths, output lengths and concurrent requests you expect. Record:

• Input and output tokens per second, measured separately.
• Time to first token and total request latency at p50 and p95.
• GPU memory use and whether the model fits without out-of-memory errors.
• Throughput as concurrency rises, including queueing and timeouts.
• Quality on your own task examples, not only a generic benchmark.

Use the measured throughput in the break-even formula. Do not take a model's advertised maximum context length as proof that it will serve that context cheaply or quickly; long prompts consume memory and compute, and can reduce concurrency.

7. Security and reliability costs people leave out

Self-hosting a model does not automatically make an application private or secure. A GPU instance is still a server exposed to account compromise, misconfigured firewall rules, vulnerable containers, stolen API keys and accidental data retention.

• Network exposure: do not leave the model API publicly accessible without authentication and access controls. Prefer private networking or a reverse proxy with TLS and authentication.
• Secrets: keep provider keys and model tokens out of images, source control and logs. Use a secret manager or an appropriate environment-specific mechanism.
• Data retention: decide whether prompts and responses are logged, how long they are kept, and who can access them.
• Updates: pin and patch the operating system, container images, drivers and serving runtime on a planned schedule.
• Backups and recovery: back up configuration and application data, but avoid backing up large model files unnecessarily if they can be downloaded again safely.
• Availability: define what happens when a GPU host fails, a model does not load, or capacity is exhausted. A single GPU instance is a single failure point.

These controls consume engineering time. If your team does not want to maintain them, a managed API may be better value even when its token rate looks higher.

8. Common mistakes when estimating cloud inference costs

Comparing different model capabilities

A low-cost open model and a premium managed model may not produce equivalent answers. Evaluate output quality, tool use, structured outputs, context length and refusal behaviour on the same tasks before treating their prices as substitutes.

Assuming the GPU is busy all the time

Dividing the hourly price by theoretical maximum throughput gives an optimistic estimate. Use measured throughput and realistic productive utilisation. If the application has quiet periods, consider a serverless design or shut down development machines between sessions.

Ignoring input-token processing

Long system prompts, retrieved documents and conversation history can make input processing a significant part of the workload. Measure prefill and decode separately, and include the full prompt/output mix in the calculation.

Forgetting the engineering bill

Someone must deploy the server, monitor it, secure it, upgrade it, diagnose failures and plan capacity. For a small workload, a few hours of engineering time can outweigh months of GPU savings. This is especially important when you are hosting a client's production application.

Choosing by hourly price without checking region

A cheap GPU in a distant region may introduce latency or data residency problems. Confirm current capacity in the intended region and test from where your users actually are. A price shown on a provider's main page does not guarantee that the same configuration is available in every region.

9. A practical decision process for developers and small teams

1. Define the workload. Identify the model class, context length, input/output token mix, concurrency and latency target.
2. Start with an API baseline. Calculate the current or expected monthly cost using the provider's official pricing and your estimated token volumes.
3. Pick two GPU candidates. Compare a lower-cost inference GPU and a higher-memory GPU that can actually fit the model.
4. Benchmark the same task. Use representative prompts and measure output quality, input throughput, output throughput, time to first token and p95 latency.
5. Calculate productive utilisation. Estimate how much of the paid GPU time will be spent processing useful requests.
6. Add the hidden costs. Include persistent storage, network, monitoring, backups, taxes and engineering time.
7. Choose the simplest option that meets the requirements. If a managed API meets your quality, privacy and cost needs, avoid self-hosting just because it sounds cheaper. If a GPU is clearly better at steady volume, run a limited pilot before moving production traffic.

10. How this connects to your existing local AI setup

If you already run models locally, you do not have to replace that workflow overnight. Keep local inference for development and private experiments, then use a cloud GPU for jobs that need more memory or for a controlled production pilot. A consistent API interface can make it easier to switch between local and hosted runtimes, but model behaviour and supported parameters may still differ.

• Gyan Aangan: llama.cpp Server and OpenAI-compatible local APIs — useful background on local model serving and endpoint security.
• Gyan Aangan: running a model on consumer hardware — useful context for deciding when local memory or performance is no longer enough.
• Gyan Aangan: LocalAI model failover and API serving — useful when planning how an application responds to model or backend failures.
• Gyan Aangan: Ollama security and network exposure — a reminder to protect local and remote model endpoints.

Frequently asked questions

Is a cloud GPU cheaper than an LLM API?

Sometimes. It depends on the model, throughput, utilisation, token mix, quality requirements and operational overhead. Low or bursty usage often favours a managed API. Sustained, well-utilised inference on a suitable open model can make GPU rental competitive, but measure it before deciding.

Is RunPod always cheaper than AWS?

No. Specialist GPU platforms may publish lower headline rates for certain GPUs, but AWS pricing varies by region and instance type, and its broader services can matter to an existing architecture. Compare equivalent hardware where possible and calculate the total architecture cost.

Should I rent an H100 for a small AI application?

Not automatically. First determine whether the model fits on a lower-memory GPU and test the throughput you need. An H100 may be justified for larger models or demanding concurrency, but unused capacity is expensive.

Can I use vLLM with any model?

No. Compatibility depends on the model architecture, vLLM version, hardware and required features. Check the official vLLM documentation and the model's own requirements before provisioning a GPU.

What is the safest way to test the economics?

Run a short pilot, record real token throughput and latency with representative requests, calculate productive utilisation, and compare the result with the current API bill. Terminate unused compute and check for continuing storage or other charges.

Conclusion

Cloud GPU pricing is only the first line in an inference budget. The important figure is the cost of delivering acceptable results at your required latency and traffic level. RunPod's published rates make it easy to estimate a pilot; Lambda offers comparable GPU instance options; AWS may fit better when the rest of your stack already lives there. None should be selected solely from a headline price.

My recommendation is to begin with a managed API baseline, rent a GPU for a short and measurable experiment, benchmark the same workload, and use your actual throughput and utilisation in the cost model. If the GPU wins after storage, operations and idle time are included, move gradually. If the API remains simpler and cheaper at your scale, that is a good engineering decision too.

Pricing references checked on 14 October 2026. Published rates may change and can exclude taxes, storage, network usage or other services. The worked throughput example is illustrative only; it is not a measured benchmark of RunPod, Lambda, AWS or any particular GPU.