Self-hosting cost map
Self-Hosted AI Model Costs: GPUs, Monthly Bills, and the API Break-Even
Open-weight models keep getting better, but nobody tells you what running one actually costs. This page answers the practical chain: pick a model → see the GPU rig it needs → see the rented monthly bill → compare it against live API token prices from our daily-updated dataset.
Model-by-model cost map
Rented on-demand rates (indicative, verified 2026-07-08); 4-bit quantization; throughput is a conservative batched-serving estimate — see assumptions below.
| Model | Min VRAM | Rig (rented) | Rig $/hr | 24/7 monthly | Self-host $/1M tok* | Nearest API $/1M out |
|---|---|---|---|---|---|---|
| Llama 3.1 8B Instruct Llama Community · Q4 (GPTQ/AWQ) | 6 GB | 1× RTX 4090 24GB | $0.69 | $504 | $0.10 | $0.08 Llama 3.1 8b Instant |
| Qwen3 14B Apache 2.0 · Q4 | 10 GB | 1× RTX 4090 24GB | $0.69 | $504 | $0.15 | — |
| Mistral Small (24B, open weights) Apache 2.0 · Q4 | 14 GB | 1× RTX 4090 24GB | $0.69 | $504 | $0.21 | $0.30 Mistral Small 4 |
| Gemma 3 27B Gemma Terms · Q4 | 17 GB | 1× NVIDIA L40S 48GB | $0.99 | $723 | $0.34 | $0.12 Gemma 3n E4B Instruct (Together) |
| Qwen3 32B Apache 2.0 · Q4 | 20 GB | 1× NVIDIA L40S 48GB | $0.99 | $723 | $0.46 | $0.59 Qwen3 32B |
| Llama 3.3 70B Instruct Llama Community · Q4 | 42 GB | 1× NVIDIA A100 80GB | $1.39 | $1,015 | $0.55 | $0.79 Llama 3.3 70b Versatile |
| GLM-4.5 Air (106B MoE, 12B active) MIT · Q4 | 62 GB | 1× NVIDIA A100 80GB | $1.39 | $1,015 | $0.39 | $3.20 GLM-5 (Together) |
| Llama 4 Scout (109B MoE, 17B active) Llama Community · Q4 (int4) | 65 GB | 1× NVIDIA H100 80GB | $2.89 | $2,110 | $0.89 | $0.30 Llama 4 Scout |
| Qwen3 235B-A22B (MoE) Apache 2.0 · Q4 | 135 GB | 2× NVIDIA H100 80GB | $5.78 | $4,219 | $2.01 | $0.60 Together Qwen3 235B A22B Instruct 2507 |
| Llama 4 Maverick (400B MoE, 17B active) Llama Community · Q4 (int4) | 230 GB | 4× NVIDIA H100 80GB | $11.56 | $8,439 | $2.68 | $0.60 Llama 4 Maverick |
| DeepSeek V3.x / R1 (671B MoE, 37B active) MIT / DeepSeek License · Q4 | 380 GB | 8× NVIDIA H100 80GB | $23.12 | $16,878 | $4.28 | $0.28 DeepSeek V4 Flash |
| GLM-4.x large (355B MoE, 32B active) MIT · Q4 | 205 GB | 4× NVIDIA H100 80GB | $11.56 | $8,439 | $3.21 | — |
*Cost per 1M generated tokens if the rig runs at full batched utilization 24/7 — the best case. At 10% utilization, multiply by 10. That is the whole self-hosting trade in one footnote.
Self-host vs API calculator
Enter your expected monthly volume. We compare an always-on rented rig against the cheapest matching hosted API from our live dataset.
Where to rent the GPUs
On-demand rates move weekly and vary by region — always quote the exact GPU and region before committing.
DigitalOcean GPU Droplets
Per-second billing, 5-min minimum; on-demand from ~$0.76/GPU-hr, reserved cheaper.
Check DigitalOcean pricing →Novita (managed API)
Managed open-model endpoints — the fallback when the API side of the break-even wins.
Check Novita pricing →RunPod
On-demand anchor rates used in our tables (H100 $2.89/hr, A100 $1.39/hr, RTX 4090 $0.69/hr); referral may add a $5 signup credit after first $10 added.
Check RunPod pricing →Vast.ai
Marketplace pricing — often the cheapest hourly rates, variable reliability.
Check Vast.ai pricing →Assumptions (read this before quoting us)
- VRAM figures assume 4-bit quantization (GPTQ/AWQ/int4) with moderate context; long-context serving needs significant extra KV-cache memory.
- Throughput figures are conservative aggregate estimates for batched serving with vLLM-class stacks — single-stream chat is far slower, well-tuned batch pipelines can be faster. We label these as assumptions, not benchmarks.
- GPU hourly rates are public on-demand list prices verified 2026-07-08. Reserved, spot and marketplace capacity is cheaper; enterprise regions can be pricier.
- The API comparison uses the cheapest hosted route for a matching model from our live pricing dataset — not necessarily the same quality, latency or context window.
- Self-hosting adds costs we don't model: engineering time, monitoring, storage, egress, and redundancy. If those matter, add 20–50%.
Frequently asked questions
How much does it cost to self-host an open-source LLM in 2026?
A single rented RTX 4090 (~$0.69/hr on-demand) runs an 8B model 24/7 for about $504/month. A 70B model at 4-bit quantization needs one 80GB card (~$1.39–2.89/hr, $1,015–2,110/month). Frontier MoE models like DeepSeek V3.x need eight H100s — $16,000+/month rented. Whether that beats API pricing depends entirely on your monthly token volume; use the calculator above.
What GPU do I need to run Llama, Qwen, GLM or DeepSeek locally?
At 4-bit quantization, budget roughly 0.6GB of VRAM per billion parameters, plus headroom for the KV cache. 8–14B models fit a 24GB consumer card (RTX 4090/L4). 27–32B fit a 48GB L40S. 70B and MoE models around 100B total parameters need one 80GB card (A100/H100). 235B-class MoE needs two H100s, and 400B+ needs four or more.
Is self-hosting cheaper than using an AI API?
Only at sustained high volume on small or mid-size dense models. A rented GPU costs the same whether you use it or not, so it wins when you keep it busy — typically above tens of millions of output tokens per month. For frontier MoE models (DeepSeek, Llama 4 Maverick), hosted APIs are almost always cheaper because providers batch thousands of users per GPU. The usual reasons to self-host anyway: data privacy, compliance, latency control, and no rate limits.
What is the cheapest way to rent a GPU for LLM inference?
Marketplace providers (Vast.ai) list the lowest hourly rates but with variable reliability. On-demand rates at RunPod, DigitalOcean and Vultr are the practical middle ground — an A100 80GB around $1.39–2.20/hr, an H100 from about $2.89/hr. Reserved or spot capacity cuts 30–60% off if your workload tolerates it.