Self-hosting cost map
Self-Hosted AI Model Costs: GPUs, Monthly Bills, and the API Break-Even
Open-weight models keep getting better, but nobody tells you what running one actually costs. This page answers the practical chain: pick a model → see the GPU rig it needs → see the rented monthly bill → compare it against live API token prices from our daily-updated dataset.
Model-by-model cost map
RunPod Secure Cloud public on-demand rates (verified 2026-08-10); 4-bit quantization; throughput is a conservative batched-serving estimate — see assumptions below.
| Model | Min VRAM | Rig (rented) | Rig $/hr | 24/7 monthly | Self-host $/1M tok* | Nearest API $/1M out |
|---|---|---|---|---|---|---|
| Llama 3.1 8B Instruct Llama Community · Q4 (GPTQ/AWQ) | 6 GB | 1× RTX 4090 24GB | $0.69 | $504 | $0.10 | $0.05 Llama 3.1 8B Instruct |
| Qwen3 14B Apache 2.0 · Q4 | 10 GB | 1× RTX 4090 24GB | $0.69 | $504 | $0.15 | — |
| Mistral Small (24B, open weights) Apache 2.0 · Q4 | 14 GB | 1× RTX 4090 24GB | $0.69 | $504 | $0.21 | $0.60 Mistral Small 4 |
| Gemma 3 27B Gemma Terms · Q4 | 17 GB | 1× NVIDIA A40 48GB | $0.44 | $321 | $0.27 | $0.12 Gemma 3n E4B Instruct |
| Qwen3 32B Apache 2.0 · Q4 | 20 GB | 1× RTX A6000 48GB | $0.49 | $358 | $0.34 | — |
| Llama 3.3 70B Instruct Llama Community · Q4 | 42 GB | 1× NVIDIA A100 80GB | $1.39 | $1,015 | $0.55 | $0.40 Llama 3.3 70B Instruct |
| GLM-4.5 Air (106B MoE, 12B active) MIT · Q4 | 62 GB | 1× NVIDIA A100 80GB | $1.39 | $1,015 | $0.39 | $0.14 AutoGLM-Phone-9B-Multilingual |
| Llama 4 Scout (109B MoE, 17B active) Llama Community · Q4 (int4) | 65 GB | 1× NVIDIA H100 80GB | $2.89 | $2,110 | $0.89 | $0.30 Llama 4 Scout |
| Qwen3 235B-A22B (MoE) Apache 2.0 · Q4 | 135 GB | 2× NVIDIA H100 80GB | $5.78 | $4,219 | $2.01 | $0.58 Qwen3 235B A22B Instruct 2507 |
| Llama 4 Maverick (400B MoE, 17B active) Llama Community · Q4 (int4) | 230 GB | 4× NVIDIA H100 80GB | $11.56 | $8,439 | $2.68 | $0.60 Llama 4 Maverick |
| DeepSeek V3.x / R1 (671B MoE, 37B active) MIT / DeepSeek License · Q4 | 380 GB | 4× NVIDIA H200 141GB | $17.56 | $12,819 | $3.05 | $0.03 DeepSeek-OCR 2 |
| GLM-4.x large (355B MoE, 32B active) MIT · Q4 | 205 GB | 4× NVIDIA H100 80GB | $11.56 | $8,439 | $3.21 | — |
| Phi-4 Mini Instruct (3.8B) MIT · Q4 | 4 GB | 1× NVIDIA L4 24GB | $0.39 | $285 | $0.05 | — |
| Phi-4 (14B) MIT · Q4 | 10 GB | 1× RTX 4090 24GB | $0.69 | $504 | $0.16 | — |
| Mixtral 8x7B Instruct (46.7B MoE, 12.9B active) Apache 2.0 · Q4 | 30 GB | 1× NVIDIA L40S 48GB | $0.99 | $723 | $0.55 | — |
*Cost per 1M generated tokens if the rig runs at full batched utilization 24/7 — the best case. At 10% utilization, multiply by 10. That is the whole self-hosting trade in one footnote.
Self-host vs API calculator
Enter your expected monthly volume. We compare an always-on rented rig against the cheapest matching hosted API from our live dataset.
Where to rent the GPUs
These are source-backed provider references, not a ranking: the GPU classes differ, marketplace rows move in real time, and the managed API has no fixed monthly GPU bill. Always quote the exact GPU and region before committing.
| Provider | Reference configuration | $/hour | 24/7 month | Basis |
|---|---|---|---|---|
| DigitalOcean GPU Droplets | 1× NVIDIA L40S 48GB | $1.570 | $1,146 | on-demand Per-second billing with a 5-minute minimum. Region and availability can change. |
| Vultr Cloud GPU | 1× NVIDIA L40S 48GB | $1.671 | $1,123 | hourly plan Official public API plan. Published locations were empty at check time, so confirm deployability before relying on it. |
| RunPod | 1× RTX 4090 24GB | $0.690 | $504 | Secure Cloud on-demand Calculator anchor for the small-model examples. |
| Lambda | 1× RTX A6000 48GB | $1.090 | $796 | on-demand Published on-demand rate; availability varies. |
| Vast.ai | 1× RTX 4090 24GB marketplace offer | $0.402 | $294 | marketplace snapshot Timestamped marketplace reference, not a fixed list price; reliability and availability vary. |
| Novita (managed API) | DeepSeek V3.1 managed API | Usage-based | No fixed bill | pay per token Managed-API fallback, included as the non-rental alternative rather than a GPU price. $0.27/1M input · $1.00/1M output |
Checked 2026-08-10. Monthly estimates use 730 hours unless the provider publishes a monthly-cost field; taxes, storage, egress, and engineering are excluded.
DigitalOcean GPU Droplets
Per-second billing with a 5-minute minimum; our reference row uses one 48GB L40S.
Check DigitalOcean pricing →Vultr Cloud GPU
Official public API prices include fractional A16/A40 plans and full L40S instances; our reference row uses one 48GB L40S.
Check Vultr pricing →Novita (managed API)
Managed open-model endpoints — the fallback when the API side of the break-even wins.
Check Novita pricing →RunPod
On-demand anchor rates used in our tables (H100 $2.89/hr, A100 $1.39/hr, RTX 4090 $0.69/hr); referral may add a $5 signup credit after first $10 added.
Check RunPod pricing →Vast.ai
Marketplace pricing — often the cheapest hourly rates, with inventory and host reliability varying by offer.
Check Vast.ai pricing →Assumptions (read this before quoting us)
- VRAM figures assume 4-bit quantization (GPTQ/AWQ/int4) with moderate context; long-context serving needs significant extra KV-cache memory.
- Throughput figures are conservative aggregate estimates for batched serving with vLLM-class stacks — single-stream chat is far slower, well-tuned batch pipelines can be faster. We label these as assumptions, not benchmarks.
- GPU hourly rates are RunPod Secure Cloud public on-demand list prices verified 2026-08-10. Reserved, spot and marketplace capacity is cheaper; enterprise regions can be pricier.
- The API comparison uses the cheapest hosted route for a matching model from our live pricing dataset — not necessarily the same quality, latency or context window.
- Self-hosting adds costs we don't model: engineering time, monitoring, storage, egress, and redundancy. If those matter, add 20–50%.
Frequently asked questions
How much does it cost to self-host an open-source LLM in 2026?
A single rented RTX 4090 (~$0.69/hr on-demand) runs an 8B model 24/7 for about $504/month. A 70B model at 4-bit quantization needs one 80GB card (~$1.39–2.89/hr, $1,015–2,110/month). Frontier MoE models like DeepSeek V3.x need a multi-GPU rig; our moderate-context preset uses four H200s at about $12,800/month rented. Whether that beats API pricing depends entirely on your monthly token volume; use the calculator above.
What GPU do I need to run Llama, Qwen, GLM or DeepSeek locally?
At 4-bit quantization, budget roughly 0.6GB of VRAM per billion parameters, plus headroom for the KV cache. 8–14B models fit a 24GB card (RTX 4090/L4). 27–32B fit a 48GB A40, RTX A6000, or L40S. 70B and MoE models around 100B total parameters need one 80GB card (A100/H100). 235B-class MoE needs two H100s, and 400B+ needs four or more.
Is self-hosting cheaper than using an AI API?
Only at sustained high volume on small or mid-size dense models. A rented GPU costs the same whether you use it or not, so it wins when you keep it busy — typically above tens of millions of output tokens per month. Frontier MoE rigs such as four H200s for DeepSeek V3.x are usually more expensive than hosted APIs because providers batch thousands of users per GPU. The usual reasons to self-host anyway: data privacy, compliance, latency control, and no rate limits.
What is the cheapest way to rent a GPU for LLM inference?
Marketplace providers (Vast.ai) list the lowest hourly rates but with variable reliability. On-demand rates at RunPod, DigitalOcean and Vultr are the practical middle ground — an A100 80GB around $1.39–2.20/hr, an H100 from about $2.89/hr. Reserved or spot capacity cuts 30–60% off if your workload tolerates it.