Unlocking the Power of Cloud Infrastructure for Federated Learning
October 2, 2026
Why Edge AI Inference with Cloud Training Creates Optimal ML Architecture
October 2, 2026
GPU Cloud, Dedicated GPU Cluster or GPU Colocation? A Cost and Fit Guide
Short answer: rent GPUs by the hour when usage is bursty or uncertain, commit to reserved or dedicated capacity when the GPUs stay busy most of the month, and consider colocation when you already own GPU servers and need somewhere with the power and cooling to run them. Utilization decides the cost: at published list prices, a one-year GPU reservation only beats on-demand once the GPUs are busy about 64% of the time.
The four ways to get GPU capacity
| On-demand GPU cloud | Reserved GPU cloud | Dedicated GPU cluster | GPU colocation | |
|---|---|---|---|---|
| How you pay | Per hour, only while running | Discounted hourly rate for a fixed term | Monthly or term contract for specific servers | Rack space, power and connectivity; you buy the servers |
| Commitment | None | 1 to 5 years, depending on the provider | Contract term | Contract term plus your own hardware purchase |
| Who owns the hardware | Provider | Provider | Provider | You |
| GPU sharing | Varies by instance type | Varies by instance type | Not shared when the provider guarantees no oversubscription | Not shared |
| Capacity risk | Instances may be unavailable when you need them | Capacity reserved for the term | Fixed to the servers you contracted | Fixed to the servers you own |
| Best fit | Experiments, bursty training, short projects | Steady training or inference you can forecast | Long-running training or inference that needs predictable performance | Teams that already own GPU servers or want to depreciate them |
Spot or low-priority GPU instances are a fifth, much cheaper option, but they can be interrupted, so they suit only checkpointed, restartable jobs.
The utilization break-even
For any commitment, the break-even utilization is simply the committed hourly cost divided by the on-demand hourly price. Below that share of the month, on-demand is cheaper; above it, the commitment wins.
A worked example with one published list price: an 8 × NVIDIA H100 virtual machine in a US East region of a major public cloud (Linux, September 2026).
| Pricing option | Price per hour (8 GPUs) | Per GPU-hour | 730-hour month at 100% use | Break-even vs on-demand |
|---|---|---|---|---|
| On-demand | $98.32 | $12.29 | $71,774 | — |
| 1-year reservation | ≈ $62.92 | ≈ $7.87 | ≈ $45,932 | ≈ 64% |
| 3-year reservation | ≈ $43.16 | ≈ $5.40 | ≈ $31,507 | ≈ 44% |
| 5-year reservation | ≈ $39.33 | ≈ $4.92 | ≈ $28,711 | ≈ 40% |
| Spot (interruptible) | $18.17 | $2.27 | $13,264 | Not comparable: can be interrupted |
Source: Microsoft Azure retail prices for the ND96isr H100 v5 virtual machine (eight NVIDIA H100 GPUs), East US, Linux, checked September 2026 (Azure virtual machine pricing, ND H100 v5 series). Reservation hourly rates are the published term price divided by the hours in the term. List prices change; check the source before you plan around them.
Use the same formula for a dedicated cluster or colocation quote: divide the monthly cost you are offered by the on-demand hourly price multiplied by 730. If your GPUs will be busy for a larger share of the month than that result, the fixed option is cheaper.
Power: the question colocation turns on
GPU servers draw far more power than general-purpose servers. NVIDIA rates the H100 SXM at up to 700 W per GPU (the H100 NVL at 350–400 W), and lists a maximum of 10.2 kW for a single 8-GPU DGX H100 system (NVIDIA H100, DGX H100 user guide). Before you ship GPU hardware to any facility, ask:
- How many kW can one rack draw, and how is it fed (single or dual power feeds)?
- What cooling does the rack position support at that density?
- How much will the power and space for your server count actually cost?
Questions to ask any GPU provider
- Which GPU, which form factor? SXM and PCIe/NVL versions of the same GPU differ in power, memory bandwidth and interconnect.
- Is the GPU dedicated? Ask whether GPUs are passed through to your workload or time-sliced across tenants.
- How do nodes talk to each other? Multi-node training depends on the interconnect between servers, not just GPU count.
- Where does the training data live, and what does moving it cost? Egress fees apply every time data leaves some clouds.
- What are the commitment terms? Minimum term, exit terms and what happens if capacity is unavailable.
- What is included? Storage, networking, support and monitoring are often priced separately.
Where MarQi Cloud fits
| GPU model | Dedicated GPU clusters: a separate compute pool with direct PCIe passthrough and no GPU oversubscription, not time-sliced or shared GPU instances. Air and liquid cooling options. |
|---|---|
| Pricing | GPU nodes are quoted per build and have no list price. VM unit prices are published on the pricing page. |
| Bring your own GPUs | Colocation for customer-owned hardware in Zone1, with rack space, power and connectivity; per-rack power is quoted per build. |
| Data movement | No egress fees. |
| Location | Zone1, Alpharetta, Georgia (Atlanta metro). Zone2 is under construction. |
If your GPUs will stay busy for most of the month and you want dedicated GPU clusters close to your storage, request a GPU build quote. For background, see our guides to GPU colocation and GPU cloud for LLM training.
Frequently asked questions
When is on-demand GPU cloud cheaper than a reservation?
When the GPUs are busy for less of the month than the reservation’s break-even utilization. At the list prices above, that is about 64% for a one-year term, 44% for three years and 40% for five years.
Is a dedicated GPU cluster the same as a reserved cloud instance?
No. A reservation is a billing commitment on shared cloud infrastructure; a dedicated cluster is specific hardware assigned to you. Ask whether GPUs are passed through or time-sliced.
How much power does a GPU server need in colocation?
It depends on the GPU and server. NVIDIA lists up to 10.2 kW for an 8-GPU DGX H100 system, so confirm per-rack power and cooling with the facility before shipping hardware.
