For about twenty years, renting compute only got cheaper. Then, on a Saturday in early January 2026, AWS quietly bumped its EC2 Capacity Blocks for ML up by roughly 15% — and broke the streak. The 8x H200 box (p5e.48xlarge) in US East (Ohio) went from $34.608/hr to $39.799/hr. In N. California it jumped from $43.26 to $49.749. No blog post, no announcement. People found out when their reservation quotes came back higher.
That’s the news hook, but it points at something bigger. GPU supply is tight, demand isn’t slowing, and for the first time a hyperscaler decided the market would bear a price increase on accelerated compute. If you’re training, fine-tuning, or serving models, this is the moment to actually look at what you’re paying — because the gap between the hyperscalers and the specialized “neocloud” providers has never been wider.
I’ll lay out real per-hour rates for H100, H200, and B200 as of mid-2026, go provider by provider, and give you a decision tree at the end. Prices move weekly, so treat every number here as a snapshot to sanity-check against live rates, not gospel.
What actually changed in 2026
The AWS increase hit Capacity Blocks — the reservation product where you book GPUs for a fixed window, from a single day up to six months out. AWS’s line was that “EC2 Capacity Blocks for ML pricing are dynamic and vary based on supply and demand patterns.” Translation: H200 demand is brutal, so the price went up. The next scheduled repricing is July 2026, and nobody I’ve talked to expects it to go down.
Here’s why it matters beyond the dollar figure. The whole mental model of “cloud compute trends toward zero” assumed slack capacity. With Nvidia accelerators, there is no slack. Lead times on the newest silicon are measured in quarters. When the dominant cloud provider signals it can raise prices without losing customers, the rest of the market notices. So the cost discipline that felt optional in 2024 is now the difference between a viable unit economic and a money pit.
The flip side: a whole class of GPU-only providers has spent the last two years undercutting the hyperscalers by a wide margin, and the AWS hike just made them look even better.
The price ladder, top to bottom
Let’s start with the single most-rented chip, the H100, on-demand, per GPU per hour. These are mid-2026 figures.
- Vast.ai (marketplace): as low as ~$1.87, with H100 SXM commonly around $2.27. Spot-style interruptible listings dip under $0.50 if you can tolerate eviction.
- RunPod: roughly $1.99–$2.69 depending on Secure Cloud vs Community Cloud.
- Lambda: about $3.99.
- CoreWeave: around $6.16.
- AWS / GCP / Azure on-demand: effectively $6–10+ per GPU once you do the math on the full 8-GPU instances, and frequently reservation-only for the newest parts.
So the spread on the exact same chip is roughly 3–4x from cheapest neocloud to hyperscaler on-demand. That’s not a rounding error. On a single 8x H100 node running a month straight, the difference between $2/hr and $6/hr per GPU is about $23,000 a month. Multiply by however many nodes a serious training run needs and you understand why model teams obsess over this.
H200 follows the same shape. GMI Cloud lists it near $2.60/GPU/hr on-demand; CoreWeave is around $6.31 for a single H200; Lambda sits near $5.50. The H200’s extra memory (141GB vs the H100’s 80GB) matters a lot for serving large models without sharding, so it commands a premium over the H100 even on the cheap end.
B200: pay more per hour, pay less per result
Blackwell is where the hourly-rate instinct steers you wrong. B200 on-demand runs higher — roughly $5.50 at Lambda, ~$6/hr at neoclouds, and ~$9.36/GPU/hr on AWS Capacity Blocks. Sticker shock if you stop there.
But the B200 is about 2.5x faster than an H100 for training throughput, and the inference story is more lopsided. SemiAnalysis’s InferenceX benchmarks from April 2026 put B200 at roughly $0.02 per million tokens on GPT-OSS-120B with TensorRT-LLM, against about $0.09 per million tokens on H100 with vLLM — call it 4.5x cheaper per token. Even comparing apples to apples on FP4 inference, B200 came out around 26% cheaper per token despite costing three times as much per hour.
The takeaway isn’t “always buy Blackwell.” It’s that cost-per-hour is the wrong denominator. What you care about is cost-per-training-step or cost-per-million-tokens. A chip that’s three times the price but four times the throughput is a discount, not a splurge. The catch in mid-2026 is availability — B200 is still reservation-heavy and genuinely hard to get on-demand at the cheaper providers. If you can’t actually book it, the great cost-per-token doesn’t help you.
Going provider by provider
RunPod is where I send people who just want a GPU in the next five minutes. Self-serve, clean API, per-second billing, and the cheapest credible on-demand H100s outside the pure marketplaces. Community Cloud (hosts renting out their hardware) is cheaper than Secure Cloud (RunPod’s own datacenters); the trade is consistency and compliance. For prototyping, fine-tuning jobs, and bursty inference, it’s hard to beat. For a 200-GPU synchronized training run, it’s not the tool.
Lambda charges more than RunPod but earns a loyal research crowd. The workflow is friendly, the instances are predictable, and they’ve been in the GPU-cloud game long enough to be a known quantity. The recurring complaint is capacity — popular instance types go “unavailable” during demand spikes, and you wait. Budget for that if Lambda is your only supplier.
CoreWeave sits at the enterprise end and the pricing reflects it. What you’re paying for isn’t the single-GPU rate — it’s InfiniBand fabric, large contiguous clusters, and the kind of networking that makes 100+ GPU distributed training actually scale instead of choking on gradient sync. If your job fits on one node, CoreWeave is overpriced for you. If you’re doing multi-node training where interconnect bandwidth is the bottleneck, the per-GPU number stops being the thing you optimize.
Vast.ai is the marketplace play: hosts list hardware, you bid, prices float in real time. It’s reliably the cheapest way to get an H100, and the spot/interruptible tier is absurdly cheap if your workload can checkpoint and resume. The cost is variance — SLA, reliability, and host quality differ listing to listing, and a great price from a flaky host isn’t a great deal. Filter on verified/high-reliability hosts and the picture improves, at a higher price.
GMI Cloud, DataCrunch, Nebius, and the rest round out a crowded neocloud middle. They’re worth a quote, especially for H200, where GMI’s ~$2.60 on-demand undercuts most of the field. Just don’t assume a tracker’s headline number is what you’ll actually pay after region, commitment, and availability.
On-demand vs reserved vs spot — pick the right billing model
Same hardware, three pricing models, and the right one depends entirely on your usage pattern.
On-demand is for spiky, unpredictable, or short work. You pay the most per hour and owe nothing when idle. If your GPUs sit cold half the day, on-demand almost always wins despite the higher rate.
Reserved or committed (AWS Capacity Blocks, annual neocloud contracts) is for steady, predictable load. You trade flexibility for a lower effective rate — but you’re now paying for idle time, and after the AWS hike the reserved discount on hyperscalers is thinner than it was. Don’t reserve capacity you can’t keep busy; an empty reservation is the most expensive GPU there is. If you’re already weighing commitment levels on AWS, the same logic from AWS Savings Plans vs Reserved Instances vs Spot carries straight over to GPU capacity.
Spot and community tiers are the cheapest by a mile and can vanish mid-job. They’re perfect for fault-tolerant training that checkpoints frequently, batch inference, and anything you can restart without crying. They’re wrong for a live inference endpoint with an SLA.
And watch the line items that don’t show up in the headline rate. Egress fees on the hyperscalers can quietly dwarf the compute cost if you’re shipping large datasets or model weights around. Storage, idle-reservation charges, and inter-AZ transfer all add up. Neoclouds tend to be simpler here, which is part of why the real-world gap is often even bigger than the per-hour comparison suggests.
So when does the hyperscaler actually win?
After all that, you’d think the answer is “always go neocloud.” It isn’t. There are real reasons to pay AWS, GCP, or Azure 3x the rate.
You stay on a hyperscaler when compliance demands it — HIPAA, FedRAMP, SOC 2 scopes that your security team has already signed off on for that provider, and that a marketplace of anonymous hosts can’t satisfy. You stay when your data, VPC, and the rest of your stack already live there and egressing it all to a neocloud would cost more than you’d save on compute. You stay when you need an enterprise SLA with a throat to choke at 3am, not a community host who might be a teenager with a gaming rig. And you stay when you need the newest silicon at scale and your hyperscaler has the only allocation you can actually get.
For everything else — research, fine-tuning, batch jobs, cost-sensitive inference, anyone whose data isn’t already locked into a hyperscaler — the neoclouds win on price and it isn’t close. This pairs naturally with picking the right AI inference platform once you’ve decided where the raw GPUs come from.
A rough migration checklist
If the math says move, here’s the order I’d work through it:
- Measure utilization first. If your current GPUs idle most of the day, switching to on-demand neocloud may save more than any reservation deal, regardless of provider.
- Price the whole job, not the chip. Cost-per-token or cost-per-training-step, including egress and storage — not the per-hour sticker.
- Map your compliance boundary. Anything that has to stay in a certified environment doesn’t move, full stop. Split the workload.
- Pilot on a marketplace, run production on Secure tiers. Use Vast.ai or RunPod Community to validate cheaply, then decide what graduates to a higher-reliability tier.
- Keep a fallback supplier. Capacity evaporates. Don’t single-source your GPUs, especially if you depend on one provider’s availability during demand spikes.
Here’s the honest take: the AWS hike didn’t make AWS a bad choice, it made the default choice more expensive — and defaults are where money leaks. Pull last month’s GPU bill, find your single biggest line item, and get a live quote for that exact workload from RunPod and one hyperscaler. The delta will tell you whether this is worth an afternoon or a quarter.
Rates cited are mid-2026 snapshots and move constantly — check each provider’s live pricing before you commit budget.