How to Forecast AI and GPU Infrastructure Spend

Last updated: June 2026

Forecasting AI and GPU spend means projecting cost from the units that drive it, tokens, inferences, and training runs, rather than extrapolating last month's dollar total. AI cost is hard to predict precisely because those units move fast and nonlinearly: a launched feature can double inference traffic overnight, and a single large training run is a lump that ordinary trend lines miss. A unit-driven forecast multiplies projected driver volumes by unit costs derived from history, then adjusts for utilization and commitments, producing a number that holds up as the workload grows and that ties directly to the rest of the FinOps practice.

This article is part of our AI, GPU and ML cluster. For the full picture start with the complete guide to AI and GPU cost optimization, the pillar this piece links up to. Forecasting is the Run-step companion to how to build an AI cost FinOps scope for 2026, which produces the unit metrics this forecast consumes.

Why is AI and GPU spend so hard to forecast?

AI spend is hard to forecast because it scales on volatile units rather than steady utilization. Ordinary compute grows roughly with instance-hours, which trend smoothly; AI cost is driven by token volume, inference traffic, and discrete training runs, which jump when a feature ships or a job runs. Add that accelerator and token prices change as providers release new hardware and models, and a simple month-over-month extrapolation breaks down. The fix is to forecast the drivers separately and apply unit costs, which is what the steps below do. This is also why AI belongs in its own forecast, not buried in a general compute projection.

Step 1: How do I separate the cost drivers?

Split AI spend into three buckets that scale differently: inference, training, and supporting services. Inference, whether on a model API or your own served GPUs, scales with traffic and token volume. Training scales with the number and size of runs, and is lumpy rather than continuous. Supporting services, vector databases, storage, data pipelines, scale with their own usage. Forecasting them together hides the dynamics; forecasting them apart lets you apply the right driver to each. This separation mirrors the cost-source inventory in your AI scope and is the foundation of an accurate projection.

Step 2: How do I establish unit costs from history?

Derive your unit costs, cost per token, cost per inference, and cost per training run, from recent actual spend, because forecast accuracy depends on real rates, not list prices. Divide actual inference spend by the tokens or requests served to get cost per unit; divide actual training spend by the runs completed to get cost per run. These empirical unit costs already bake in your real utilization, discounts, and inefficiencies, which makes them far more reliable forecast inputs than vendor rate cards. Recompute them each cycle, since they shift as you optimize and as prices change.

Step 3: How do I project the drivers forward?

Forecast each driver from product and roadmap inputs rather than from cost history. Project inference token volume and traffic from expected user growth, feature launches, and adoption curves; project training runs from the model roadmap, how many runs are planned, at what size, and when. Pull these from the product and engineering teams who own the roadmap, because they are the only source of the future driver volumes. Multiply each projected driver by its unit cost from Step 2 to get a first-pass spend forecast per bucket. This is where a launched feature's cost becomes visible before it lands, not after.

Step 4: How do utilization and commitments change the forecast?

Adjust the forecast for GPU utilization and any committed capacity, because effective cost is not the same as nominal cost. For GPUs you run yourself, low utilization raises the real cost per unit, so apply your measured utilization rather than assuming full use, the same idle-capacity effect described in what is GPU utilization and why idle accelerators cost so much. For committed or reserved capacity, reflect the lower committed rate on the base load and on-demand rates on the burst above it. This adjustment is what turns a rough estimate into a forecast that matches the bill, and it is where commitment planning and forecasting meet.

Step 5: How do I add scenarios and review against actuals?

Build low, base, and high scenarios and review the forecast against actuals every cycle, because a single point estimate gives false confidence on a volatile line. The scenarios should flex the uncertain drivers, adoption rate, training cadence, and price changes, so finance sees the range, not just a number. Each cycle, compare forecast to actual, explain the variance, and tighten the unit costs and driver assumptions. Verify current accelerator and token prices against provider documentation when you refresh, since they move. This review loop is what makes the forecast a living model and the discipline that keeps it useful.

Frequently asked questions

Why is AI and GPU spend hard to forecast?

AI and GPU spend is hard to forecast because it is driven by units that move fast and nonlinearly, token volume, inference traffic, and discrete training runs, rather than the steady instance-hours of ordinary compute. A single launched feature or large training job can shift the bill sharply, and provider prices for accelerators change, so a forecast must be driven by unit metrics and revised often.

What drives an AI cost forecast?

The drivers are token volume and inference traffic for serving, the number and size of training runs for training, and utilization for any GPUs you run yourself. You forecast each driver from product and roadmap inputs, multiply by the unit cost derived from history, and adjust for utilization and commitments to reach effective spend.

How accurate can an AI spend forecast be?

A unit-driven forecast with low, base, and high scenarios can be reasonably accurate for inference, which scales smoothly with traffic, and less precise for training, which is lumpy. Accuracy improves as you review forecast against actuals each cycle and tighten the unit costs and driver assumptions, so treat it as a living model rather than a one-time estimate.

How often should I update an AI cost forecast?

Review it on a regular cadence, monthly is common, and update it whenever a major feature ships, a large training run is planned, or a provider changes accelerator or token prices. Because AI cost inputs move faster than ordinary compute, a stale forecast loses value quickly, so the review cadence matters as much as the initial model.

The short version

Forecast AI and GPU spend by separating inference, training, and supporting services, deriving unit costs from actuals, projecting each driver from the roadmap, adjusting for utilization and commitments, and reviewing scenarios against actuals on a cadence. The model must be unit-driven and revised often because AI cost scales on fast-moving units. When you want that forecast built and maintained so finance can plan around the AI line, that is exactly what our FinOps implementation service delivers.

Written by Fredrik Filipsson and reviewed by Morten Andersen, applying the See, Cut, Lock, Run method. Independent and vendor neutral.