The Capacity
Planning Practice

6. Approval is not allocation

In short: approving an ask in a common unit doesn't deliver machines of the right shape, in the right place, on time, and the gap needs an owner.

A declaration, however well scored, is only a claim on capacity. Say the planning forum (the standing group that approves and ranks asks) approves it, and months later the team still has no usable machines.

Between a declared forecast and productive work, demand passes through seven steps (Figure 15): declared, approved, allocated to a shape and a place, delivered, live, adopted and productive. Each is a separate act with its own owner, and each can leak; unlike in a sales funnel, every loss is a failure, except designed over-issue (Section 6.1). Hixson and Guliani's lead-time table runs a similar line, from generating planning numbers through approving and provisioning resources to "Ready to serve" [6]. The funnel adds adoption and productive use after capacity is live. These steps are not Section 5.4's stages: a stage says how firm a declaration is, a step how far approved demand has traveled. The slow steps must start before the declaration has firmed up, which Sections 6.5–6.7 take up.

Schematic seven-step funnel: Declared, Approved, Allocated, Delivered, Live, Adopted, Productive. Lettered gaps (a)–(f) name each leak and its owner, from forecast error (declaring team) to occupancy without goodput (workload owner). Notes mark firmness classes at approval, the limits of markets over steps 2–3, and supply contracts creating liabilities over steps 3–4. Netting sits between approved and allocated. Delivered is itself a chain from order to turn-up, where each link's need date is the next link's start. Approvals stay uncommitted until supply answers.
Figure 15. A common unit prices demand so it can be approved. The steps after that are where ownership most easily goes unassigned. Hixson and Guliani's lead-time table runs a similar line [6]; the funnel adds adoption and productive use after capacity is live. Open full size: Figure 15

"Delivered" hides a chain of its own (order, build, ship, receive, test, turn up), and each link can leak: a vendor slip, a failed test, a queue at turn-up.

Netting: the step between approved and allocated. Every cycle, supply planners take approved demand, subtract what's on hand, on the way, on order or due back from reclaim, and add back what's scheduled to retire. The rest, the net need, is ordered part by part, each on its own lead time: the core of manufacturing's material requirements planning. Here's the catch: every number in that subtraction is a forecast too. A reclaim that doesn't happen, or a decommission that slips, becomes a shortage that looks like a supply failure. In my experience, many undesigned gaps are born there, in the netting just after approval.

The rest of the section sets out the problem (Sections 6.1 and 6.2), then the tools: buffers, bridges and timing (Sections 6.3 to 6.5). Then come the ways a step can miss on either side (Sections 6.6 and 6.7).

6.1 A common unit prices; it doesn't clear

Planning functions normalize asks into a common unit (a normalized machine, a dollar, a megawatt) so executives can compare and approve unlike demands. RAS's relative resource units are a published example [54]. Section 7 invents one, the normalized machine unit (NMU). A standard machine of 50 cores and 200 GB is 1 NMU, and a high-memory machine of 50 cores and 600 GB is 1.4. Its forum approves an ask as 4 NMU; the cheapest machines that fit it come to 4.8. That gap starts here, because an approved 1,000 units must still become:

  • a CPU-to-memory ratio;
  • storage capacity and access rate (IOPS), which are separate resources;
  • a placement within latency and redundancy limits (Auxon's example: within "30 milliseconds of network latency" of the caller [1]);
  • and hardware that exists in time.

The unit's weights decide which asks look cheap. Section 7 weights by component cost: a core is 0.016 NMU, a GB 0.001. But under scarcity, the binding dimension can be worth more than its cost share when the machines on hand can't match the ask's shape. Say memory binds and only standard machines are on hand: every extra 200 GB brings 50 cores along, needed or not. Those cores are stranded: left unusable because another dimension ran out first. Counting them, each extra GB costs 1 NMU ÷ 200 = 0.005 NMU, five times its weight; that's memory's shadow price here. Swap in a high-memory machine and 400 more GB costs 1.4 − 1.0 = 0.4 NMU, or 0.001 per GB. Memory still binds, but nothing is stranded, so its shadow price falls back to the weight.

So a cost-weighted unit under-prices asks heavy on what's scarce, and approving in it predictably creates stranding for fulfillment to absorb. Re-weighting toward the binding dimension each cycle is a partial fix; a shadow price is the full one. Stranding occurs per machine, rack (power against slots), hall, failure domain and network port, mostly fixed when hardware is physically placed, months before any job runs. Microsoft's Pond paper measures the mirror case: "up to 25% of DRAM becomes stranded as more cores become allocated to VMs" [57].

Fulfillment is packing boxes into bins with both a weight and a volume limit, where some boxes fit only certain bins. Published optimizers treat it as just that: multi-resource (vector) bin packing under placement constraints. Auxon represents requirements as "a giant mixed-integer or linear program" [1]. RAS re-assigns servers to reservations with a mixed-integer solver "every few tens of minutes" [54]. Flux keeps each region within "its available supply" [2]. Their solvers' outputs are one way to estimate Section 4's shadow prices.

Much efficiency comes from re-shaping an ask rather than cutting it, such as a faster storage tier in place of a design with many more disks, when access speed binds. Before the order is placed, that's nearly free; after, it can mean change fees, rework, a new test cycle and a new lead time (Section 7). The SRE chapter's Meet case shows re-shaping at work: doubling CPU and memory per instance halved the instance count [7]. In my experience, re-shaping is a conversation more than a calculation. An ask often carries a design assumption the team didn't know it made, such as putting everything on spinning disk. Question the total, and you get a defensive answer. Question the assumption ("which part of this really needs fast access?"), and you often get a different mix, such as fewer spindles and a slice of flash. The team ends up with a better design, not a haircut, and that's why it agrees.

Not every gap is a failure. Borg limits production-priority quota "to the actual resources available in the cell" and deliberately over-sells lower priorities: "every user has infinite quota at priority zero" [17]. Quota Marketplace credits "are fully decoupled from physical resources" [18]. Google's pooled quota, as the SRE chapter describes it, "relies on oversubscription" [7]. A stockout "occurs when oversubscription fails," so pooled services need "stockout resiliency" [7]. Three cases follow:

  • Capped issue limits approval to sourceable supply.
  • Designed over-issue sells entitlements beyond supply under a stated firmness class (firm, best-effort, preemptible), so holders know their odds.
  • The undesigned gap is an approval treated as firm that supply can't meet in the shape, place or time asked.

An approval becomes capped issue when supply answers it with a dated commit or a named gap (Section 10.1). Until then, label it uncommitted, a firmness class of its own, so its holder knows the odds.

The gap itself isn't new. Hixson and Guliani list approving and provisioning resources as separate steps [6], and Auxon's allocation plan reports requirements it could not satisfy [1]. The 2026 SRE chapter states the goal plainly: capacity has to reach "the right consumer, in the right location, at the right time" [7]. That's "not a standard multidimensional bin-packing problem," because you can reshape both the bin and what goes in it [7]. Its figure of stranded and overloaded cells by resource family and region is this section's problem in one picture [7]. Cloud providers say it in their documentation and guidance, too: "Quota and capacity are separate checks" [58], and "Capacity does not equal quota" [59]. What this paper adds is an owner and a ledger for the gap between approval and allocation (mechanism 5).

6.2 Supply is ordered long before it arrives, and bought through contracts

Picture trucks already on the highway: what arrives next quarter was ordered long ago, how long ago depending on the layer. Hixson and Guliani give the order time as an example: it "could be five years if you erect your own buildings," against "10 minutes if you run a small service in a cloud and have good automation and a credit card on file" [6]. The table gives public values where they exist; Appendix B has the sources.

Layer Lead time What the public record says
Land, shell, utility power, large electrical gear Years One operator: capacity "planned years in advance due to long lead times for land, utility infrastructure, and construction of physical buildings with cooling infrastructure" [8]. Grid connections and large transformers each take years [60–62] (Appendix B).
Long-haul network Months to years Meta's backbone planners describe amplifier-site space and power, and optical and IP gear, as "taking months or even years to deliver." They plan in two horizons: "the next 6 months to 2 years" and "2 to 5 years in the future" [63].
Machines and parts Quarters One operator: "over 6 months for some components" [8]. One accelerator vendor: "extended lead times of more than 12 months" in its own supply [64]. No published purchase-order-to-dock time found.
Turn-up Weeks per job; months to staff No published turn-up duration found.

Power needs care. A large new load waits in the utility's own queue, not the one for new power plants, and little is published on how long that takes [65] (Appendix B). Where nothing is published, the paper uses a plainly hypothetical value. For example, if standard servers take one quarter and a new accelerator platform takes three, the two need different commit points even inside one launch. Figure 16 shows the layers and their lead times.

Illustration of three trucks on a highway heading to a building marked launch, each carrying one supply layer. The door is their need date, the start of the next layer, not launch day; at the door, turn-up is a crew working at a rate, not a truck. Below, a ladder of lead times: land, shell and power in years; long-haul network in months to years; machines and parts in quarters; turn-up in weeks per job; placement from minutes to days. Turn-up and order-to-dock times have no published values.
Figure 16. Supply is already on the highway. The slower the layer, the longer ago it had to leave, so each layer has its own commit point even inside one launch. Open full size: Figure 16

Placement is two decisions on different clocks. Physical placement picks the site, hall, power domain and network fabric. It's fixed with the order or at turn-up, costly to undo, and decides which demand a machine can ever serve. Job placement puts work on existing machines in minutes to days; Flux re-cuts regional placement "in weeks-long cycles" [2].

Each lead time is a chain, a spread and partly a queue. The slowest link sets the pace (in my experience, not always the same link). In my experience, the spread is lopsided: late arrivals are common, early ones rare. For example, if machines usually take one quarter but one order in five takes two, a commit point one quarter out is a gamble in a bad year. So this paper plans on a cautious lead time, near the slow end of what's been seen (Section 8). And since vendor allocation, test positions and turn-up crews are limited, lead times stretch when everyone orders at once.

Layers stack. The hall must be ready before the machines, the machines before turn-up, turn-up before the load tests that precede launch. So a layer's need date is the next layer's start, not launch day. Get the order wrong and the fast layer waits on the slow one. Microsoft's chief executive described exactly that: "It's not a supply issue of chips; it's the fact that I don't have warm shells to plug into" [66].

Turn-up is a rate, not an order. It's set by people, dock space, test positions and network ports, staffed months ahead, so it works like a checkout line. For example, if a quarter's deliveries fill four halls, turn-up takes two weeks per hall and a site works two halls at once, they take a month to go live. A slipped batch of four more halls landing alongside waits a month longer. So a supply slip often shows up twice. In my experience, per-site turn-up throughput, with retests and refresh work competing for the same crews, often binds, and delivered-but-not-live is where fulfillment backlogs quietly age. Figure 17 draws the three adjustments.

Three panels. (a) Five orders placed at −5: four arrive in one quarter, one in two, so plan on two and commit at −8 for a need date of −2; in my experience, late arrivals are common and early ones rare. (b) Hall, machines (two quarters, −8 to −2), turn-up and load tests run as a relay; each layer is needed when the next starts, not at launch. (c) Four halls at two weeks each, two at a time, take a month; a slipped batch of four more waits a month longer. Hypothetical values.
Figure 17. Three adjustments that move a commit point earlier than "launch minus lead time": plan on the slow case, work back from the next layer's start, and remember that turn-up is a queue. Hypothetical values. Open full size: Figure 17

Padding travels upstream. Padded asks, a rational response to expected rationing, get amplified into supplier orders: the "rationing game," one of the four causes of the bullwhip effect Lee and colleagues list [67]. Illustratively, if three layers each add 10% to be safe, the order at the top is 1.1³ ≈ 1.33 times real demand. When supply loosens and the padding comes off, orders drop as sharply.

Supply is bought through contracts as much as orders: prepayments and deposits, take-or-pay and capacity commitments, and non-cancelable orders. There are also power agreements and tariffs with minimum bills that run for years (Appendix B has examples). A signed contract turns its forecast into a liability, owed whether demand arrives or not once its cancel window (the last date you can still walk away cheaply) closes. So each contract needs signed demand behind it and a register entry with the liability, the cancel window and an owner. The risk operators name is a mismatch: long commitments behind shorter customer contracts [68, 69].

A cancel window is rarely one date. What you owe usually rises in steps toward delivery, and some orders are non-cancellable from the start. For example, if cancelling a year out costs a tenth of the price, a quarter out might cost half, and inside the vendor's build time all of it. These are hypothetical numbers; I found no published terms of this kind. Many agreements also let you push a delivery out, or pull it in, for less than cancelling. Since the most common timing miss is a slipped launch (Section 6.6), the right to push out is often worth more than the right to walk away. Figure 18 draws the steps.

Step chart of the share of an order's price owed on cancelling, by time before delivery: a tenth a year out, half a quarter out, all of it inside the vendor's build time. A dashed line at 100% marks orders that are non-cancellable from the start. A two-way arrow says a delivery can usually be pushed out or pulled in for less than cancelling. A marker at the edge of the cheap step defines the cancel window as the last date you can still walk away cheaply. A side card says that once the window closes the forecast is a liability, recorded with its cancel window and owner in mechanism 11. Hypothetical.
Figure 18. What you owe on cancelling rises in steps toward delivery, and some orders are non-cancellable from the start. Since slips are the most common timing miss, the right to push a delivery out is often the more valuable one. Hypothetical numbers; no published terms of this kind were found. Open full size: Figure 18

Quantity flexibility bounds how far an order may move. Tsay describes one such contract. It "couples the customer's commitment to purchase no less than a certain percentage below the forecast with the supplier's guarantee to deliver up to a certain percentage above" [70]. In its rolling form, each period's quantity may be revised only by a set percentage per planning round, so more change is tolerated far from delivery and less near it [71]. That's supply's own version of Section 5.4's stages, and Section 8 borrows it.

Suppliers ration, and they score you. In my experience, constrained parts are allocated as much as sold. Microsoft, short of capacity for many quarters, said: "We say no to some of the demand that may be something that we could serve, but it's not in our long-term interest" [72]. So a buyer of constrained memory, flash, accelerators or optics faces a second clearing problem (Section 6.4), run by a supplier deciding whose forecast to believe. It has the same reason to discount a buyer whose orders have run high as the planner has to discount a team's intents; that's my reading of the rationing game [67]. So send a vendor the base, hold the high as options, and keep internal padding from reaching it.

The rule follows: commit early only to what is slow, defer what is fast, and make early commitments fungible (able to serve whichever launch turns out to need them). Operators already do. Microsoft calls its data center leases "very long-lived assets," at "15 to 20 years," while for short-lived chips "the lifetimes of these and the lifetimes of the contracts are very similar" [72].

Every step down the chain spends some fungibility. Powered land serves almost any launch; a shell with a cooling design, fewer; a tested rack in one hall, only demand of its shape that can run there. Supply chains answer with postponement: keep supply generic until the decision that says which shape, as Feitzinger and Lee describe at Hewlett-Packard [73]. In capacity, commit to buildings, power and common parts on the slow clock, and to configured machines on the fast one. A shell stays fungible only if its specification (power per rack, cooling type) is written against the high scenario. In my judgment, a shell built for legacy air-cooled density may not host current accelerator racks. Figure 19 shows the chain.

A chain from powered land, which can serve almost any launch, to a shell with a cooling design, which serves fewer, to a tested rack in one hall, which serves only demand of its shape there. Buildings, power and common parts are committed on the slow clock and configured machines on the fast one. An inset defines an option as a deposit that holds the right to buy later.
Figure 19. Postponement [73] in capacity: keep supply generic until the decision that says which shape, and hold the slow, generic layers open with options. Open full size: Figure 19

Options keep the slow layers open. An option is a small payment now for the right, but not the duty, to buy later, like a deposit that holds a restaurant table. For most hardware it's a cancelable or reschedulable order with a fee schedule. In practice:

  • take options on the long-lead layer (land options, utility capacity reservations with release terms, deposits on long-lead electrical gear);
  • stage sites ahead of need: Meta describes "staging data center sites so we can spring up capacity quickly in future years as we need it," and partnerships "that give us option value for future compute needs" [74]. For one campus, it leases on "a four-year initial term with options to extend," backed by a residual value guarantee for the first 16 years [75]. That's a short commitment plus options on a long-lived asset;
  • buy the medium-lead layer against a base scenario;
  • write down the trigger for converting options into orders.

The option value of waiting argues for buying late [76]; lead time and shortage cost argue for buying early. A written table of substitutions per workload class keeps commitments fungible, and needn't be complete. Jordan and Graves found that "limited flexibility (i.e., each plant builds only a few products), configured in the right way, yields most of the benefits of total flexibility," with the most benefit when it chains "products and plants together" [77]. Here's the read-across to capacity (an analogy, not a result tested on fleets). If each workload can run on two machine shapes and the pairs link into a chain, you get most of the benefit of making everything run everywhere.

6.3 Buffers are a priced decision

In short: name each buffer and size it on which mistake costs more, running short or holding spare. Long lead times push buffers up; pooling pulls them down.

Consider a plan with one line called headroom, a percentage nobody can explain. Buffers come in at least six kinds, each with its own owner and sizing. They are redundancy (N+k), maintenance reserve, queueing headroom, forecast-error buffer, launch reserve (capacity held against a declared high scenario) and inventory buffer (parts and ready machines in stock). RAS publishes one operator's split: about 94% of servers as guaranteed capacity, 2% for random failures and 4% for correlated failures [54]. Hixson and Guliani's "blow-up" factors itemize storage the same way: 1.2 for RAID5, 1.3 for overhead and 2 for a second site come to 3.12 bytes per byte stored [6].

Not every buffer is installed. One operator describes its stock. It says "we would stock inventory of component parts based on forecasts so that we could quickly build and deliver machines," and "Component inventory is planned to high quantile forecasts" [8]. A stocked part can become many machine shapes, so its buffer pools across all of them (postponement, Section 6.2) and can be smaller than the installed buffers it protects. Someone funds and replenishes it, so a draw on it is a debt with a date (Section 7).

How much to hold. A newspaper seller stocks copies before knowing how many will sell; the right number depends on which mistake costs more. So compare the cost of running short with the cost of holding spare: this paper calls that comparison the shortage-to-idle ratio (textbooks call the whole problem the newsvendor problem). It sets how far up the range of likely demand you buy:

C_u/(C_u + C_o)

In words: the cost of being short, over the sum of the costs of being short and being over [78]. Picture ten equally likely outcomes for next year's demand, lowest to highest. If being short costs four times what a spare unit costs (a 4 : 1 ratio), the fraction is 4/(4 + 1) = 0.8. So cover the eighth-lowest, and accept running short in the two above it. Hixson and Guliani frame the same choice without the formula. Within the planning horizon, "you must understand the risk of spending too much money, or of running out of capacity," and "That tolerance should define how aggressively you provision" [6]. The SRE chapter says capacity planning "heavily parallels the concept of error budgets", and "provisioning for every statistical edge case is the wrong target for capacity" [7]. Its ancestor is the first SRE book's "100% is the wrong reliability target for basically everything" [79]. Figure 20 pictures it.

A newspaper seller stocks copies before knowing demand. Below, ten equally likely demand outcomes from lowest to highest; with running short costing four times holding a spare, the fraction is 0.8, so the plan covers the eighth-lowest and accepts running short in the two above. A note credits the textbook newsvendor formula [78], which this paper calls the shortage-to-idle ratio.
Figure 20. Buy up the range until the next unit is as likely to sit idle as to be needed, weighted by which mistake costs more. At 4 : 1, cover the eighth of ten outcomes. The rule is the textbook newsvendor formula [78]. Open full size: Figure 20

For durable capacity under growing demand, the cost of being over is, in my reading of the capacity-expansion literature [80], mainly carrying a unit until demand grows into it. That means capital tied up, depreciation while it waits, power if it's energized, and the newer generation you didn't wait for. For machines the last two are material; for shells and power, small.

The cost of being short is harder to see, and it's usually argued, not measured. The SRE chapter points to the evidence: "Always measure when demand shaping occurs," because the record shows the value of the shed work [7]. It asks teams to choose intentionally between investing more in supply and accepting the cost of shedding [7]. So, one way to think about it, the shed and degradation log is product's half of the ratio, as carrying cost is supply's. For example, say a quarter's shed events cost about $200,000 in lost sales. The spare that would have prevented them would have cost about $50,000, about the size of the shortfall. Then the log points to roughly 4 : 1 for that layer (illustrative). It's a starting point for the signature, not a formula.

Lead time inflates buffers. Safety stock, the buffer against forecast error, must cover whatever demand does while the trucks are on the highway. If demand is independent from period to period, safety stock grows with the square root of lead time [78]. Illustratively, a ±10 monthly wobble becomes about ±20 over four months (√4 = 2), not ±40, because ups and downs partly cancel.

In my experience, capacity forecasts rarely behave that way. Errors run in the same direction cycle after cycle (statisticians say positively autocorrelated), so they don't cancel; a growth-rate error keeps adding up, like a ship one degree off course. Illustratively, forecast 3% monthly growth and get 4%, a miss that could as easily have gone the other way. Then demand ends about 12% above plan after a year (1.04¹² ÷ 1.03¹² ≈ 1.12), and about 26% above after two. So the error at the need date can grow linearly with lead time or faster. Measure it: score past forecasts by horizon and size the buffer from that curve. Persistent bias is different: correct it at intake (Section 9.3), don't buffer it.

The effect can be startling. One operator's first statistical forecast of component demand "required us to triple our safety stock inventory in order to meet our on-time delivery goals," and was never implemented; the team then set a separate forecast quantile per component type [8]. And since, as Section 6.2 notes, deliveries are rarely early and sometimes very late, the buffer must cover a late truck as well as a demand surprise. Section 8 does that with a cautious lead time.

Pooling is the main economy. Two regions whose bad days don't coincide need less than two full spares. For n identical, uncorrelated normal demands, pooled safety stock is √n times one location's rather than n times [81]. Illustratively, two regions that each hold a spare of 10 need about 14 together, not 20; four need about 20, not 40. The 2026 SRE chapter makes the same point at the scale of a fleet. Provision for "the peak of the summed demands, rather than the sum of their individual peaks". In its example, pooled demand peaks at 48 units against 60 provisioned separately, and still needs a smaller buffer [7]. Google reports that pooling this way saves "approximately 5–20%" in its systems, and that easy access to capacity "reduces the incentive for teams to hoard" [7]. Correlation and heavy tails erode the gain [82]. Joe Weinman, whose book the chapter cites for this math, puts the condition plainly in a later article: "the assumption of workload independence is a key one" [83]. Queueing headroom pools the same way: at a fixed probability that a request must wait, a many-server pool's spare grows roughly with the square root of load [84]. Easy access to the pool reduces hoarding, the chapter notes [7]. I'd add that its buffer should be priced, so it isn't re-hoarded. Pooling has a counterweight, too: Meta describes moving AI training "from a model of scheduling globally fungible resources to pre-regionalized capacity contracts" [31]. Figure 21 puts the section's three ideas side by side.

Three panels. (a) Six named buffers: redundancy, maintenance reserve, queueing headroom, forecast-error buffer, launch reserve and inventory buffer, with one operator's published split of about 94% guaranteed, 2% for random failures and 4% for correlated failures [54]. (b) Independent monthly errors of ±10 grow to about ±20 over four months, not ±40. A growth-rate error, 4% a month against a 3% forecast, ends about 12% above plan after a year and about 26% after two. (c) Pooling: two regions holding 10 each need about 14 together, not 20; four need about 20, not 40. Illustrative except the cited split.
Figure 21. Name each buffer and size it on which mistake costs more. Lead time pushes buffers up, more than the textbook says when errors run the same way; pooling pulls them down. Illustrative, except the cited split [54]. Open full size: Figure 21

Buffers need not sit idle. Borg reports about 20% of a median cell's workload running in reclaimed resources [17]. A backfill policy must say whether batch tenants pay for the buffer they borrow.

6.4 Bridges and clearing

Picture the 9 p.m. shortage email: approved ask, late machines, launch still on the calendar. When approved demand can't be sourced in time, you bridge. The levers differ in speed and premium (speeds are my judgment of orders of magnitude, not sourced figures):

Bridge Time to capacity Main cost
Reclaim idle or over-held capacity Days to weeks Effort; friction with the holder
Shift demand across regions or time Hours to weeks for stateless serving; longer for stateful Latency, data residency, migration
Reallocate between tenants Weeks Migration; churn
Re-peg supply from a slipped launch to an on-time one Days Effort; the slipped launch re-confirms its date
Defer decommission of older hardware Weeks, if not yet retired and if its space and power aren't needed by incoming hardware Power, efficiency and reliability penalty
Third-party capacity Months, with a standing agreement; longer without one Price premium; capital becomes operating spend; usually a minimum term of its own
Facility redesign or retrofit Quarters to years Capital

Shifting demand is routine at scale: Flux reports global workload shifts every six weeks, each "reshuffling capacity for 100k-300k servers globally" [2]. In the SRE chapter's Meet case, capacity came partly from "borrowing machine resources from other teams across Google," a tenant-to-tenant bridge run under incident pressure [7]. Oversubscription is left out on purpose: it's better run as a standing policy, backed by peak prediction and power capping, than as an emergency lever [85–87]. The SRE chapter's pooled quota runs it that way [7]. Microsoft reports that predicting power peaks enabled "a 2× increase in oversubscription with minimum impact to critical workloads" [A3]. A later Borg study reports production compute usage at "only around 30% of the allocated level," with over-commitment as a standing bet [88].

A bridge has a lead time too. It works on your clock only if it was arranged before you needed it, so a standing third-party agreement is itself a slow-layer option. Power can be bridged too, with temporary on-site generation [60] (Appendix B).

Pegging makes the cheapest bridge possible. Supply planners link every order to the demand it was placed for; they call it pegging. It lets a late delivery be traced to the launches it hits, and a slipped launch to the deliveries it can hand to someone else. In my experience launches and deliveries both tend to run late, so the two can often cancel out, but only if the link exists.

Every bridge carries a premium, and the premium has a cause: a late ask, a forecast miss, an off-cycle launch. Charge the premium to whoever caused it. Say one team asks late, and the stopgap machines cost extra. If that extra is split across every team, the late team pays only a sliver of it. Everyone learns that asking late costs almost nothing, so more teams ask late. If the late team pays the full premium, its next ask tends to come on time. One case needs care: when a shared service caused it, say with a release that raised its own costs. That service recovers its costs through the rate every caller pays. So keep the premium out of the rate, as a cost the service carries itself. Otherwise every caller pays for one service's miss.

Clearing is the other half: one team holds capacity it doesn't need, another needs exactly that shape, there, now. Matching them is what Jevons called a "double coincidence" [89]: the baker who wants shoes must find a shoemaker who wants bread. Done by people brokering trades, it's a market with no price, and it scales poorly; the quota auctions and markets of Section 3 try to give it one [16, 18]. RAS automates part of it within one operator's region-wide server reservations [54]. The Quota Marketplace preprint does so for ML training accelerators, with teams bidding credits from an income that administrators "assign … to reflect their relative priority" [18]. The authors report average occupancy of 93% against 75% in a comparable conventional pool. That's the operator's observational comparison, not a controlled experiment, and occupancy is not goodput (useful progress; Section 11). Because the market can't see which accelerators share an interconnect domain, it produces preemptions "where quota exists but geometry does not" in about 1% of cases.

So markets automate clearing for fungible, short-horizon resources, and human work moves to weights, envelopes, shapes and long-lead supply, where the calendar, not just the shape, decides the outcome.

Human clearing runs on information. For example, if one team has spare cores but needs memory in three regions this quarter, someone has to find the other half. The trade clears only if someone already knows which team holds that memory there and needs the cores. Four things must match at once: the resource, the amount, the place and the time. That knowledge comes from a weekly habit of asking teams what's coming and what's idle, more than from any report. That's also why Section 10.4 says to log every trade: what the broker knows should outlive the broker.

6.5 Two clocks

In short: the slower a supply layer, the earlier you must order it and the less you know when you do. For persistent demand in a growing portfolio, this paper argues for aiming higher in the range.

Go back to the launch of Section 5.4: a rumor, then a date, then a configuration. The power for its building may have been ordered before anyone heard the rumor. Demand information and supply commitments run on different clocks. Alphabet's chief executive put the supply side in one line: "how we close the gap this year is a function of what we have done in the prior years" [90].

Each supply layer is committed at its own commit point: its need date minus its cautious lead time (Section 6.2). Hixson and Guliani advise planning at least as far ahead as the order time, and answer unresolved demand with fungibility [6]; this section works that out layer by layer. What matters for each layer is the residual uncertainty at its commit point. That's how much of the declaration is still open when the layer must be ordered, and how sure its arrival date is. Hixson and Guliani point at a closely related quantity. They tell finance to check "the sensitivity that those estimates have to their time horizon and how far into the future people are being asked to forecast" [6].

Figure 22 draws the two clocks as lanes. On top, the declaration's open range narrows at decision events (diamonds). Below, each supply layer's bar is its lead time, its dot the commit point, and the orange bracket above the dot the residual then. Land, power and shell commits before any declaration exists, with the whole range open. For simplicity every bar ends at launch; since each layer must be ready before the next starts, slow layers really commit earlier than drawn (Section 8 does it properly).

Put Section 7's launch on it. Machines and parts commit at −6 months, when the declaration is Dated: a base of 60,000 and a high of 90,000 requests per second. That leaves a 30,000 spread still open, and, as Section 7 shows, the launch could still land below the base. Network commits at −15, at Intent: 40,000 to 100,000, so 60,000 open. The further left the dot, the wider the bracket. The table generalizes Figure 22 and adds physical placement (commit points are illustrative and vary by operator and market):

Schematic timeline from 24 months before launch to 6 months after. Top, the demand clock: a declaration's open size range narrows in steps at decision events (funding approval, design review, go/no-go; a named owner and a completed ramp are marked hollow, as evidence rather than events), from Intent (months −18 to −9) to Dated (−9 to −3), Configured (−3 to launch), Live (launch to +3) and Organic (after ramp). Bottom, the supply clock: five layers, each a bar ending at launch whose left end is its commit point: land, power and shell (−24, before any declaration exists); network and large contracts (−15, at Intent); machines and parts (−6, at Dated); turn-up and allocation (−1, at Configured); placement and admission (launch, Live). Orange brackets mark the range still open at each commit point; at the land, power and shell commit point the whole range is still open. Instruments on the right: envelope for the portfolio and options; buy toward the high through options and cancel windows; buy to the base with the high as options, a reserve or a bridge; rank, cut line and grants; bridges and admission control. Illustrative. At each dot a tag names the stage the declaration has reached. At −15 the range still open is 40,000 to 100,000 requests per second; at −6, a base of 60,000 and a high of 90,000. A note says the bars are simplified; the method stacks need dates and uses cautious lead times.
Figure 22. Demand firms up on one clock; supply commits on another. At each layer's commit point, the range still open in the declaration, not the point forecast, picks the instrument. Open full size: Figure 22
Supply layer Typical commit point Where the declaration usually is by then Instrument
Land, power, building shell One to several years ahead Intent, or none An envelope sized for the portfolio, not one launch; options; a fungible specification (Section 6.2)
Long-haul network; large supply contracts Several quarters Intent to dated Buy toward the high, as options or with cancel windows
Machines and parts One to three quarters Dated Buy to the base; hold the high as options, a launch reserve or a named bridge
Physical placement (site, hall, power domain) With the machine order, or at turn-up Dated to configured A placement plan; spread across failure domains
Turn-up and allocation Weeks per job; the rate is staffed months ahead Configured Rank against supply; move the cut line (the line on the ranked list below which asks go unfunded); issue grants; book turn-up slots
Job placement and admission Minutes to days Live Bridges (shift, reclaim, defer decommission); admission control as the last line

Buy slow layers toward the high scenario. A shortage lasts as long as the short layer takes to fix or bridge: weeks for turn-up, quarters for machines, years for power when no third-party space is available. An excess costs the layer's carrying cost (Section 6.3) until growth absorbs it, which depends on the growth rate, not the lead time. And per unit of compute it enables, a slow layer is cheap to carry. Where power is paid for as drawn, an empty powered hall costs far less than a shortage of the machines it would hold. Take-or-pay power or a leased shell narrows that gap. 9So the cost of being short rises with lead time while, for shells and power, the cost of being over doesn't rise much. In this paper's argument (not a published result), the shortage-to-idle ratio therefore rises with lead time. Illustratively, for persistent demand in a growing portfolio:

Layer Shortage : idle cost Fraction Buy to about
Turn-up 1 : 1 1/(1 + 1) = 0.50 The median
Machines 4 : 1 4/(4 + 1) = 0.80 The 80th percentile
Power and shell 19 : 1 19/(19 + 1) = 0.95 The 95th percentile

Turn-up is the exception: it's a rate you staff (Section 6.2), so its ratio sets how much rate to staff, not how much to buy. Figure 23 draws the table as three ranges.

Three range bars. Turn-up, ratio 1 to 1, covered to the median. Machines, 4 to 1, to the 80th percentile. Power and shell, 19 to 1, to the 95th, mostly as options. Notes give the two conditions: layers must work together, and the high is held mostly as options. Illustrative; the paper's argument, not a published result.
Figure 23. For lasting demand in a growing business, the paper argues that the cover point should rise with lead time: median for turn-up, about the 80th percentile for machines, about the 95th for power and shells. Illustrative, not a published result. Open full size: Figure 23

Two conditions keep that sound:

  1. The layers have to work together (economists call them complements). A high quantile on power pays only if machines can still be bought once demand firms; that postponement lets the fast layers stay at the base. The "warm shells" remark of Section 6.2 is this condition failing the other way.
  2. "High" is a big amount at the slow end, where the open range is widest. Hold it mostly as options (Section 6.2).

One operator publishes the slow end of this: it plans "to a high quantile demand forecast" because "the cost of empty data center space is much less than the shortage cost of not being able to deploy compute and storage when needed" [8]. Its per-component quantiles, chosen to meet on-time delivery targets [8], show the quantile already varying by resource. Meta's chief executive put the argument in operator terms: "aggressively front-load building capacity so that way we're prepared for the most optimistic cases"; if superintelligence takes longer to arrive, "we'll use the extra compute to accelerate our core business," and in the worst case "slow building new infrastructure for some period while we grow into what we build" [74]. That last clause is the carrying cost of excess, paid as a pause in building. Meta's finance chief said in July 2026: "we believe near term capacity is more valuable than long term capacity" [91].

The argument weakens if growth stalls, since excess then waits years, not quarters. It also weakens if leased or third-party space can bridge the slow layer, or if the launches behind the high scenario share a driver (Section 6.6). For a transient step, or a launch that may not happen, the cost of being over is the whole unit. There, the high belongs in options and bridges, not owned capacity.

Specify in lead-time order. The clocks can move toward each other: in Figure 22, supply can slide its dots right, and demand can narrow its range sooner. Supply does it with shorter lead times, which Hixson and Guliani count among the most valuable things a company can do to cut planning risk and cost [6]. The SRE chapter says the same: faster supply shrinks "the window of time for which you actually need to predict capacity" [7]. It also does it with contracts that allow cancelling or pushing a delivery out. Demand does it most cheaply through what the intake asks first. Size and date usually firm last. But what drives the slowest supply (region, redundancy tier, hardware class and power density, data stored) is, in my experience, often decided early and not asked. The chapter describes the supply-side twin: bringing spatial awareness of datacenter footprints into "the long-range planning process" early, so demand forecasts are physically achievable [7]. Ask for it at the dated stage, with size as a range. For the slowest layer even that is too late; only the portfolio envelope can serve it.

Place the gates between the events. A planning gate is most useful just after the demand event that resolves something and just before the supply commit point that needs it. In Figure 22, that's between a diamond and the next dot. When a commit point falls before its event, as Appendix A's high-memory order does, there are three choices:

  1. Ask for the decision earlier.
  2. Buy an option that carries the commitment past the event.
  3. Commit to the stage's record and name the owner of the residual.

Write down which you chose: what was known, when, by whom, and what would have changed it. Figure 24 walks the choice for one layer, with Appendix A's case.

Bound how much the declaration may still move. Buying against the range open at a commit point works only if the declaration then stays inside it. Manufacturing's time fences handle this (the closer an order is to its date, the less it may change), as do quantity-flexibility contracts (Section 6.2). One way to apply it here is a change band per stage. For example, say an intent may move by half in either direction, a dated ask by a quarter and a configured ask by a tenth. Then a change inside the band is the plan working. One outside it becomes a priced event recorded against the declaration, not a surprise (Section 8, step 7). Figure 25 shows where the gates go and how the bands narrow.

Decision tree for one supply layer, in the same order as the method's flowchart. The first question asks whether any declaration exists at the layer's commit point. If none does, the portfolio envelope serves the layer and the launch's claim on it is recorded, as for land, power and shell. If one does, a second question asks whether the attribute that drives the layer (region, redundancy tier, hardware class and power density, or data stored) is known by its commit point. If yes, the planner goes on to cover, net and split. If no, there are three unranked choices: ask for the decision earlier; buy an option that carries the commitment past the event; or commit to the stage's record and name the owner of the residual. Each choice ends in a decision note recording what was known, when, by whom, and what would have changed it. A worked row from Appendix A: high-memory machines commit at −11 while the launch is an intent. Stored data is asked at −12 as a range of 8 to 20 TB, and one machine is ordered reschedulable until −3. Illustrative.
Figure 24. When an order is due before the event that would size it, ask earlier, buy an option past the event, or commit to the stage's record and name who owns the rest, and write down which. Before any declaration exists, only the portfolio envelope can serve the layer. Illustrative example from Appendix A. Open full size: Figure 24
(a) A decision event, then a gate, then a commit point. When the commit point comes first, there are three choices: ask earlier, buy an option past the event, or commit to the stage's record and name the owner of the residual. (b) Change bands narrow from ±50% to ±25% to ±10% around 60,000; the launch landed at 40,000, six weeks late, outside both bands. Panel (a) uses generic timing, panel (b) Appendix A's. The bands follow manufacturing time fences and quantity-flexibility contracts [70, 71]. Illustrative.
Figure 25. Put each gate between the event that answers a question and the order that needs the answer. Agree how far the declaration may still move at each stage. Illustrative bands. Open full size: Figure 25

6.6 Timing, size and persistence

In short: a launch can be wrong on size or on timing, can last days or years, and can share a driver with other launches. Each changes what to buy and who pays.

A size miss is paid for on what was bought for the size, mostly the slow and middle layers. Too high, and capacity waits for later demand; too low, and the gap needs bridges. A timing miss is, in my experience, most often a slip. It's paid for on everything committed for the launch, for the length of the slip, but shows mostly at the machine and turn-up layers. For example, if a building serves twenty launches, one arriving six weeks late hardly registers; the machines turned up for it do.

Section 5.2's variance decomposition shows neither error as what it is. A size miss shows as a smaller bill; a slip may not show at all, depending on how the consumer is billed:

Billed on realized peak, held flat for the year (Section 7's rule) Billed on coincident-peak contribution, hour by hour
Size miss A smaller bill (volume term) A smaller bill (volume term)
Slip Invisible: the consumer pays for the slipped weeks as if it had launched on time The bill drops for those weeks; their cost moves to the service's unused line

Capacity bought for demand that never arrives is billed to no one under either rule; it lands on the unused line. The service owner owns that line (mechanism 4), but the cost isn't booked to the declaration that caused it unless grants expire (Section 9.3). Each error needs its own owner.

Persistence is the second distinction. A transient step (a sale, a live broadcast, a launch-day spike) needs capacity for days; a persistent step (a new product, a new region), for years. Transient demand wants capacity that can be returned: shifted work, reclaimed and borrowed buffers, and third-party capacity. At the top of the peak, it wants admission control, the limit Alibaba accepts for an event it can predict but not plan for [33]. Persistent demand wants owned capacity. Many launches are both, a spike on a step. Size owned capacity on the spike and you buy permanent capacity for a temporary peak, so a declaration should carry the two shapes separately. Figure 26 shows what each kind of miss costs, and the two shapes.

Four panels. (a) A size miss is paid mostly on the slow and middle layers: too high, and capacity waits for demand; too low, and the gap needs bridges. (b) A slip is paid mostly at the machine and turn-up layers, for the length of the slip. A building serving twenty launches hardly notices one six weeks late. (c) A launch as a persistent step with a transient spike on top. The step wants owned capacity; the spike wants returnable capacity (shifted work, reclaimed and borrowed buffers, third-party capacity, admission control at the top). Sizing owned capacity on the spike buys permanent capacity for a temporary peak. (d) On the bill: a size miss shows as a smaller bill, in the volume term; a slip is invisible on a flat realized-peak bill, and under hour-by-hour billing its cost moves to the service's unused line. Schematic.
Figure 26. A size miss is paid on the slow and middle layers; a slip, on the machines and turn-up committed for the launch. A spike on a step needs two shapes: owned capacity for the step, returnable capacity for the spike. On a flat bill, a slip can be invisible. Schematic. Open full size: Figure 26

The third distinction is correlation. Hixson and Guliani suggest planning for only some of several launches to succeed, "a very specific way to trade increased risk for the organization against lowered cost," and tell finance to check whether products "appear to be correlated with each other in terms of growth and cost" [6]. They also name what ties products together: technical coupling, and growth linked by brand, news or cross-promotion [6]. That check is what makes the trade safe, because planning for only some launches to land works like pooling (Section 6.3): independent bets average out. Illustratively, four independent launches each half likely to land will usually deliver about two; all four land only about one time in sixteen (0.5⁴ ≈ 6%). Tie all four to one model release (or one strategy, partner or marketing date), and they land or slip together. Half the time all four land, and a plan for two is short by two. So name any shared driver in the register, and apply a portfolio haircut (planning for fewer successes than were declared) only within groups of bets that lack one. Name the driver; don't infer independence from a low measured correlation. Risk managers warn that dependence goes "beyond simple linear correlation," and the tails are where it shows [A4].

The same idea can size the launch reserve. Holding each launch's high scenario on its own is, in the chapter's terms, a sum of peaks [7]; the same pooling logic applies to launches. Hixson and Guliani already collapse several products' upside into one plan, with someone in a position to resolve conflicts [6]. The chapter pairs centralized inventory with a correlation check [7]. One way to size it: hold one reserve for a group of launches with no shared driver, sized to the group's high. Then give each launch a ranked claim on it (Section 8.3). Illustratively, four independent launches each have a 25,000 high slice, each half likely to be needed. Four separate slices make 100,000. At the 80th percentile, the group needs 75,000. Tie the four to one model release and they land together, so the group needs all 100,000. The chapter's own advice applies: first find which parts of a forecast "are correlated and which can be treated as independent" [7]. Figure 27 draws the two cases, and Figure 28 the reserve each one needs.

Left: four independent launches, each half likely (a coin marked 50%); usually about two land, and all four only about one time in sixteen. Right: the same four tied to one driver land or slip together; half the time all four land, and a plan for two is short by two. Planning for only some launches, and checking that they aren't correlated: Hixson and Guliani [6]. Illustrative.
Figure 27. Independent bets average out; bets tied to one driver land or slip together. Hixson and Guliani's correlation check [6] is what makes planning for fewer successes safe. Illustrative. Open full size: Figure 27
Two panels on one scale, in thousand requests per second. Separate reserves: four launches each hold a 25,000 high slice, 100,000 in all. One pooled reserve: with each launch independently half likely to need its slice, the outcomes 0, 25, 50, 75 and 100 thousand occur 1, 4, 6, 4 and 1 times in 16. Fifty thousand covers 11 of 16, under the 80th percentile; 75,000 covers 15 of 16, so the 80th percentile is 75,000. A strip notes that launches tied to one driver land together and need all 100,000. Peak of sums, after [7]; planning for only some launches [6]. Illustrative.
Figure 28. One reserve for a group of independent launches, sized to the group's high, holds less than the sum of their high slices. Tie the launches to one driver and the saving disappears. Illustrative. Open full size: Figure 28

6.7 Supply has step changes too

The new site promised for one date now has another. Supply moves in steps too: a new site comes online, or a hardware generation changes shapes and coefficients (Section 5.3). A vendor rations a constrained part, a decommission retires a block of capacity, or a supply shock delays everything at once. No trend predicts these; someone must declare them. Auxon's authors list a slipped cluster delivery, alongside a service's efficiency regression, among the events that break a hand-built plan [1].

Supply declarations firm up in stages of their own: planned, contracted, under construction, delivered, live. Utilities and operators already report their power this way, with contracted load far ahead of what is under construction or active [68, 92] (Appendix B).

Supply can go live in phases, too. The SRE chapter describes phased delivery that brings basic compute, power and network online months early [7]. It replaces treating capacity as a binary "ready" state, and the early capacity carries a higher risk of stockouts [7]. Public examples exist elsewhere. One AI campus's first phase, "two buildings and 200+ megawatts," was "expected to be energized in the first half of 2025" [93]. Alberta's grid operator allocated an interim connection limit of 1,200 MW and moved other large-load requests to a second phase [94].

Here's where the two sides can meet; the parts are the sources', the field is my exploration. The chapter already sorts work this way, routing work "resilient to transient capacity shortfalls" to the early phases [7]. It calls the property stockout resiliency, and has clients tag their own traffic [7]. Hixson and Guliani ask each product what happens if growth exceeds planned capacity [6]. My exploration is to make it a number, with a duration, on every declaration. So it helps to ask each declaration one more question: how much shortfall, for how long, can it take? Call it shortfall tolerance. Demand that declares some can be pegged to a site that's only partly live, or hold an over-issued entitlement (Section 6.1). Demand that declares none waits for firm supply. And since a higher availability tier is a driver that raises the rate (Section 5.2), a declared tolerance should lower it. My guess is that this field, more than any forecast, decides how much early supply a fleet can use.

So score each supply declaration like a demand declaration, promised size and date against actual (Section 9.2, mechanism 11). A planner who discounts demand intents but takes supply promises at face value has half an outside view. The mirror holds too. A supply team that commits to a demand signal that changes every month inside its lead time has no plan to keep. And its own supplier scores it the same way (Section 6.2).

Slips travel. A supply slip opens the approval-allocation gap from the supply side: demand approved against a promised building or generation has nowhere to land. A late transformer delays the building, then every rack and turn-up already committed on the old date; the slipped batches then queue at turn-up together (Section 6.2). Nor does value arrive in proportion. A rack missing one cable isn't 99% of a rack. A training cluster with one block of its network missing may not run the job it was built for at all (Section 11). So fulfillment watches the scarcest small part as closely as the most expensive one. Figure 29 puts supply's stages beside demand's, and follows one slip.

Two parts. At the top, demand's stages (Intent, Dated, Configured, Live, Organic) mirror supply's (planned, contracted, under construction, delivered, live), with both scored on promised against actual size and date by stage (mechanisms 2 and 11); a planner who discounts demand intents but takes supply promises at face value has half an outside view. Below, a chain: a late transformer delays the building, then every rack and turn-up committed on the old date, and the slipped batches queue at turn-up together, so a slip often shows up twice. Side notes: decommission is negative supply [2], so a slipped retirement can stall an on-time delivery; and four sites that wait on one transformer supplier or utility slip together. A footer says a supply slip is an envelope change in time, handled as a re-cut. Schematic.
Figure 29. Supply declares in stages too, and is scored the same way. One late part travels down the chain and arrives at turn-up with everything else that slipped. Schematic. Open full size: Figure 29

Supply steps are usually lumpier than demand steps: one building's slip reaches every launch it serves. Tie a long-lead supply step to the portfolio, not one launch, and it stays fungible when any launch moves. A supply slip is an envelope change in time, handled as a re-cut (Section 9.1); a shock across many layers is the same re-cut, larger.

Decommission is negative supply. Flux records large-scale decommissioning of servers in three regions, "resulting in a reduction of supply in those regions," and manages "projected regional capacity deficits such as those caused by regional hardware refresh" [2]. A retirement is a supply order with a minus sign. It has a date, and it slips, usually because tenants can't migrate off in time. And old racks in an existing building usually have to leave before new ones can use their power and floor. So a slipped decommission can stall a delivery that arrived on time. In my experience, the fastest bridge for a late new site is often an older one. Refreshing it or deferring its decommission gives worse hardware, sooner, where it isn't in the new hardware's way.

Check supply for shared drivers. The correlation check of Section 6.6 works for supply too. Four new sites with independent risks will usually deliver most of what was promised; if all four wait on the same transformer supplier or the same utility, they slip together. So the supply register names shared dependencies, and the plan counts on supply averaging out only where it really is independent. My guess, and it is only a guess, is that in a tight market lead times stretch together across vendors, because every queue fills at once. If so, supply errors average out least exactly when it matters most, and the cautious lead time of Section 8 should be more cautious then.