13. Where this breaks, and what is unknown
In short: where this paper's approach breaks, and what hasn't been measured in public, as far as I can find.
Caching, batching, statefulness. Per-request coefficients fail when cost depends on cache state, batch composition or data size. Miss-rate coefficients, declared apportionment rules and load-test models help, but Flux's residuals are partly a modeling problem, not only an ownership one [2].
Elastic cloud. In cloud, fulfillment moves to the provider but comes back as regional stockouts and reservation planning. Even guaranteed cloud capacity has a lead time. AWS accepts a future-dated capacity reservation "between 5 and 120 days in advance," recommends asking "at least 56 days (8 weeks) in advance," and may decline it [109]. Below materiality, the mechanisms cost more than they return, as the SRE chapter also advises (Section 12) [7].
No reference class. Take a step that's new at the resource level, such as a first accelerator cluster or a first region on a new continent. 12The planner can only bound it, phase it and buy options.
Demand no one can declare. For a step no one sees coming, such as a product that spreads far faster than planned, what remains are buffers, bridges and, at the limit, admission control [33]. Hixson and Guliani's Plan B for a "success disaster" covers the same case: graceful failure modes, efficiency work with an estimated time to deploy, and alert thresholds [6]. The SRE chapter calls it unplanned inorganic demand, and its Meet case is the public example [7]. Google's public account reports "30x demand" in three months, met while working against "a previously unthinkable 50x growth forecast" [110]. Its planners "could no longer rely on the extrapolation of historical data" [110]. Under the same shock, Azure's allocator team reports that "we increased the utilization limits in each cluster by 1%," spending buffer on purpose [111].
A public case, read through this paper. Here's one way to read the SRE chapter's Meet case with this paper's tools. It's my reading; the chapter's lessons are its own. The first signal was pages for regional capacity issues [7]. Declarations still fed the loop, since its loop started from four reviews, one of them customer onboarding and promotion plans [7]. But as I read it, none could have been made in time for the slow layers. So most slow layers' commit points had already passed. The team's daily loop asked whether there were enough resources "for the load we expect one week from now" [7]. That's the commit-point question with a one-week horizon, set by what could still move. What moved was what could move inside weeks. Machines came through regular ordering and through borrowing from other teams, a tenant-to-tenant bridge (Section 6.4) [7]. Bigger instances were a re-shape (Section 6.1), and efficiency work also moved capacity [7]. Then came "escape hatches" to degrade quality, built during the response before a crisis forced the team's hand [7]. That's the named slice of Section 8, step 6, built just ahead of need.
Two things follow. Where no one can declare in time, the named slice is what's left, so build it first. And the team had "deliberately avoided investing in deployment automation or capacity optimization" because the return wasn't there [7]. Skipping the discipline below materiality is a fair choice. It has a tail, and the chapter says so. Figure 47 puts the case on the two clocks of Figure 22.
![Two lanes from Figure 22, redrawn for the Meet case. Demand lane: the first signal is pages for regional capacity [7], and the stages arrive almost at once near the right edge; in the author's reading, declarations arrive too late for the slow layers. Demand grew 30x in three months [110]. Supply lane: the commit points of land, power and shell, and network sit far to the left, greyed as already past, also the author's reading. Machines are not greyed: they came through regular orders and borrowing [7]. Near the right edge, four fast levers: borrow, a bridge; re-shape; efficiency work; and escape hatches built ahead of need, inside a daily loop asking whether there is enough for the load one week out [7]. Schematic; a reading of a public case.](/figures/N2.svg)
Indicators become targets. By Goodhart's law, any indicator in Section 9.2 can be gamed once rewarded, so read them as a set.
What is unknown. This is argument from published sources and practice, not a measured study. In the literature I could read, 1I found no time series of coefficient drift, and no study of how often coefficients are recalibrated. Nor did I find published data on how well step-change declarations are calibrated by stage, or 2how often launches slip. One preprint describes arbitration among internal teams for ML training capacity [18]. Operators have stated splits and priority orders between internal and external demand [90, 102], but 3I found no published mechanism for arbitrating scarce accelerators between them. On how much uncertainty operators carry at a commit point, the published evidence I found is observational. One example is a "2x difference in the machine count forecasts during annual planning discussions" [8]. 6Attribution for stateful services and batched inference is open, and the tests of Section 9.2 have not been run.
On supply, 4I found no published order-to-dock time, turn-up duration or hardware contract terms. And Lawrence Berkeley National Laboratory finds little published on 5how long large new loads wait for power [65]. The commit-point method of Section 8 is untested. 15The most useful next contribution would be multi-organization data on bias, unallocated share and fulfillment age, before and after these mechanisms are adopted. Figure 48 sorts the paper's main pieces by the evidence behind them.
![Evidence ledger. Published: the cost-driver formula, computed at scale [1, 2]; attribution coverage and residuals [2] (tracing 52% of capacity usage in 2022; residuals of 3 to 5% for a stateless web product and 22% falling to 15% for databases), still open for stateful services and batched inference; and the nearest results on the mechanisms, parts safety stock at about 0.6 times under triangulation [8] and planning cut from three to four months to about one [34]. Published in part: lead times by layer [6, 8, 60–64, 109]; how long large new loads wait for power, on which little is published [65]; uncertainty at a commit point, with observational evidence only, a 2x difference in machine count forecasts [8]; and the pieces of the thirteen mechanisms [8, 31, 34]. Published in part, too: the slow end of the cover point rising with lead time [8]. Argued here: the cover point rising with lead time, not a published result; the thirteen mechanisms joined, argued from practice; and the commit-point method, untested. Not found in public sources: order-to-dock times, turn-up durations, hardware contract terms, coefficient drift over time and how often coefficients are recalibrated, how well declarations are calibrated by stage and how often launches slip, and mechanisms for arbitrating scarce accelerators between internal and external demand (one preprint covers internal teams only [18]; operators state splits and priority orders, not mechanisms [90, 102]). The tests of Section 9.2 have not been run. A footer says not found means in the sources the author could read, including the second edition of Site Reliability Engineering's capacity chapter [7].](/figures/E21.svg)