Lessons from the reading list.

Bits need atoms

Frontier training runs need so much computation that no ordinary computer could finish them in a human lifetime; they need tens of thousands of accelerators in purpose-built buildings. Most historical data centers were small rooms inside other buildings; the dedicated, campus-scale facility is a post-internet artifact.

GPT-4 training compute (estimate)

≈2 × 10²⁵ FLOP

Frontier compute growth

≈4–7× per year

Rack unit

1U = 1.75 in; racks commonly 42U or 48U

Source: Brian Potter, Institute for Progress / Construction Physics, 2024Liquid-cooled GPU rackData hall

Data centers are measured in megawatts, not square feet

A single server draws a few hundred watts, but tens of thousands together make a large data center a 100 MW load, comparable to tens of thousands of homes. Campuses of several buildings reach the scale of a large power plant.

100 MW (as estimated)

≈75,000 homes

Air needed to remove heat

≈120 CFM per kW (≈12 million CFM for 100 MW)

Source: Brian Potter, Institute for Progress / Construction Physics, 2024High-voltage transmission lineCRAC / CRAH units

Efficiency kept total demand flat for a decade

PUE fell, power supplies got far more efficient, idle servers learned to throttle down, and chips got more efficient per operation. That held aggregate data-center electricity roughly flat for years despite huge growth in traffic, but most of the easy overhead savings are already captured.

Average PUE, 2007

≈2.5

Average PUE, early 2020s

≈1.5

Best hyperscale PUE

≈1.1

Server power conversion

≈60–70% (old) to ≈95% (modern)

2015–2022 (IEA est.)

Workloads +340%, energy +20–70%

Source: Brian Potter, Institute for Progress / Construction Physics, 2024Server PSU / VRM chainUninterruptible power supply

What the tiers actually require

Higher tiers do not just add generators: they set siting distances from flood zones and airports, fire ratings, floor loading, fuel on hand and the number of active power paths. Most large US facilities sit between Tier III and IV, and human error is still the usual cause of outages.

Generator fuel on site

Tier I/II 8–24 h · III 72 h · IV 96 h

Floor loading

≈150–175 psf (I/II) · 250 psf (III/IV)

UPS redundancy

N or N+1 · N+1 · 2N

Power paths

1 · 1 active + 1 passive · 2 active

Source: Brian Potter, Institute for Progress / Construction Physics, 2024Diesel generatorsFuel storage and polishingUninterruptible power supplyFoundations and structural slab

AI racks outgrow air

Racks drew about 1 kW in the early 2000s and 10–20 kW in modern enterprise and hyperscale halls. A 32-GPU H100 rack exceeds 40 kW and a GB200 NVL72 rack about 120 kW. Air tops out around 20–30 kW per rack (perhaps 50 kW with rear-door exchangers), so AI pushes buildings toward direct-to-chip liquid and, later, immersion.

Early 2000s rack

≈1 kW

H100 SuperPOD rack

>40 kW

GB200 NVL72 rack

≈120 kW

Air cooling limit

≈20–30 kW (≈50 kW with RDHx)

Source: Brian Potter, Institute for Progress / Construction Physics, 2024Liquid-cooled GPU rackCold platesRear-door heat exchanger

Small globally, huge locally

Data centers were roughly 1% of world electricity in 2022, but clustering makes them dominant in some grids, which is why some regions have paused or restricted new builds and utilities now ask for minimum-take commitments.

World electricity, 2022 (IEA est.)

≈1–1.3%

Ireland

≈18% of electricity

Virginia (one utility)

≈24% of power sold

Projection to 2030 (one estimate)

≈3–4.5% of global demand

Source: Brian Potter, Institute for Progress / Construction Physics, 2024High-voltage transmission lineUtility revenue metering

Buildings take a year or two; power takes years, or never

Utilities plan for decades-long assets while data centers move on short cycles, and new transmission is slow to build. That mismatch pushes builders toward new regions, on-site generation and long-term commitments to utilities.

Source: Brian Potter, Institute for Progress / Construction Physics, 2024High-voltage transmission lineOn-site gas turbines / bridging powerBattery energy storage system

Four raw inputs: land, power, water, fiber

Before any building, a site needs stable ground with room to expand, firm power with redundancy, water for heat rejection, and fiber. Power is the scarce one.

Direct cooling water for 100 MW (as reported)

≈2–4 million L/day

Source: Kelly Greer et al., Crucible Capital, 2025High-voltage transmission lineWater service entranceCarrier fiber entrance

No permits, no data center

Zoning and environmental approvals start before everything else. Local opposition has blocked or delayed tens of billions of dollars of projects.

Zoning and land use

≈3–12 months, up to 36

Environmental review

≈6–18 months

Source: Kelly Greer et al., Crucible Capital, 2025Security perimeter and gatehouse

Front of the meter or behind it

Grid power is cheaper to start but slow to get; large loads can wait years in interconnection queues. Behind-the-meter generation trades that wait for fuel, permitting and isolation from the grid. Many developers still file the grid request to keep options open.

Load study

≈1–3 months

Grid request <75 MW

≈6–24 months

Grid request >75 MW

≈2–5 years

Behind-the-meter build

≈2–5 years

Source: Kelly Greer et al., Crucible Capital, 2025High-voltage transmission lineOn-site gas turbines / bridging power

Transformers are the long pole

Transformers are often the longest-lead item on the project. Faster transformer delivery shortens the whole critical path, including permits and interconnection approvals that depend on equipment dates.

Typical lead time (as reported)

≈50–100 weeks

Source: Kelly Greer et al., Crucible Capital, 2025Main step-down transformerUnit substation transformer

Every power source is a trade

Gas is fast and firm but emits; small reactors are clean and firm but not yet commercial; solar and wind are cheap per MWh but need huge land and storage for 24/7 supply.

Gas capex

≈$720–1,680 / kW · 5–10 acres per 100 MW

SMR capex (est.)

≈$4,000–6,000 / kW · 10–20 acres

Solar PV

≈$1,530–1,790 / kW · 500–700 acres

Onshore wind

≈$1,430–1,810 / kW · 3,000–5,000 acres

4-hour batteries

≈$600–1,400 / kW · 1–2 acres

Source: Kelly Greer et al., Crucible Capital, 2025On-site gas turbines / bridging powerBattery energy storage system

Redundancy has a vocabulary

Uptime Tiers describe resilience: Tier III is concurrently maintainable (N+1 overall); Tier IV is fault tolerant (2N). Each step up buys uptime with duplicated equipment.

Tier I

99.671% (<28.8 h/yr down)

Tier II

99.741% (<22 h/yr)

Tier III

99.982% (<1.6 h/yr)

Tier IV

99.995% (<0.4 h/yr)

Source: Kelly Greer et al., Crucible Capital, 2025Uninterruptible power supplyDiesel generators

If there is no fiber, build it (slowly)

Screen for nearby routes, carriers and dark fiber first. Extending fiber to a remote site takes design, permits and construction, and costs scale with distance and terrain. Diverse routes are mandatory.

Average build cost (as reported)

≈$100k per mile, up to ≈$2M for a difficult mile

Timeline

≈6–24 months

Source: Kelly Greer et al., Crucible Capital, 2025Carrier fiber entranceDiverse fiber route

Offtake unlocks capital

Hyperscalers once self-funded; now even they tap private credit. For independent developers, signed compute offtake (who will rent the capacity) is what makes debt and equity available on good terms.

Traditional build

≈12–24 months

Modular build

≈6–12 months

Levered equity IRR (as reported)

≈12–19%+

Source: Kelly Greer et al., Crucible Capital, 2025Prefabricated AI pod modulesGPU cloud control plane

Rack density is the antagonist

Each accelerator generation raises power per rack faster than electrical and cooling systems can scale, so the site, not the silicon, sets the ceiling on cluster size.

Pascal (2016)

≈10–20 kW/rack

Volta (2017)

≈15–25 kW

Ampere (2020)

≈25–40 kW

Hopper (2022)

≈40–60 kW

Blackwell (2024)

≈130–250 kW

Rubin-class (announced)

Up to ≈600 kW–1 MW

Source: Kelly Greer, Meltem Demirors, Crucible Capital, 2025Liquid-cooled GPU rack

Power and cooling dominate the non-IT budget

Power and cooling systems are roughly a third of facility capex and the majority of operating cost. It is a better-power and better-heat-management problem, not only a more-power problem.

Power + cooling share of capex (as reported)

≈35%

1 MW of new GPU equipment (as reported)

≈$40M

Source: Kelly Greer, Meltem Demirors, Crucible Capital, 2025

The return of direct current

Legacy AC chains convert repeatedly (AC to DC for batteries, back to AC, then DC at the server), losing a noticeable share of power. 800 V DC rectifies once at the facility edge, cuts copper per rack, and steps down at the rack with SiC or GaN converters.

Legacy conversion losses

≈10–20% end to end (as reported)

Copper busbar per legacy dense rack

Up to ≈200 kg

800 V DC claims

≈45% less copper, ≈98% conversion efficiency

Source: Kelly Greer, Meltem Demirors, Crucible Capital, 2025Server PSU / VRM chainOverhead buswayRack PDU

Batteries, BESS and supercapacitors each have a job

Training jobs swing power in milliseconds. Lead-acid UPS reacts in tens of milliseconds; supercapacitors smooth sub-second spikes; BESS handles seconds to hours. Smoother load also makes utilities friendlier.

UPS bridge

≈5–15 minutes

Training spikes

Up to ≈3× nominal for milliseconds (as reported)

LFP BESS

≈95% round trip, 6,000+ cycles

Source: Kelly Greer, Meltem Demirors, Crucible Capital, 2025Uninterruptible power supplyBattery energy storage system

Warm water deletes chillers

Liquid carries far more heat than air. With supply water at about 45 °C, direct-to-chip loops can reject heat without chillers for most of the year, which saves energy, capex and water. Not every vendor's design runs that warm yet.

Air cooling practical limit

≈15–20 kW/rack

Chiller share of capex (as reported)

≈15–20%, roughly $2M/MW

Source: Kelly Greer, Meltem Demirors, Crucible Capital, 2025ChillersCoolant distribution unitDry coolersCold plates

Cooling fails, and it matters

Cooling problems remain a meaningful share of reported outages. Every liquid deployment still feels custom to operators; seals, fittings and fluid chemistry are where things go wrong.

Cooling share of failures in 2024 (Uptime, as reported)

≈13%

Source: Kelly Greer, Meltem Demirors, Crucible Capital, 2025Leak, flow, pressure and temperature sensingRack manifolds and quick disconnectsCoolant distribution unit

Immersion and the balance sheet

Immersion is thermally excellent, but servicing is messy, warranties get complicated, and submerged GPUs are harder to repossess and resell, which matters when GPUs secure the loans.

Source: Kelly Greer, Meltem Demirors, Crucible Capital, 2025Immersion cooling tanks

Monitoring should produce work orders, not noise

Operators drown in alerts. Useful monitoring is real time, GPU-aware and API-first, and it ties telemetry to manuals and history so the output is a prioritized maintenance job.

Source: Kelly Greer, Meltem Demirors, Crucible Capital, 2025DCIM / BMS / EPMSNetwork / operations center

The network decides how much of the compute you get

Even well-run training rarely uses more than about half of theoretical FLOPs, largely because of networking and memory stalls. Interconnect can be a large share of cluster power.

Strong training MFU (as reported)

≈40%

Network share of cluster power (as reported)

Up to ≈30%

Source: Crucible Capital, 2026AI scale-out fabricGPU cloud control plane

North-south is the front door; east-west is the building

Traffic to users and storage is north-south. GPU-to-GPU traffic is east-west, split into scale-up (inside the rack, NVLink-class) and scale-out (between racks). Almost all networking innovation targets east-west.

Source: Crucible Capital, 2026AI scale-out fabricScale-up interconnectSpine switches

InfiniBand versus Ethernet

InfiniBand brought RDMA and dominated AI clusters into the early 2020s. An industry coalition then built Ultra Ethernet to add RDMA, congestion control and in-network features to Ethernet, and large training runs proved Ethernet viable.

InfiniBand share of AI clusters, 2023 (as reported)

≈80%

UEC 1.0 specification

2025

Source: Crucible Capital, 2026AI scale-out fabric

Copper keeps retreating toward the chip

As link speeds double every two to three years, copper's useful reach shrinks: a few meters at 56G-class, one to two at 100G-class, and at 800G only the last meter or two. Beyond that, light wins because fiber does not need re-driving every few centimeters.

Source: Crucible Capital, 2026Top-of-rack switchOptical transceiversScale-up interconnect

CPO, LPO, XPO and optical circuit switching

Co-packaged optics cut optical power but are hard to service and early in supply. Linear pluggables capture much of the benefit with familiar hot-swap modules. Larger cold-plated pluggables and optical circuit switches are other paths; expect several to coexist.

CPO optical power savings (as reported)

≈30–40%+

LPO share of CPO benefit (as reported)

≈70–80%

Source: Crucible Capital, 2026Optical transceivers

Hardware needs software to reach its peak

Collective communication libraries, kernel optimization, congestion management and network observability decide how much of the fabric's bandwidth jobs actually see.

Source: Crucible Capital, 2026AI scale-out fabricGPU cloud control plane

The cloud began as a symbol

Telephone engineers around 1970 drew networks they could not fully specify as cloud shapes. The cloud has always meant turning real, messy infrastructure into one logical object: virtualization.

Source: Tung-Hui Hu, 2015

New networks are grafted onto old ones

Long-haul fiber often runs under 19th-century railroad rights-of-way; one major carrier's name came from a railroad's internal network. The logical internet looks distributed, but its physical layer remains concentrated on old routes.

Source: Tung-Hui Hu, 2015Carrier fiber entrance

The perfectly distributed network was never built

Paul Baran's famous 1962 mesh diagram described a resilient network that economics and geography never allowed. Diverse routes still converge on a few corridors and buildings.

Source: Tung-Hui Hu, 2015Diverse fiber routeMeet-me room

Time-sharing invented the user and the utility

1960s time-sharing split one costly computer among many people and billed them by usage, inventing user accounts and the dream of a computer utility. The cloud revived that model: in 2014 an hour of rented compute cost about a thousandth of an hour of a 1963 mainframe in nominal terms.

IBM 7094 time, 1963

≈$450 / hour

Cloud instance, 2014

≈$0.45 / hour

Source: Tung-Hui Hu, 2015GPU cloud control plane

There is nothing special about a data center (from outside)

Most data centers are squat, windowless warehouses in office parks and exurbs, often reusing old factories, telephone exchanges or Cold War bunkers. Siting follows cheap power, water and land near fiber.

Source: Tung-Hui Hu, 2015Data hallSecond data hall / powered shell

Security, disaster recovery and cold storage

The industry sells safety: fortified halls, disaster recovery that fails over in milliseconds, and cold archives meant to last decades. Digital media still decays; formats and readers go obsolete faster than the data.

Source: Tung-Hui Hu, 2015Security perimeter and gatehouseSecurity mantrap and access controlStorage systems

The cloud converts water and electricity into computation

Calling it a cloud hides a physical, resource-intensive industry. Seeing the substations, cooling towers and supply chains is the point of this atlas.

Source: Tung-Hui Hu, 2015Cooling towersHigh-voltage transmission line