9 minute read
When you query an AI chatbot with questions such as: How does a data center work? Or: Why does my pouty basset hound prefer cheese over sausage? Here is what happens behind the technical scenes:
Your beautiful and meaningful text is converted into digital data (turning letters into strings of 0s and 1s). This data is sent via Wifi then fiber optic cables at the speed of light to the data center, directed to servers, bounces around amongst the GPUs which run trillions of calculations, tokens/next words are predicted, data is sent back through the fiber optic cable and to your screen where you go, hmmm interesting…
Basically, the query requires some serious computation. Every computation requires hardware. Each hardware cluster needs power. Every watt that enters the machine eventually becomes heat that must be removed or malfunction, fire in the bowery, death. A typical AI chat prompt requires .34 watt hours (or a 10 watt LED light bulb running for 2 minutes) and produces 1,224 joules of heat energy that must be removed (how much heat a sitting human body produces in the span of 12 seconds). With ChatGPT receiving more than 2.5 billion prompts/messages per day, this is enough heat to boil 3.6 Olympic swimming pools of water (850 MWh of thermal energy).
The more powerful the system, the more extensive the physical needs become. The U.S. Department of Energy has stated that in 2023 data centers accounted for 4.4% of U.S. electricity consumption. By 2028 this number could almost triple to reach 12%. To put this in perspective, all residential homes in the U.S. account for 37.3% of electricity consumption. So in 2028 data centers are expected to consume as much electricity as 43 million residential homes, compared to a previous, measley 5.8 million homes.
A.I. queries, depending on mode size, chip efficiency, prompt length, caching, output type, and data center design can use several times to 1000x more energy than traditional search. A typical Google search now prompts an AI overview.
It’s important to note that the danger of AI use becoming excessive is not one person asking an AI chatbot to create an image of a monkey eating a hamburger. It is that every single digital product becomes an AI product, every company uses AI by default, hordes of self-replicating and self-learning AI agents populate the web (AI bot traffic has already surpassed human traffic on the internet), and the physical system behind it all either grows to meet the incredible demand, or grows to meet a world where AI is not as prevalent as previously predicted.
If AI becomes the default for almost all of digital activity online, you can imagine why the hyperscalers are frantically building data centers like it’s the Manhattan Project during WWII.
J.P. Morgan forecasts $2.9 trillion spent on building data centers by 2028.
It is important to note here the geopolitical pressure (and investment) as well: AI capacity is linked to cybersecurity, scientific research, and military planning. If China has superior A.I. models than the U.S. (and the Chinese frontier models are currently nipping at the heels of Anthropic and OpenAI), this will make Americans anxious. This is why the president of the U.S. announced government support of the $500 billion to be invested in Stargate.
The hyperscalers not only fear geopolitical weakness, but that data center demand will arrive faster than their infrastructure’s ability to serve it. The hyperscalers also dream of AI helping us to cure cancer, produce unlimited clean energy, and to make custom-made watches for select OpenAI employees with prices tags of $180,000.
While in the early dinosaur days of enterprise computing, data centers were considered primarily as internal cost centers, places to support business operations, they are now considered strategic financial assets.
Strategic financial assets? Why? How? Capitalism, what did you do?
One of the factors behind the shift was the emergence of “Infrastructure as a Service model,” which changed computing resources into billable units. This process, first used profitably by Amazon Web Services (AWS), is what skyrocketed Amazon’s growth and profitability as a company in the mid 2000s. With an excess of data centers and compute, which Amazon created to manage its own website and data, AWS could rent their compute, storage, and networking capacity demand (one of their early customers: Netflix). Using Amazon’s compute infrastructure now generates significant revenue. AWS routinely generates 50-75% of Amazon’s total corporate operating profit.
Basically, with “infra as a service model,” data centers could now be treated like telecommunications networks, power plants, or transportation systems: assets that require enormous upfront capital investment, but have long operational lifespans and produce returns over extended periods of time through continuous utilization.
What does this “continuous utilization” resemble for data centers? To run a GPU for an hour, it costs a data center a few pennies in electricity. They can charge up to 4 dollars for an hour of GPU time. This is attractive economics. Build it (a 1 GW data center has 700,000 – 1 million GPUs), and let the money come streaming in. No sweat or problem-solving, just click and charge.
At the feverish moment we’re living through, in which hyperscalers are building huge campuses with the expectation that capacity will be filled over time as demand for AI and cloud services grows, forecast models are used to justify this buildout. These forecast models account for long term trends in compute demand (see Chapter 6 on demand) network traffic growth, and regional market expansion.
AI training and inference require extremely high density of compute environments, large scale GPU infrastructures, and specialized networking fabrics.
Let’s look at the specifics of these complex environments. Building a typical AI data center takes place in five steps: buying land, construction, installing electrical systems like substations with transformers and switchgears, mechanical systems including chillers and cooling towers, and IT equipment such as servers and networking gear. While summarizing the process, I will use OpenAI’s buildout of Stargate as a case study.
Step 1: Buy Land!
OpenAI did not actually buy the land for the first Stargate campus, but rather Lancium (a privately held Houston-based energy-infrastructure and data center developer) had bought the land in Abilene, Texas back in December of 2021, years before Stargate was announced. It purchased the land (around 1,000 acres) for an estimated $4 million (this information is not public, the $4 million was calculated using adjacent land sales). It had plans to build a $24 billion data-center campus, initially 200 MW but expandable beyond 1 GW. Stargate’s ambition is 1.3 GW for the Abilene campus, then a whopping total of 10 GW across all of their campuses. The Abilene location was chosen for three, principle reasons.
- Cheap land
- Proximity to abundant wind and solar power generation (excess energy)
- Highly accommodating local government
Power availability is the most important factor in modern data center site selection.
2.) Construction!
The first step for building a data center is flattening the soil. For Stargate, satellite tracking indicated that land clearing for the first two buildings began around May 2024. Mass excavation/balanced cut-and-fill costs around $31,000/acre. There were 8,500 workers on site daily. So for 1,000 acres, the flattening of soil cost between $40-$125 million. This is less than 1% of the $15 billion project.
By June 2024, a month later, formal building construction was taking place. They were working day and night, 6-7 days/week, in 10-12-hour shifts. By June 2025, the first GPUs arrived: Crusoe confirmed that the first two buildings were energized within a year. The 12-month dirt-to-live compute timeline was a record for greenfield hyperscale development.
So what was happening within the building? There are three stages for data center building construction:
Shell, Core, Fit out.
For the shell, steel or reinforced concrete can be used. Crusoe used structural steel for the first 485,000 square foot, single-story building, constructed with steel columns, beams and frames. The shell makes a weatherproof box. A normal building of this design takes 6-8 months, but Crusoe compressed it to five weeks. For the Core, Crusoe used insulated metal exterior panels (IMPs).
The Fit out is where the AI factory gets built. Each Stargate has a central utility hall and four ~25 MW data halls, giving around 100 MW of IT load per building. The entire campus will comprise of eight buildings. At the time of this writing, the shells of the eight buildings have been erected, and at least 4/8 buildings are estimated to be operational.
We should note here that the cost of IT equipment within this phase can often exceed the cost of building and infrastructure itself, especially in AI focused deployments. The buildings also come online incrementally.
Once you have these GPUs installed, you don’t just plug in 50,000 GPUs turn everything, and shout, “We’re rich!” Every system has to be tested beforehand. Engineers test power failure, UPS response, generators’ responses, pump failure, cooling load increases, electrical component failing, network failures, fire alarms, but we’re getting ahead of ourselves (this sound familiar?) Before these tests, you need…
3.) Electricity!
Electricity is often the largest ongoing expense in data center operations. There is always consideration of sites located near existing substations or transmission corridors. Data center architects analyze utility engagement to understand available capacity.
Climate of the land will also sometimes play a role in the choice: the average temperature will determine the cooling efficiency. This is why a small, cold village in Fife Scotland, Auchtertool, has been fighting against ILI Cato LTD, which has proposed a 600 MW hyperscale AI data-center campus right next to the village. The Auchtertool Action Group has formally opposed the project, generating about 1,600 objections, since the campaigners estimate that at full operation, this AI data center’s electricity consumption could be comparable to roughly half of Scotland’s households.
Which is why hyperscalers often deeply analyze the regulatory environment. This may be even more important than technical concerns like low/high latency to population centers or natural disaster risks. Texas isn’t as cool as Scotland, but the local government officials are more likely to give the green light for infrastructure projects.
Abilene granted Stargate an 85% property-tax abatement for 10 years. It was initially 100%, but the arrangement was subsequently amended and restructured. The 85% number means that $22.6 billion of cumulative property value-years are removed from the tax base over a ten-year schedule.
Texas has a major data-center sales tax exemption: Texas exempts qualifying data centers from its 6.25% state sales and use tax on things including: servers and computing hardware, data-storage devices, electrical systems, cooling systems, software, generators, i.e. all data center things.
The availability of transformer capacity has become a limiting factor in data center development times. Some of the components have long lead times (the time it takes for a component to arrive when you order it). This is why Amazon is building their next data center using single gas turbines (18-24 month lead time) rather than multiple gas turbines (2-3 years), even though single turbines are worse for the environment.
The existence of wind and solar in West Texas was attractive because West Texas frequently had more wind/solar generation than the transmission system could economically move to population centers.
In 2022, ERCOT (Electric Reliability Council of Texas) curtailed 5% of all available Texas wind generation and 9% of the available utility-scale solar generation, electricity that could have been generated but wasn’t, because of grid constraints. Abilene sits near a transmission bottleneck between wind-rich West Texas and the rest of the grid.
When the CEO of Lancium selected the location in 2021, he discussed how generators in the region were effectively paying the market to take its electricity (negative-price electricity).
Today, Stargate’s Abilene campus has 1.2 GW ERCOT-approved grid interconnection.
It is important to keep in mind that this excess solar/wind power is not enough to continuously supply a full-scale Stargate campus. Power will be supplied by the ERCOT grid (a mix of wind, solar, natural gas, nuclear, coal, batteries), new solar (Lancium plans to build 1,000 MW of new solar), enormous batteries (Lancium’s plan shows a proposed 1,000 MW – 4,000 MWh battery system), and natural gas (Lancium’s plan shows a 360 MW natural-gas power plant). This pressure on the electricity grid could increase energy bills for Texans.
Wait, why all of these diverse energy sources? Why wouldn’t excess wind/solar energy be enough?
Enormous electricity is needed, not only for running the GPUs, but for backup generators. Due to the sensitive nature of the data (hospitals, military, your cousin’s ex-girlfriend’s baby photos) there are many backup generators, dozens or even hundreds of units, the aggregate capacity of these systems is often equal to or greater than the full capacity load, ensuring that no single point of failure can compromise operational continuity. There are high capital costs to house, fuel, and maintain them.
Uninterruptible power supply systems (UPS) form an intermediate layer. Typically, these systems rely on large battery banks, because instantaneous power is needed between utility and generator supply.
The next challenge is distributing power within the data center facility, so patient records and videos of bears eating burritos are maintained. This is done through a hierarchy of switchgear, busway systems, power distribution units, and rack level power supplies.
Modern artificial intelligence workloads, like asking a chatbot whether Socrates or Phil Collins have had a greater impact on human civilization, may require power delivery in the range of tens or even hundreds of kilowatts per rack.
Again: power consumption is the single largest ongoing cost in most data center environments, and the efficiency of the electrical systems directly affects profitability.
Let’s talk about switchgears and substations:
A hyperscale data center like Stargate acts like an enormous electrical funnel. Electricity arrives at extremely high voltage, and a sequence of substations, transformers, and switchgears progressively turn it into electricity the GPUs can actually use.
Electricity arrives at voltage between 138 kV and 345 kV, depending on the site (high voltage allows you to move enormous amounts of electrical power with much less energy lost as heat). A transformer, through electromagnetic induction (AC electricity creates a changing magnetic field in the transformer’s core), induces a lower voltage in another winding.
Switchgear doesn’t normally change voltage, but rather decides where electricity goes and whether a circuit should be connected or disconnected. Inside switchgear you have busbars (massive conductors carrying electricity), protective relays (detects abnormal electrical conditions), and meters/sensors (measures voltage).
Switchgears also allows essential redundancy: if a transformer in a substation fails, a switchgear can isolate it and allow another electrical path to support the load, depending on the facility’s redundancy design. Meanwhile, batteries/UPS systems can bridge the extremely tiny interruption while switching occurs or generators start.
You can never NOT have electricity running through these GPUs.
To sum it up: substation receives enormous amounts of electricity from the grid and distributes it around campus, the transformers convert the voltage to manageable levels, and switchgear acts like the electrical traffic controller and emergency protection.
We’ve come a long way from Benjamin Franklin flying a kite in a lightning storm and Edison’s light bulb.
4.) Cooling!
As previously mentioned, the consumption of electricity by GPUs produces enormous amounts of heat that needs to be dissipated.
There are various ways to cool GPUs, and the different methods have tradeoffs. There’s air cooling, liquid cooling, and emerging thermal technologies.
Cooling via air flow is inefficient. Liquids like water more efficiently transfer heat.
In the news there has been citizen outrage against data centers due to the water consumption that many of these data centers are using to cool their GPUs.
EESI has noted that large data centers can consume up to 5 million gallons of water per day.
Stargate will use direct to chip liquid cooling, but rather than pulling water from the surrounding ecosystem, it will use liquid cooling in a closed loop.
This closed loop system is necessary because the Nvidia GB200 systems are packed too densely for conventional room air-conditioning to do most of the cooling efficiently. Crusoe has claimed that their closed system is non-evaporative, meaning that it repeatedly recirculates the same cooling water rather than continually evaporating water through cooling towers. Around 23% of data centers require evaporative/adiabatic heat rejection, or water being constantly pulled from an outside source.
The closed loop system works as follows:
A cold plate sits directly on the GPU. Rather than fans blowing cold air across the chips, this metal plate is attached to the hot components. Coolant flows through tiny channels in that plate. The liquid carries away the heat from the rack. The warmer coolant travels through pipes to a CDU – Coolant Distribution Unit. The CDU contains pumps, heat exchangers, controls, filters and sensors. Crusoe says that Stargate rejects the heat using air-cooled chillers. Outside the buildings, you’ll find enormous arrays of heat-rejection equipment with fans. Thermal imagery of buildings allow observers to identity which Stargate buildings are functional.
“Of a 2 ton rack of GPUs, 1.95 tons of that is used for cooling,” said Jensen at the 2025 U.S. Saudi Investment forum, “just imagine how tiny that little supercomputer is.” The cooling and electricity challenges are why data centers in space (with solar panels) are so compelling: send those tiny supercomputers up to space once the cost of sending cargo goes down, then have them use the sun’s energy and send the heat exhaust out into the void.
One building on Earth, engaged in 100 million watts of computing, must continuously dispose of 100 MW of heat at full IT load, plus deal with other facility loads. This heat could boil water in 10.3 Olympics swimming pools per day (25.8 million liters).
5.) Installing IT equipment such as servers and networking gear!
Once Crusoe (which has leased the land from Lancium) has finished testing the infrastructure, making sure that everything is IT ready, Oracle leads deployment of the actual IT hardware and software.
Oracle brings in the Nvidia rack systems, position and connect them, cable networking, bring the systems online, and configure the OCI environment and commission the compute. OpenAI consumes the compute for training and inference.
Oracle committed to purchasing roughly 400,000 Nvidia GB200 chips ($40 billion), and will be paid by OpenAI for using those chips. Oracle is not unique in making billion-dollar investments in AI, but is rather 1/5 hyperscalers going all in (including Amazon, Alphabet/Google, Microsoft, Meta). In the next chapter we will take a deep dive into their trillion-dollar bets, and the exact scale of this infrastructure buildout.
*My basset hound’s distinct preference for cheese over sausage comes down to chemical addiction, olfactory stimulation, and stubborn breed psychology.*