GPT-5 has not yet been released, and OpenAI seems to have begun training GPT-6, but electricity may have become a “stuck neck” problem.

Kyle Corbitt, co-founder and CEO of AI startup OpenPipe, revealed that he recently spoke with a Microsoft engineer responsible for the GPT-6 training cluster project. The latter complained that deploying InfiniBand-level links between GPUs across regions is really painful.

When Corbitt asked why the training clusters were not concentrated in the same region, the Microsoft engineer answered, “Oh, we've tried that, but if you put more than 100,000 H100 GPUs in a state, the power grid will collapse.”

What is the concept of 100,000-piece H100? As a reference, a report by market research agency Factorial Funds shows that OpenAI's Wensheng video model Sora needed 720,000 H100 pieces during the peak period — enough to crash power grids in seven states, according to the engineers mentioned above.

Is the data center profitable and how long will it take to build? Look at the “eye color” of electricity

It just came to an end$S&P Global (SPGI.US)$At Cambridge Energy Week 2024 (CERAWEEK), energy industry executives from around the world talked about the advancement of AI technology to the industry, and also discussed AI's huge demand for energy.

“By 2030, AI will consume more electricity than households.” Toby Rice, CEO of EQT, the largest US gas producer, quoted such a prediction in his speech.

$Amazon (AMZN.US)$Bill Vass, vice president of network services engineering, notes that the world adds a new data center every three days.

Bill Gates said that electricity is the key to determining whether data centers can be profitable, and the amount of electricity AI consumes is amazing. The use of AI will drive energy demand, and in the next few years, the development of AI may be constrained by chip design and power supply.

This is not groundless — as new data centers are being built faster than new power plants, the gap between supply and demand has begun to appear. American Commercial Real Estate Services, Inc.$CBRE Group (CBRE.US)$(CBRE Group, Inc.) revealed that due to delays in electricity supply, the construction period of the data center has been extended by 2 to 6 years.

“Energy behemoth”

AI's title of “energy behemoth” is not for nothing.

OpenAI's Sam Altman once “complained” about AI's energy needs, especially electricity. At the Davos Forum at the beginning of the year, he said that the development of AI requires breakthroughs in energy, and AI will bring electricity demand far exceeding expectations.

According to some data, ChatGPT consumes more than 500,000 kilowatt-hours of electricity every day to process about 200 million user requests, which is more than 17,000 times the daily electricity consumption of American households; as for search giant Google, if it calls AIGC in every user search, the annual electricity consumption will increase to about 29 billion kilowatt-hours, which is even higher than the annual electricity consumption of countries such as Kenya and Guatemala.

Looking back at 2022, when AI had yet to set off such a boom, data centers in China and the US accounted for 3% and 4% of the total electricity consumption of their respective societies, respectively.

As the scale of global computing power gradually grows, Huatai Securities's March 24 research report predicts that by 2030, the total electricity consumption of data centers in China and the US will reach 0.95/0.65 trillion kilowatts and 1.7/1.2 trillion kilowatts of electricity, respectively, which is more than 3.5 times and 6 times that of 2022. Under an optimistic scenario, the amount of electricity used by AI in China and the US in 2030 will reach 20%/31% of the electricity consumption of the entire society in 2022.

The analyst further pointed out that since data centers are unevenly distributed, regional power shortages will be the first (such as Virginia in the US). contemplates$American Electric Power (AEP.US)$There has been almost no growth in history, so AI will be an important driver for electricity to return to positive growth in developed overseas regions.

Where is the increase in electricity coming from?

Shortage of electricity naturally requires “new electricity”, but where does “new electricity” come from? Under the global wave of carbon neutrality, clean energy represented by photovoltaics and wind power seems to be the first choice, but this is only an “ideal choice.”

“We can't build a new 100 gigawatt renewable energy (power plant) in a few years. It's a bit difficult to do.” Former US Secretary of Energy Ernest Moniz admits.

EQT CEO Toby Rice added that technology companies need sufficiently reliable electricity. Renewable energy such as wind energy and solar energy cannot do this. As for large-scale nuclear facilities (currently only one under construction in the US), construction has always been expensive and time-consuming. “Technology companies won't wait 7-10 years for this infrastructure; they will only use natural gas.”

The US gas giant executive said that technology companies that have built data centers have asked about buying natural gas from EQT, and Rice was asked, “How fast are your deliveries?” “How much gas can we get?”

US stocks are “no longer in a hidden corner”

First the “GPU shortage” and then the “power shortage,” it's hard to say that the development path of AI has been smooth sailing.

It is worth noting that US stock investors who want to seize the AI wave have already set their sights on this corner.

One of the largest electricity producers and retail energy suppliers in the US$Vistra Energy (VST.US)$America's largest energy company$Constellation Energy (CEG.US)$, the largest green power company in the United States$NRG Energy (NRG.US)$Stock prices have all more than doubled in the past year, and they all hit record highs this week.

Judging from the range of ups and downs in the past year and this year, although the performance of these three companies is not as good as “the strongest stock on the surface”$NVIDIA (NVDA.US)$However, it also left Microsoft, the “company behind” OpenAI.

edit/new