The massive growth in data centers is bringing waves of new people into the industry. There are people coming from other industries who don’t understand why things are done the way they are in the data center industry.
And they’re arriving just in time.
“The new folks are the ones that are going to save our butts,” said Sean James, Distinguished Engineer for Energy Systems at NVIDIA, speaking on a keynote panel April 22 at Data Center World. “We're excited about your new ideas on how to solve these problems.”
James spent more than 20 years at Microsoft data center and energy systems research before moving to NVIDIA earlier this year. During the keynote panel discussion, he offered up “a mission” to all those data center industry newcomers:
“I'm going to ask that when something doesn't make sense to you, speak up, because it probably doesn't make sense. We're just holding on to it. Folks like me being in the industry over two decades, we're just holding on to it. I think that there's a lot of lessons to be learned from other industries that have gone from small to mega.”
James offered the example of the auto industry, which learned how to mass produce vehicles using chassis platforms that still left lots of room for customization and model variations. Data center design is moving in that direction.
James was joined on the Data Center World keynote panel by Ram Nagappan, Vice President of AI Infrastructure at Oracle Cloud Infrastructure, and Varun Sakalkar, Distinguished Engineer at Google Datacenters. Here are three other nuggets from the panel:
Google is taking a “campus as a product” mindset to data center development: Google and everyone else in the industry used to put up a data center building and then fill it up over time with servers as the workloads grew. Now, it will bring an entire gigascale campus live at one time. “Within a three-month, six-month period, you would like the whole thing to be up and running and serving customers,” Sakalkar said. Even in that tight window, however, Google is going to make late-game decisions on how that space is used, whether to run AI workloads like Gemini, be used for Google Cloud GPUs that customers tap, or some other use case. With modular components and prefabricated systems, data center development is increasingly about integration of systems as well as more conventional construction. “So, modularity principles are extremely important so that you can maintain fungibility without losing velocity,” he said.
Data center design varies based on two different AI workload types: To support workloads to train AI models, the data center design might require more than 100,000 GPUs connected together as physically close together as possible, “so the campus needs to be very dense so that you can minimize the latency,” Nagappan explained. But a brief outage isn’t as severe with AI training, since the workload is running over a long period and can generally recover from the last checkpoint. With AI inference workloads, availability needs to be very high because it involves an individual person or agent requesting a single real-time task. “So, there is a nuance to it, and we take all of that into account while we are building our data center so that we can meet those requirements,” Nagappan said.
“Behind the meter” power is prevalent but not preferred: To get capacity online faster, data centers fairly often turn to running their own on-site power generation, be it a solar microgrid, gas turbines, fuel cells, or some other source. On-site power lets operators skip what can be a years-long queue to connect to the electrical grid. But James stressed that the industry and individual operators are better served if they can work with utilities to get those sources integrated. Behind the meter “should be temporary, and we should be trying to figure out how to get those power plants on the grid someday in the future,” he said.
Data Center World registered attendees can view the keynotes and other conference sessions on the DCW mobile app or web platform: https://datacenterworld.com/digital-event-resources/