Nvidia has dominated the GPU market for a long time, leaving competitors like Intel and AMD in the dust. But lately, the company has shifted its focus to AI factories.
So, what’s an AI factory? Think of it as an advanced data center that runs the whole AI lifecycle. It handles everything from data intake to training models and final outputs. You need a ton of storage, networking, and computing power to make it work.
Nvidia’s CEO, Jensen Huang, is a big advocate for the AI factory model, seeing it as crucial to the company’s AI strategy. In 2019, Nvidia made a significant move by acquiring Mellanox, a company that specializes in high-performance computing networking.
Kevin Deierling, senior vice president of networking at Nvidia, puts it simply: “High-performance computing and AI are nearly the same thing. We’re making lots of computers act as one.” He emphasizes that today’s computers aren’t just boxes; they’re entire data centers filled with GPU resources. Nvidia’s approach is to cram more GPUs into a smaller space to enhance performance.
For businesses looking to harness AI factories, scaling and fine-tuning AI models is essential. Deierling explains, “When you fine-tune a model or use techniques like retrieval-augmented generation, you’re leveraging the AI factory’s capability to build at scale.”
But it’s not all smooth sailing. AI factories demand a lot of energy. To tackle this, Nvidia is placing GPUs closer together and using liquid cooling systems to manage heat. Even with these strategies, they’re focused on maintaining performance without compromising sustainability. “We have to keep driving higher performance per watt to make this sustainable,” Deierling notes.
Other players, like Dell, are also stepping into the AI factory arena, making this an interesting landscape to watch.