Even when you think you're not buying Nvidia, you might still be buying Nvidia
The proliferation of custom AI ASICs, or XPUs, from OpenAI, Meta, Microsoft, and others has led many to question Nvidia's grip on the market. After all, if everybody's building their own, who needs the GPUs the AI arms dealer has made its fortune on?
But Nvidia isn't concerned in the least and has instead invited its competition to raid – or, rather, license – the GPU giant's IP holdings, particularly those related to networking.
On Monday, MediaTek became the latest to embrace Nvidia's NVLink Fusion high-speed interconnect technology for its fledgling datacenter XPU offering. In exchange, Nvidia has invested $3.5 billion in convertible bonds issued by MediaTek. The GPU giant will also continue to license the Taiwanese SoC provider's own designs for use in future DGX and RTX Spark systems.
If you're curious, we explored MediaTek's datacenter ambitions last month. But the designer, best known for its Arm-based smartphone and tablet processors, is only one of several high-profile chip designers that plan to integrate the tech. Amazon, Fujitsu, Qualcomm, Arm, and Marvell are all on the same list.
NVLink Fusion started as a commercialized version of Nvidia's high-speed inter-GPU interconnect, introduced early last year. Initially, it was offered in two varieties: a chip-to-chip (C2C) variant for connecting CPUs to GPUs (or XPUs) in a memory coherent fabric, and a switched fabric used to stitch together multiple accelerators into one big logical rack-scale chip. It's since expanded to become the blanket offering by which Nvidia licenses its semiconductor IP.
Given how long it's taken Broadcom and others to develop competitive alternatives to NVLink, it's not surprising companies like MediaTek would license the tech, rather than reinventing the wheel, or piping alternative interconnects, like UALink, over regular old Ethernet.
By licensing Nvidia's NVLink Fusion tech, companies can focus on building competitive accelerators without worrying about how they're going to scale in production.
And because Nvidia's MGX rack designs are part of the Open Compute Project, MediaTek and its partners not only benefit from Nvidia's scale-up networking tech, but also can essentially take existing NVL72 racks and slot in their compute blades. This should dramatically reduce the system design and mechanical engineering experience required to go from silicon to AI racks.
And this isn't theoretical. Amazon is doing just that with its Trainium series of AI accelerators. As we wrote at the time, Amazon used Nvidia's MGX NVL72 reference design for its Trainium3-based rack systems launched last year. Meanwhile Trainium4, expected late this year, will ditch Amazon's in-house NeuronLink interconnect tech for NVLink Fusion.
So, what does Nvidia get from letting rival chip designers piggyback off its hard-won networking tech?
A lot more than licensing fees: It's also a way to keep companies hooked on its other products.
When NVLink Fusion was first announced, customers had two options. They could pair their XPUs with Nvidia's CPUs or their CPUs with Nvidia's GPUs. Customers also were on the hook for NVSwitches if they wanted to take advantage of the tech's scale up networking capabilities.
So, on top of licensing the tech, Nvidia had the potential to sell roughly two GPUs per partner CPU, a Grace or Vera CPU for every two partner XPUs, and up to nine NVSwitches for every 72 or so accelerators, and that's just the rack.
If you're already buying NVLink, why not use InfiniBand or Nvidia's Spectrum-X Ethernet kit to switch those racks together to train and serve even larger models at scale?
