At Supercomputing 2024, NVIDIA introduced the H200 NVL, a new GPU based on its Hopper architecture. The chip is tailored for data centers with air-cooled rack designs, delivering top-tier performance for artificial intelligence (AI) and high-performance computing (HPC) while prioritizing energy efficiency.
NVIDIA’s move comes in response to growing market demand. According to company insights, around 70% of data centers operate GPU racks under 20 kW, relying on air cooling. With its PCIe connection, the H200 NVL enables multiple GPUs to be used within a single rack, making it a flexible solution for these setups.
H200 NVL has great evolution over Ampere architecture
The NVIDIA H200 NVL GPU offers significant advancements over its predecessor, the H100 NVL. With 50% more memory and 20% more bandwidth, the H200 NVL delivers up to 70% better performance for large language models (LLMs) and 30% higher performance in HPC tasks. Compared to the NVIDIA Ampere architecture, it provides up to 2.5 times the performance.
Thanks to NVLink technology, up to four H200 NVL GPUs can be interconnected. The new generation of this connection enables communication speeds up to seven times faster than PCIe Gen 5, making it well-suited for the high demands of LLM inference in data centers.
The H200 NVL is already making an impact in academia. The University of New Mexico is upgrading its systems to this new GPU. “By adopting the H200 NVL, we can accelerate various applications, including data science projects, bioinformatics, genomics research, physics simulations, astronomy studies, climate modeling, and more,” said Patrick Bridges, the university’s director of research in advanced computing.
Significant companies like Dell, HP, Lenovo, and Supermicro are also preparing to integrate the H200 NVL into their offerings. The GPU is set to launch globally in December 2024.
