NVIDIA to reveal details on next-gen Hopper GPU & Grace CPU at Hot Chips 34
Hot Chips is an annual event that brings system and processor architects and allows for companies to discuss details, such as technical details or the current performance of their products. NVIDIA is planning to discuss the company’s first server-based processer and the new Hopper graphics card. The NVSwitch interconnects the chip and the company’s Jetson Orin system on a module or SoM. The four presentations during the two-day event will offer an insider view of how the company’s platform will achieve increased performance, efficiency, scale, and security. NVIDIA hopes that it will be able to “demonstrate a design philosophy of innovating across the entire stack of chips, systems, and software where GPUs, CPUs, and DPUs act as peer processors.” So far, the company has already created a platform that operates AI, data analytics, and high-performance computing jobs inside cloud service providers, supercomputing centers, corporate data centers, and autonomous AI systems. Data centers demand flexible clusters of processors, graphics cards, and other accelerators transmitting massive pools of memory to produce the energy-efficient performance that today’s workloads require. Jonathon Evans, a distinguished engineer and 15-year veteran at NVIDIA, will describe the NVIDIA NVLink-C2C. It connects processors and graphics cards at 900 Gb/s with five times the energy efficiency of the existing PCIe Gen 5 standard, thanks to data transfers consuming 1.3 picojoules per bit. NVLink-C2C combines two processors to create the NVIDIA Grace CPU with 144 Arm Neoverse cores. It’s a CPU constructed to unravel the world’s most significant computing concerns. The Grace CPU uses LPDDR5X memory for maximum efficiency. The chip enables a terabyte per second of bandwidth in its memory while maintaining power consumption for the whole complex to 500 watts. The NVIDIA NVSwitch merges numerous servers into a single AI supercomputer using NVLink, interconnects running at 900 gigabytes per second, and above seven times the bandwidth of PCIe 5.0. Anyone can build custom chiplets using NVLink-C2C to coherently connect to NVIDIA GPUs, CPUs, DPUs, and SoCs, expanding this new class of integrated products. The interconnect will support AMBA CHI and CXL protocols used by Arm and x86 processors. NVSwitch lets users link 32 NVIDIA DGX H100 systems into an AI supercomputer that delivers an exaflop of peak AI performance. Alexander Ishii and Ryan Wells, two of NVIDIA’s veteran engineers, explain how the switch lets users build systems with up to 256 GPUs to tackle demanding workloads like training AI models with more than 1 trillion parameters. Jack Choquette, a distinguished senior engineer with 14 years at the company, will provide a detailed tour of the NVIDIA H100 Tensor Core GPU, aka Hopper. Using the new interconnects to scale to unparalleled heights fills many cutting-edge features that boost the accelerator’s performance, efficiency and protection. Among other new features:
Hopper adds virtualization support for multi-tenant, multi-user configurations. New DPX instructions speed recurring loops for fine mapping, DNA, and protein-analysis applications. Hopper packs support for enhanced security with confidential computing.
The NVIDIA Jetson AGX Origin integrates 12 Arm Cortex-A78 cores and an NVIDIA Ampere architecture GPU to deliver up to 275 trillion operations per second on AI inference jobs. Michael Ditty, an architecture manager with a 17-year tenure at the company, will provide new performance specs for NVIDIA Jetson AGX Orin, an edge AI, robotics, and advanced autonomous machines engine. News Source: NVIDIA All the new chips support the NVIDIA software stack that accelerates more than 700 applications and is used by 2.5 million developers. Based on the CUDA programming model, it includes dozens of NVIDIA SDKs for vertical markets like automotive (DRIVE) and healthcare (Clara), as well as technologies such as recommendation systems (Merlin) and conversational AI (Riva). The NVIDIA AI platform is available from every primary cloud service and system maker.
