NVIDIA’s flagship datacenter GPU, the Hopper H100, is featured in all its glory. (image credit: CNET)

At GTC 2022, NVIDIA unveiled its Hopper H100 GPU, a compute powerhouse designed for next-generation data centers. It’s been a while since we talked about this powerful chip, but it looks like NVIDIA has given select media a close-up of its flagship chip.

NVIDIA Hopper H100 GPU: First delivers high-resolution images with 4nm and HBM3 technology

CNET managed to capture not only the graphics board to which the H100 GPU is attached, but also the H100 chip. The H100 GPU is a monster chip loaded with the latest 4nm technology and includes 80 billion transistors with bleeding-edge HBM3 memory technology. According to Tech Outlet, the H100 is built on a PG520 PCB board with over 30 power VRMs and a massive integral interposer that uses TSMC’s CoWoS technology to combine the H100 GPU with a 6-stack HBM3 design.

Next-gen NVIDIA GeForce RTX 4090 with top AD102 GPU could be the first gaming graphics card to break past 100 TFLOP

NVIDIA Hopper H100 GPU images (Image credit: CNET):

Two of the six piles are kept to ensure yield integrity. But the new HBM3 standard allows up to 80Gb of capacity at 3Tb/s speeds which are insane. For comparison, the current fastest gaming graphics card, the RTX 3090 Ti, only offers 1 TB/s bandwidth and 24 GB of VRAM capacity. In addition, the H100 Hopper GPU also packs in the latest FP8 data format, and through its new SXM connection, it helps accommodate the 700W power design the chip is designed around.

NVIDIA Hopper H100 GPU Specifications at a Glance

So as per the specifications, the NVIDIA Hopper GH100 GPU is made up of a massive 144 SM (Streaming Multiprocessor) chip layout that is clocked in a total of 8 GPC. These GPCs are a total of 9 TPCs which is further made up of 2 SM units. This gives us 18 SMs per GPC and 8 on the whole 144 GPC configuration. Each SM is made up of 128 FP32 units which should give us a total of 18,432 CUDA cores. The following are some of the configurations you can expect from the H100 chip:

The full implementation of the GH100 GPU consists of the following units:

Intel CEO Pat Gelsinger expects end of chip shortage by 2024

• 8 GPC, 72 TPC (9 TPC/GPC), 2 SM/TPC, 144 SM per full GPU
• 128 FP32 CUDA cores per SM, 18432 FP32 CUDA cores per full GPU
• 4 4th generation Tensor cores per SM, 576 per full GPU
• 6 HBM3 or HBM2e stacks, 12 512-bit memory controllers
• 60 MB L2 Cache
• Fourth Generation NVLink and PCIe Gen 5

The NVIDIA H100 GPU with SXM5 board form-factor consists of the following units:

• 8 GPC, 66 TPC, 2 SM/TPC, 132 SM per GPU
• 128 FP32 CUDA Cores per SM, 16896 FP32 CUDA Cores per GPU
• 4 4th generation Tensor cores per SM, 528 per GPU
• 80 GB HBM3, 5 HBM3 stacks, 10 512-bit memory controllers
• 50 MB L2 Cache
• Fourth Generation NVLink and PCIe Gen 5

This is an increase of 2.25x over the full GA100 GPU configuration. NVIDIA is also benefiting from more FP64, FP16 and Tensor cores within its Hopper GPUs which will greatly increase performance. And it’s going to be a necessity to rival Intel’s Ponte Vecchio which is also expected to feature 1:1 FP64.

Cache is another place where NVIDIA has paid a lot of attention, bumping it up to 48MB in the Hopper GH100 GPU. That’s 50MB of cache featured on the Ampere GA100 GPU and 3 times the size of AMD’s flagship Aldebaran MCM GPU, the Mi250X.

Completing the performance figures, NVIDIA’s GH100 Hopper GPU will offer 4000 TFLOP of FP8, 2000 TFLOP of FP16, 1000 TFLOP of TF32 and 60 TFLOP of FP64 compute performance. These record-breaking figures supersede all other HPC accelerators that have come before it. For comparison, it’s 3.3x faster than NVIDIA’s own A100 GPU and 28% faster than AMD’s Instinct MI250X in FP64 counts. In FP16 compute, the H100 GPU is 3x faster than the A100 and 5.2x faster than the MI250X, which is really bonkers.

The PCIe variant which is a cut-down model was recently listed in Japan for over US$30,000, so one can imagine that the SXM variant with the beefier configuration would cost around $50 grand.

NVIDIA Ampere GA100 GPU Based Tesla A100 Specs:

Source