[Hardware] NVIDIA GPU-N: 5nm, 8,576 Cores and Performance, Behind AMD?

[Dark]

The AMD Instinct has destroyed everything seen so far and has been a blow to the table so brutal that overshadowing as a term when referring to it can be an understatement. It is true that it is a GPU intended for DL, but it is a sketch on the other hand of what awaits us with RDNA 3 (in theory at least and saving distances). NVIDIA's answer has seen the light today in the form of a graphics card mysteriously called GPU-N, which is extremely peculiar and would be behind AMD but ...

Many months (April) have passed since we talked about the so-called NVIDIA COPA-GPU Exascale, which is nothing other than the name that Huang's give to the so-called Composable On Package, a 2.5DIC or 3D integrated circuit (depending on the design in particular, nothing new needs to be added) that would come with the Hooper architecture.

CUP-GPU

With this in mind, what could be the new GH100 chip based on this new architecture dedicated to DL and AI has been leaked, especially the first one, where as we already know, low-precision matrix mathematical performance prevails, or in other words , FP16. This chip has been specified with GPU-N and the seen certainly does not disappoint if we do not forget its end.

Two designs, two goals, one architecture
NVIDIA-Hopper

Just to review what we saw at the beginning of the year, what NVIDIA intends is to launch two different architectures focused on each purpose:

Gaming -> Ada Lovelace
DL and IA -> Hooper (MCM and monodie)
But within the latter there will be two different GPU designs that will be COPA and that differ in their capabilities and approaches. This supposed GPU-N would be focused on DL and therefore it would be the natural substitute for the current NVIDIA A100, which is important to understand the data that we are going to see now, since it is not an MCM GPU as such, but one of the modules exclusively, so it is understood that there will be multi die and mono die graphics cards.

Why is GPU-N behind AMD?
The mysterious GPU-N

Note: All benchmarks are run in NVArchSim, simulating GPU-N
High Accuracy for MLPerf Workloads

Comparison with other DL ASICs

(7 / x) pic.twitter.com/mxJExyILbM

- Redfire (@ Redfire75369) December 14, 2021

This graphics card is mono die, so from the outset it will be slower than the MI250X, but also its objective is not the AI as such, where for this if we would find MCM designs, this is different, it is one more market specific.

GPU-N will have 134 SM, which if the current structure in NVIDIA Shaders with Ampere is maintained, it would give us no less than 8,576 Cores, or what is the same, + 24% compared to its predecessor. Given these data, it is logically speculated with the 5 nm of TSMC, but the frequency would instead remain at 1.4 GHz, curious to say the least. What will change is its L2 that will be increased to 60 MB (+ 50%) which, together with the increase to 100 GB of HBM2E that it will have, would give us a bandwidth (at the same frequency, it is understood) of 2.68 TB / s.

This is due to the 6144-bit bandwidth, which could mean sizes of more than twice the memory capacity. The problem with all this is that the comparison with the MI250X is not really fair although they share certain similarities, since this GPU-N would get 24.2 TFLOPs in FP32, only 24% more than the GPU it replaces.

Instead, it achieves 2.5 times the performance in FP16 with 779 TFLOPs and here is the important thing for the sector where it is directed. The AMD Instinct MI250X achieves 95.7 TFLOPs in FP32 and 383 TFLOPs in FP16 (-2.15x).