Track Hyper | Micron GDDR7 Sample: Three giants open a new battlefield competition

Author: Zhou Yuan / Wall Street News

Recently, NVIDIA co-founder and CEO Huang Renxun updated the AI GPU technology roadmap at the Taipei Computex 2024 exhibition. At the same time, Huang Renxun clarified that Samsung Electronics' High Bandwidth Memory (HBM) did not fail NVIDIA's quality testing due to overheating issues. This means that Samsung Electronics' HBM3e (fifth generation) will soon enter NVIDIA's AI accelerator card supply chain.

Huang Renxun said, "We need a large quantity of HBM, so supply speed is crucial. We are collaborating with Samsung, SK Hynix, and Micron, and we will receive products from these three companies."

Earlier reports indicated that SK Hynix and Micron's HBM production capacity is scheduled until the end of 2025, indicating both the significant amount of HBM capacity needed by NVIDIA and the potential obstacle of HBM production capacity to NVIDIA's AI accelerator card shipments.

Looking at the technological evolution of HBM, it was originally intended as a replacement for GDDR. HBM technology iterates quickly, with increasing performance but also higher costs.

In fact, GDDR is no longer what it used to be: on March 6th this year, the Joint Electron Device Engineering Council (JEDEC) finalized the JESD239 GDDR7 graphics card technical specification standard. The performance of GDDR7 has seen a significant improvement over GDDR, with indications that some aspects of GDDR7 performance are crossing over from consumer-grade (To C) to high-performance computing (HPC) fields (To B). Therefore, GDDR7 is starting to show the capability to compensate for the high cost drawbacks of HBM.

New Battlefield: GDDR7 Arrives

On June 4th, Micron announced the availability of GDDR7 memory samples for the next generation of GPUs, with two speeds: 28GB/s and 32GB/s. The 32GB/s GDDR7 offers 60% higher memory bandwidth than GDDR6; on a 384-bit bus, the memory bandwidth can reach 1.5TB/s. GDDR6 on GPUs like the RTX 4090 has a maximum bandwidth of 1TB/s.

Micron's GDDR7 uses its 1β (1-beta) DRAM technology. In the latest roadmap, Micron revealed plans to reach 36GB and 24GB+ memory modules by 2026; Samsung's goal is to achieve a 32GB speed, but has not disclosed mid-term speed targets.

In March of this year, Samsung and SK Hynix also disclosed their respective GDDR7 technical parameters at the NVIDIA GTC conference. Now, the three major suppliers have opened up a new battlefield: GDDR7.

According to different application scenarios, the Joint Electron Device Engineering Council (JEDEC) classifies DRAM into standard DDR, LPDDR, and GDDR. DDR is mainly used in servers and desktop PCs, LPDDR is mainly used in mobile phones and laptops, and GDDR's main application field is in the image processing domain In the DARM industry landscape, Samsung, SK Hynix, and Micron are the three major players. The current mainstream competition in the DDR5/LPDDR5 market is mainly between these three companies. In addition, Taiwanese storage companies Macronix and Nanya Technology, as well as mainland Chinese storage company ChangXin Memory, are also focusing on this market.

What is GDDR?

GDDR, the English abbreviation for Graphics Double Data Rate: Double Graphics Data Rate. This is a type of synchronous double data rate dynamic random-access memory (SDRAM) designed for graphics processing units (GPUs).

In 1998, Samsung Electronics introduced the industry's first 16Mb GDDR memory chip. Since then, GPUs and CPUs have had their own independent technical paths and products. Initially, GPU graphics card memory and CPU memory could be shared. After 1998, GPUs began to have dedicated graphics memory - GDDR.

On March 6th, JEDEC released the JESD239 GDDR7 graphics card technical specification, aiming to provide higher bandwidth, higher data transfer rates, higher energy efficiency, and larger storage capacity to support the development of future high-performance computing applications.

Currently, the initial batch of GDDR7 graphics memory has a capacity of only 2GB (16Gb), consistent with GDDR6/6X. JEDEC stated that there will be 3GB, 4GB, 6GB, and even 8GB in the future. The first products to be equipped with GDDR7 are the NVIDIA RTX 50 series and AMD RX 8000 series.

According to JEDEC, JESD239 GDDR7 is the first JEDEC standard DRAM to use pulse amplitude modulation (PAM) interface for high-frequency operation.

The bandwidth of GDDR7 is twice that of GDDR6: 192GB/s. This is achieved by increasing the number of independent channels to 4. GDDR7 also supports densities from 16Gbit to 32Gbit, including support for dual-channel mode, which can double the system capacity, meeting the growing demand for high memory bandwidth in future graphics, gaming, computing, networking, and artificial intelligence applications.

Unlike HBM, which is typically used in supercomputing (high-performance computing: HPC) AI power clusters, GDDR is mainly used for gaming, professional graphic design/rendering, scientific and engineering simulations, virtual reality, visualization applications (consumer-level), and ML (machine learning) in AI edge computing.

Three Giants Competing in the Second Half of the Year

Micron's GDDR7 memory samples have a working voltage of 1.2V (compared to the previous generation GDDR6's working voltage of 1.35V), in line with JEDEC's GDDR7 technical specifications; the efficiency has increased by 50% compared to the previous generation. The main reason for achieving this is that Micron has adopted segmented voltage planes, partial device operation, and sleep modes.

In addition, Micron also uses FBGA with a thinner package height (1.1mm compared to 1.2mm) and high thermal conductivity EMC packaging, achieving 65% higher thermal resistance, providing more efficient heat management for desktops and laptops These features not only further enhance the performance of Micron GDDR7 in traditional GDDR application areas, but also have broad prospects in cutting-edge fields such as generative AI and HPC that require higher bandwidth. This means that GDDR7 has broken through the previous performance bottleneck and has begun to show performance comparable to low-end HBM.

This enables GDDR to break through its limitations: transitioning from consumer-grade GPU applications to enterprise-level high-performance markets.

According to official information from Micron, in the gaming field, Micron's GDDR7 samples are expected to increase frame rates by over 30%, especially in ray tracing and rasterization workloads.

In terms of generative AI, Micron GDDR7 provides over 1.5TB/s of high system bandwidth, expected to accelerate response times for generative AI text-to-image generation by 20%; in HPC, GDDR7 is expected to reduce processing times further, enabling seamless multitasking for complex workloads such as animation, 3D design, scientific simulations, and financial modeling.

Of course, while the performance improvement of GDDR7 is higher compared to GDDR6, it still cannot be compared to HBM3/3e, let alone HBM4. However, the overall performance improvement of GDDR7, compared to HBM2, may be on par.

Samsung Electronics also revealed some technical parameters of its GDDR7 at NVIDIA's GTC conference in March. For example, Samsung's GDDR7 can achieve a speed of 32 Gbps at only 1.1V DRAM voltage (a 33% increase compared to GDDR6X), exceeding the JEDEC's GDDR7 technology specification of 1.2V voltage standard. This performance is mainly achieved through the first application of PAM3 signaling.

At the same time, Samsung's GDDR7 can achieve a maximum memory speed of 192GB/s, with a single memory capacity of 2GB; through the application of Samsung's unique power management innovations, the energy efficiency of Samsung's GDDR7 has increased by 20%, heat dissipation reduced by 70%, and standby power consumption reduced by 50%.

As one of the three storage giants, SK Hynix naturally will not be absent. Hynix's GDDR7 has a maximum bandwidth of 160GB/s, which is twice that of its previous generation product (GDDR6 at 80GB/s), with a 40% improvement in power efficiency; memory density has increased by 1.5 times, further enhancing visual effects.

Currently, Hynix's 16Gb GDDR7 is expected to achieve mass production in the third and fourth quarters of this year, while Samsung is in the process of sampling; Micron's GDDR7 memory will be released in the second half of this year. Surprisingly, AMD, as revealed by Micron, will also enter the GDDR7 market competition