Samsung CXL, a global first, unveils 3D DRAM roadmap

According to reports, Samsung has launched a new Compute Express Link (CXL) add-in card called CXL Memory Module-Hybrid for Tiered Memory (CMM-H TM), which adds additional RAM and flash memory that can be remotely accessed by CPUs and accelerators. This expansion card combines high-speed DRAM and NAND flash memory, aiming to provide an economically efficient way to increase server memory capacity without the need for locally installed DDR5 memory, which is often impractical in oversubscribed servers.

Samsung's solution operates on Compute Express Link (CXL), an open industry standard that provides cache coherency interconnect between CPUs and accelerators, allowing CPUs to access the same memory region as the connected CXL devices. Remote memory (or in this case, hybrid RAM/flash memory devices) can be accessed via the PCIe bus, at a cost of approximately 170-250 nanoseconds latency, or roughly the cost of a NUMA hop.

CXL was introduced in 2019 and is currently in its third version, supporting PCIe 6.0.

The CXL specification supports three types of devices: Type 1 devices are accelerators lacking local memory, Type 2 devices are accelerators with their own memory (such as GPUs with DDR or HBM, FPGAs, and ASICs), and Type 3 devices consist of memory devices. Samsung's device falls under the Type 3 category.

CMM-H TM is a branch of Samsung's CMM-H CXL memory solution. Samsung states that it is the world's first FPGA-based tiered CXL memory solution, designed to "address memory management challenges, reduce downtime, optimize tiered memory scheduling, and maximize performance while significantly reducing total ownership costs."

This new CMM-H is not as fast as DRAM; however, it increases powerful capacity through flash memory, while cleverly hiding a significant amount of latency through the built-in memory caching feature on the expansion card. Hot data is moved to the card's DRAM chips to improve speed, while less frequently used data is stored in NAND storage. Samsung states that this behavior occurs automatically, but certain applications and workloads can issue performance-enhancing hints to the device via APIs. Of course, this introduces some latency in caching data, which may not be suitable for all use cases, especially those relying on strict 99% performance.

Samsung's new expansion card will provide customers with a new way to expand server memory capacity. As more advanced large language models continue to demand more memory from their hosts and accelerators, this new design paradigm becomes increasingly important

Samsung Announces 3D DRAM Plan

Samsung Electronics, the world's largest memory chip manufacturer, plans to introduce a game-changer in the artificial intelligence industry, 3D DRAM, by 2025, challenging the current global artificial intelligence semiconductor market dominated by smaller competitor SK Hynix.

3D-based DRAM chips, which use vertical interconnect units instead of horizontally placing them like current chips, can triple the capacity per unit area. In contrast, High Bandwidth Memory (HBM) vertically interconnects multiple DRAM chips. According to industry insiders in Seoul, Samsung unveiled its 3D DRAM development roadmap at the Memcon 2024 global chip manufacturers' gathering held in San Jose, California last month.

The South Korea-based giant plans to introduce an early version of 3D DRAM based on vertical channel transistor technology in 2025, where channels for electron flow are vertically set in the transistors that make up the unit, serving as switches. The company also plans to launch a stacked DRAM in 2030, where all units are stacked together internally.

Currently, DRAM contains as many as 62 billion units on a motherboard, densely integrated on a flat plane, leading to unavoidable leakage and interference. With 3D DRAM, as transistors can accommodate more units on the same plane, it is expected to increase the capacity per chip.

The basic capacity of 3D DRAM is 100 GB, nearly three times the maximum capacity of currently available DRAM at 36 GB. It is speculated that by 2030, the global 3D DRAM market could grow to $100 billion. As the market is still in its early stages, this technology is expected to help Samsung challenge the throne of the global AI semiconductor industry, currently dominated by SK Hynix in the AI chip sector, where they hold a 90% share in the HBM and DRAM global markets," industry insiders stated.

While Samsung's competitors (including SK Hynix and Micron Technology) have been researching this technology, they have not yet disclosed any 3D DRAM roadmap. SK Hynix has presented the concept of its 3D DRAM at various industry conferences. Micron began developing 3D DRAM in 2019 and holds about 30 patents related to this technology, the most among the three major chip manufacturers.

For over a decade, as electronic devices equipped with DRAM such as smartphones have become smaller and more functional, the global DRAM industry has been developing smaller chips with greater data processing capabilities. The rapid development of artificial intelligence requires fast and large-scale data processing, a trend that is intensifying. 3D DRAM is expected to meet the demand for such chips as it is smaller and has a larger capacity than existing DRAM chips In the short term, new types of semiconductors may be used in small information technology devices such as smartphones and laptops, which require high-performance DRAM to implement AI functions on the devices. The automotive industry is expected to use 3D DRAM for the long term, as electric vehicles and autonomous driving vehicles need to process the big data collected from the roads in real time.

Samsung is developing leading technology in the 3D DRAM field, aiming to shrink its key size to 8-9 nanometers (nm) by 2027 to 2028. The latest DRAM is expected to be around 12 nm. The company is also actively expanding its team of 3D DRAM research and development personnel and has established a next-generation process development team for this technology in its semiconductor research center.

Source: Semiconductor Industry Observation, Original Title: "Samsung CXL Global First, 3D DRAM Roadmap Revealed"