The computing market has always been hungry for reminiscence bandwidth, significantly for high-performance functions in servers and data centers. Recently, the explosion in core counts per socket has also accentuated this want. Despite progress in DDR speeds, the bandwidth per core has sadly not seen a corresponding scaling.
The stakeholders within the business have been trying to handle this by constructing further expertise on the prime of present widely adopted reminiscence requirements. With DDR5, there are, at the moment, two applied sciences trying to extend the height bandwidth past the official speeds. In late 2022, SK Hynix launched MCR-DIMMs to work with particular Intel server platforms. However, JEDEC – the standards-setting physique – additionally developed specs for MR-DIMMs with an identical method. Each of them is constructed upon present DDR5 applied sciences by trying to mix several ranks to enhance peak bandwidth and latency.
How MR-DIMMs Work
The MR-DIMM normal is conceptually straightforward – several ranks of reminiscence modules work at average DDR5 speeds with an information buffer at the entrance. The buffer operates at 2x the pace on the host interface facet, permitting primarily double the switch charges. The challenges lie in having the ability to function the logic within the host reminiscence controller at an increased pace and protecting the ability of consumption/thermals in the examination.
The primary model of the JEDEC MR-DIMM typically specifies speeds of 8800 MT/s, with the subsequent technology at 12800 MT/s. JEDEC additionally has a transparent roadmap for this expertise, keeping it in sync with the enhancements within the DDR5 standard.
Micron MR-DIMMs – Bandwidth and Capability Performs
Micron and Intel have worked intently in the previous few quarters to convey their former first-generation MR-DIMM lineup to the market. Intel’s Xeon 6 Household with P-Cores (Granite Rapids) is the primary platform to convey MR-DIMM assist at 8800 MT/s on the host facet. Micron’s standard-sized MR-DIMMs (appropriate for 1U servers) and TFF (tall form-factor) MR-DIMMs (for 2U+ servers) have been certified to be used with the identical.
The advantages provided by MR-DIMMs are evident from the JEDEC specs, permitting elevated knowledge charges and system bandwidth, with enhancements in latency. On the capability facet, permitting added ranks on the modules has enabled Micron to supply a 256 GB capability level. It has to be noted that some distributors are additionally utilizing TSV (through-silicon vias) expertise to extend the per-package capability at normal DDR5 speeds. However, this provides further value and complexity, which can be largely absent within the MR-DIMM manufacturing course.
The tall form-factor (TFF) MR-DIMMs have a bigger floor space than the standard-sized ones. For a similar airflow configuration, this permits the DIMM to have a greater thermal profile. This provides advantages for energy efficiency as well by lowering the potential for thermal throttling.
Micron is launching a complete lineup of MR-DIMMs in both normal and tall form factors as we speak, with several DRAM densities and pace choices, as mentioned above.
MRDIMM Advantages – Intel Granite Rapids Will get an Efficiency Enhance
Micron and Intel recently hosted a media/analyst briefing to reveal the advantages of MR-DIMMs for Xeon 6 with P-Cores (Granite Rapids). Utilizing a 2P configuration with 96-core Xeon 6 processors, benchmarks for various workloads have been processed with each 8800 MT/s MR-DIMMs and 6400 MT/s RDIMMs. The chosen workloads are infamous for being restricted in efficiency by reminiscence bandwidth.
OpenFOAM is a widely used CFD workload that has advantages over MR-DIMMs. For a similar reminiscence capability, the 8800 MT/s MR-DIMM reveals a 1.31x speedup primarily based on increased expected bandwidth and IPC enhancements and decreased last-level cache miss latency.
The efficiency advantages are significantly evident with extra cores collaborating on the workload.
Apache Spark is a generally used big-data platform working on giant datasets. Relying on the precise dataset within the image, the efficiency advantages of MR-DIMMs can vary. Micron and Intel used a 2.4TB set from Intel’s Hibench benchmark suite for this benchmark, displaying a 1.2x speedup on the identical capability and a 1.7x speedup with doubled-capacity TFF MR-DIMMs.
Avoiding the need to push knowledge again to everlasting storage also contributes to speedup.
The upper pace provided by MR-DIMMs additionally helps in AI inferencing workloads. Micron and Intel display a 1.31x inference efficiency enhancement and diminished time to the first token for a Llama 3 8B parameter mannequin. Purpose-built inferencing options primarily based on accelerators will carry out higher. Nevertheless, this indicated the CPU workloads that may profit from MR-DIMMs.
Because the adage goes, there is no such thing as a free lunch. At 8800 MT/s, MR-DIMMs are positively going to guzzle extra energy in comparison with 6400 MT/s RDIMMs. Nevertheless, the sooner completion of workloads implies that the vitality consumption for a given workload will likely decrease for the MR-DIMM configurations. We might have favored Micron and Intel to quantify this facet for the benchmarks offered within the demonstration. Moreover, Micron indicated that the vitality effectivity (when it comes to pico-joules per bit transferred) is related for each of the 6400 MT/s RDIMMs and 8800 MT/s MR-DIMMs.
Key Takeaways
The standardization of MR-DIMMs by JEDEC permits several business stakeholders to participate. Prospects are usually not vendor-locked and might evaluate and distinguish choices from completely different distributors to decide on one of the best matches for their needs.
At Computex, we noticed MR-DIMMs from ADATA on show. As a Tier-2 vendor without its personal DRAM fab, ADATA’s play is on value advantages, with the potential for the DRAM die to be sourced from entirely different fabs. JEDEC specs dictate the MR-DIMM board structure, and this permits Tier-2 distributors to have their very own play with pricing flexibility. Modules are additionally constructed primarily based on buyer orders. Micron, alternatively, has an extra complete portfolio/lineup of SKUs for various use cases with the professionals and cons of vertical integration within the image.
Micron cannot be the primary one announcing MR-DIMM sampling publicly. In the final month, Samsung introduced its lineup (primarily based on 16Gb DRAM dies). It has to be famous that Micron’s MR-DIMM portfolio uses 16 Gb, 24 Gb, and 32 Gb dies fabricated in 1β expertise. Whereas Samsung’s course for the 16 Gb dies used by their MR-DIMMs is not recognized, Micron believes that their MR-DIMM expertise will present higher energy effectivity than the competitors while additionally providing clients a wider variety of capacities and configurations.