Rambus (RMBS) Rosenblatt 6th Annual Age of AI Technology Summit summary

AI-driven changes in server and memory architecture

• Recent AI advancements have driven a shift toward specialized silicon, with GPUs and CPUs collaborating more closely, especially for large language models (LLMs) that require both high bandwidth and high capacity memory systems.

• LLMs use a KV cache that grows during conversations, necessitating both HBM (high bandwidth memory) for speed and DDR memory for capacity, with CPUs orchestrating and storing larger portions of the cache.

• Demand for both HBM and DDR memory is rising in tandem, as longer and more complex AI interactions require greater memory density and bandwidth.

• Modular memory (DIMMs) allows for late-stage capacity decisions and easy upgrades, helping manage fluctuating DRAM prices and availability without sacrificing bandwidth.

• De-speccing (using lower-capacity modules) is a cost optimization strategy that maintains bandwidth by fully populating memory channels, ensuring high CPU utilization.

Market trends and technology adoption

• The ratio of CPUs to GPUs in AI servers is shifting, with more CPUs being used, though the optimal balance is still evolving.

• Adoption of Arm-based CPUs and new module types (like SOCAMM2) is increasing, with memory module solutions being adapted for various platforms, from PCs to supercomputers.

• CXL technology is gaining renewed interest due to DRAM shortages, but widespread adoption is still in early stages due to application and infrastructure challenges.

• Optical memory appliances offer higher bandwidth and longer-distance data transfer, but still require robust electrical interfaces and signal integrity solutions.

• MRDIMM technology promises to double bandwidth and capacity using existing infrastructure, with adoption expected as new server platforms from AMD and Intel launch.

Expansion into client and edge markets

• AI capabilities are expected to move from data centers to PCs and edge devices, driving demand for new memory module chipsets optimized for client platforms.

• Rambus is introducing LPCAMM2 and other client-focused modules, anticipating a gradual revenue impact as AI adoption in PCs becomes mainstream.

• Robotics and physical AI will require increased memory capacity and bandwidth, with ongoing coordination between edge devices and data centers.

• The evolution of memory modules for edge and client devices mirrors trends seen in data centers, with modularity and upgradability remaining key advantages.

• Power management and signal integrity are critical differentiators for memory module components, leveraging decades of experience and industry relationships.