Advanced Memory Architecture

This panel on advanced memory architectures, moderated by Estela Suarez of the Juelich Supercomputing Centre, brings together John Shalf (Lawrence Berkeley National Laboratory), Nicolas Dube (Intel), Abu Sebastian (IBM Research Zurich), and Dietmar Fey (FAU Erlangen-Nuernberg). The discussion opens with skepticism about repeated claims of breaking the von Neumann bottleneck, contrasted with dataflow architectures such as Cerebras that trade universality for high token throughput. The panelists debate whether NUMA domains should be exposed to programmers, arguing that data locality cannot be hidden and that richer programming abstractions are needed to express data topology, with references to PGAS languages, Chapel, HPF, and HDF5-style layout descriptions. On HBM for CPUs, they weigh capacity versus bandwidth trade-offs, latency hiding via Little's law, the chip shoreline limitation on memory channels, LPDDR as a middle point, and 3D stacking or hybrid wafer bonding as future options, citing A64FX and a new Chinese CPU-based top system achieving roughly 80 percent LINPACK efficiency. Further topics include memory disaggregation driven by KV-cache tiering for AI inference, co-packaged optics and wavelength division multiplexing for bandwidth, the economics that doomed storage class memory such as 3D XPoint against 3D NAND scaling, and processing-in-memory on HBM logic dies. The panel closes noting that floating point units are nearly free while memory dominates cost and energy, so system design should start from memory bandwidth and capacity requirements.