The Narrow Margin: Leveraging Mixed Precision to Breach the Memory Wall

In this invited talk, chaired by Kathryn Mohror of Lawrence Livermore National Laboratory, Hartwig Anzt (Technical University of Munich, with a joint appointment at the University of Tennessee, Knoxville) examines how mixed precision can overcome the memory wall. He surveys the proliferation of low-precision formats driven by machine learning, including micro-scaling formats with block-level scaling factors, and notes that FP4 tensor operations on recent hardware can be thousands of times faster than double precision. Using the roofline model, he distinguishes compute-bound dense matrix multiplication, where tensor cores and emulation schemes such as the Ozaki methods apply, from memory-bound sparse linear algebra, where performance is limited by bandwidth and lower precision helps only by moving fewer bytes. His central technique stores the Krylov basis of a GMRES solver in reduced precision while performing all arithmetic in double precision, effectively a pointwise lossy compression implemented through a memory accessor. On SuiteSparse matrices this achieves double-precision accuracy with roughly 1.4x average speedup, and the approach was integrated into the MFEM finite element package. He also applies precision-adapted algebraic multigrid preconditioning, which worked well on a synthetic Laplace problem but showed no benefit in a real cardiac electrophysiology simulation from the MicroCARD project, illustrating that mixed precision is not a black-box method. Takeaways stress reducing data movement, favoring computation over communication, and developing strategies to determine where reduced precision is numerically safe.