Optimizing OpenFOAM for Systems with CPUs and GPUs and Unified Memory Architecture

Kumar Saurabh presents joint work by AMD and Keysight Technologies on optimizing OpenFOAM, the widely used C++ computational fluid dynamics code base, for systems combining CPUs and GPUs with a unified memory architecture. OpenFOAM has historically been optimized for distributed-memory CPU computing, and the community favors GPU porting approaches that require minimal code changes, both standard offloading and OpenMP-based offloading, to preserve maintainability. The work identifies key bottlenecks in GPU offloading of OpenFOAM kernels and applies a series of optimizations that together deliver roughly 10x speedup over a CPU-only baseline, a gain particularly valuable for design and inverse problems requiring many simulations. The talk focuses on one optimization addressing kernel launch latency: OpenFOAM workloads dispatch many small GPU kernels with low arithmetic intensity, so launch overhead dominates execution time. Using field-variable arithmetic from a k-omega SST turbulence model as an example, the authors show that a simple sequence of operations generated more than 1,000 tiny kernel launches; through expression-based fusion requiring only minimal source changes (adding expression annotations to variables), these are reduced to just three kernels, cutting the measured time from 34 milliseconds to about two milliseconds, roughly a 16x improvement for that computation. The presentation serves as a preview of a fuller ISC talk covering additional optimizations.