RESEARCH POSTER AWARD FINALIST: NIMBLE: Node-Interconnect Multi-Path Balancing with On-

In this Research Poster Award finalist presentation, introduced by Tanzima Islam (Texas State University), Professor DK Panda of The Ohio State University, director of the ICICLE NSF AI Institute, presents NIMBLE, a node-interconnect multi-path balancing runtime with on-the-fly orchestration for high-bandwidth GPU clusters. The work, primarily conducted by his student, addresses the observation that GPU utilization on large-scale AI training and inference systems is often low despite dense NVLink-connected nodes and communication layers such as NCCL, MPI, and IB GDR-Sync. Using mixture-of-experts (MoE) inference as a motivating example, Panda shows that statically assigning experts to GPUs and selecting static routing paths creates link hotspots, where some links saturate while others remain underutilized, causing skew and degraded performance. NIMBLE follows an observe, optimize, and execute approach through three phases (monitor, route, and execute) implemented in a communication runtime containing a monitoring module, orchestration engine, and kernel scatter and buffer pipeline. The design is transparent, requiring no changes to upper software layers, and runs on existing InfiniBand and NVLink hardware. Benchmark results on all-to-all-v collectives show hotspot ratios from roughly 40 to 80 percent, with NIMBLE achieving about 1.23x speedup over OpenMPI and 2.79x over NCCL. For end-to-end LLM MoE inference, NIMBLE delivers up to 1.35x peak speedup and 1.13x to 1.26x across hotspot ratios, with strong benefits at token rates above 16K, and overall reported performance gains ranging from 2.3x to 5.2x.