Huawei

In this Vendor Showdown session, Zhang Haonan of the Huawei Computing Research Team presents work on low-bitwidth data formats for AI training and inference on Huawei Ascend NPUs. He outlines the roadmap for the Ascend NPU family: the 950 series entering the market this year supporting formats including half-float-eight, FP8, MXFP8, and MXFP4, and the 960 series arriving next year adding a half-float-four format. Both series allocate computational power across 16-bit, 8-bit, and 4-bit formats in a one-to-two-to-four ratio, aggressively scaling throughput as bitwidth decreases to support mixed-precision pipelines while retaining the flexibility to revert to 16-bit when needed. He details data-format conversion throughput, with BF16-to-MXFP8/MXFP4 conversion achieving 32 elements per cycle. The talk focuses on the half-float-eight format, explained through four aspects: a variable-length, prefix-encoded dot field that implicitly represents exponent bitwidth and a denormal indicator; an exponent field with an implicit fixed leading magnitude bit to avoid redundancy; sign-magnitude encoding of the exponent to prevent overlap between exponent ranges; and a denormal mode that extends exponent values to approach the range of 16-bit formats. Validation results indicate reliable accuracy and performance for large-model inference and training. In the Q&A, the speaker clarifies market positioning, with the 950PR chip targeting transformer prefill and recommendation scenarios and the 950DT chip targeting transformer decoding and training, while the 960 series serves both, and notes the denormal mechanism could in principle extend 32- or 64-bit encodings.