China Advances AI Progress Without Relying on “Cutting-Edge” Accelerators; DeepSeek Unveils “FlashMLA” Project with 8x TFLOPS Boost Using NVIDIA H800 GPUs

It looks like China’s not waiting around to boost its AI hardware capabilities, especially with in-country companies like DeepSeek paving the way. In a crafty twist, they’re applying some clever software solutions to jazz up what they’ve got at hand. DeepSeek’s latest venture shows just how much punch they are packing into NVIDIA’s streamlined Hopper H800 GPUs by maximizing efficiency in memory usage and resource allocation during inference tasks.

DeepSeek is hosting what they call an “OpenSource” week, rolling out technologies and tools that anyone can access via GitHub. They kicked off with a bang by introducing FlashMLA, a decoding kernel specifically engineered for NVIDIA’s Hopper GPUs. If you’re wondering what’s so groundbreaking about it, let’s dive into what makes this particular innovation stand out. It seems that DeepSeek has managed to boost the Hopper H800 to an astounding 580 TFLOPS for BF16 matrix multiplication—which is roughly eight times more than the average industry benchmark. And that’s not all. FlashMLA enhances memory bandwidth to an astonishing 3000 GB/s, nearly doubling the theoretical peak of the H800. The cherry on top? All these feats are achieved without any additional hardware, purely through ingenious programming.

FlashMLA incorporates “low-rank key-value compression,” a technique that efficiently categorizes data into smaller segments. This method not only accelerates processing but also cuts down memory demands by 40% to 60%. Furthermore, it features a block-based paging system that smartly adjusts memory allocation based on task requirements, rather than sticking to a fixed value. This adaptability enables models to handle sequences of varying lengths with greater agility—resulting in notable performance boosts.

In the grand scheme of AI computing, DeepSeek’s advancement underscores that success hinges on more than just raw hardware power. It’s about utilizing every trick in the book, and FlashMLA is a testament to that philosophy. Although currently tailored specifically for Hopper GPUs, it piques curiosity about the potential heights we might achieve on the H100 models with similar innovations down the road.