StepFun brings Attention-FFN disaggregation to vLLM
StepFun says it is open-sourcing its Attention-FFN disaggregation work with the vLLM team, Ant Group and FastAFD, pushing a key MoE-serving efficiency technique into the community.
Splitting the model to serve it faster
StepFun (阶跃星辰) said it is collaborating with the vLLM project, Ant Group and the FastAFD team to bring Attention-FFN disaggregation (AFD) to the open-source community. AFD is an inference technique that separates the two dominant phases of a large mixture-of-experts model onto different hardware pools: the attention phase, which is memory-bound, and the FFN/expert phase, which is compute-bound. Because a single homogeneous deployment cannot optimize for both at once, splitting them lets each scale independently and keeps expensive accelerators busier.
The approach was pioneered in production by Chinese labs — StepFun, ByteDance and Huawei among them — and underpins the efficiency of StepFun's Step-3, a 321B-parameter MoE model with 38B active that co-designs Multi-Matrix Factorization Attention with AFD. StepFun previously released StepMesh, a communication library for AFD, and its vLLM-based implementation has reported decoding throughput of up to roughly 4,000 tokens per second per GPU under a 50ms latency target on H800 hardware.
A broadening open-source effort
The collaboration dovetails with independent work such as FastAFD from UCSD's Hao AI Lab, which reported 1.35–1.45x higher per-GPU steady-state decode throughput than a tuned colocated vLLM baseline on Qwen3-235B and MiniMax-M2.5 running on GB200 NVL72 hardware. Together, the efforts move AFD from a set of proprietary in-house systems toward a shared capability in the most widely used open inference engine.
Why it matters
Serving cost, not just raw capability, increasingly decides which models get deployed at scale — a dynamic underscored by the industry's fixation on token efficiency. AFD attacks that cost directly by wringing more throughput from the same GPUs, and it is especially valuable for the large MoE architectures now common among Chinese frontier labs. Folding the technique into vLLM lowers the barrier for any team to run big MoE models efficiently, reinforcing the open ecosystem's role as the place where practical serving advances are shared rather than hoarded.