Baseten Agents Cut Qwen-Image Latency by 42 Percent
Baseten says an agentic system generated, tested and deployed production optimizations across image and language models.
From kernel benchmark to serving stack
Baseten has disclosed an agentic framework that profiles AI models, proposes computation-level changes, writes optimized GPU kernels and validates them inside its production inference engine. The company reports a 42.3% end-to-end latency reduction for Qwen-Image, a 15.2% reduction for FLUX.2 and a 5.5% throughput increase for MiniMax M3.
The system uses two connected tracks. A model-level agent studies complete execution traces and searches for redundant computation, memory traffic and operations that can be fused. A kernel-level process then generates and benchmarks multiple implementations for the bottlenecks it finds. Candidate changes must pass correctness checks, ablation tests and full-model performance measurements before deployment. Successful kernels are retained for reuse, while failed approaches and workload-specific constraints are recorded in a knowledge base for later runs.
Baseten tested the image-model work with SGLang on Nvidia B300 GPUs. For Qwen-Image, optimizations included absorbing bias operations into later fused steps, caching timestep-dependent modulation shared by the two classifier-free-guidance passes, and improving individual kernels. For FLUX.2, the framework combined normalization, rotary-position calculations, activation, quantization and residual operations that had previously required separate launches or intermediate memory transfers.
Why it matters
AI-written GPU kernels are no longer confined to synthetic contests in this implementation: they altered production execution paths and produced measured model-wide gains. That distinction matters because a kernel that wins an isolated benchmark can regress after integration, where tensor shapes, batching, CUDA graphs and surrounding operations change the result. The largest reported gain also landed on Qwen-Image rather than a toy workload, showing substantial remaining inefficiency in current generative-image stacks. The figures are Baseten’s own and cover selected models and hardware, so broad generalization remains premature. Even so, the work suggests inference providers could continuously specialize serving code for each model and traffic pattern, compressing an optimization cycle that currently depends on scarce kernel engineers.