Tencent Opens Hy4 Preview With One-Million Context
Tencent has released Hy4 Preview’s open weights, pairing a 770-billion-parameter MoE design with a one-million-token context window.
Open weights and a larger architecture
Tencent has released the weights for Hy4 Preview, its new flagship mixture-of-experts language model, through Hugging Face, ModelScope, GitCode and CNB. The model contains 770 billion parameters but activates 49 billion for each token. It supports a one-million-token context window and ships in full-precision and FP8 variants.
Hy4 uses 256 routed experts per mixture layer, activating eight alongside a shared expert. Its attention system combines gated sparse attention with cross-layer index reuse, while a native multi-token-prediction layer supports speculative decoding. Tencent provides deployment instructions for vLLM and SGLang, although the model’s size still puts self-hosting beyond ordinary workstations.
Built around productivity agents
Tencent is positioning Hy4 for coding, office work, game development and scientific research. It says 163 internal specialists compared model outputs across 203 engineering tasks: Hy4 averaged 2.99 out of four, narrowly exceeding GLM-5.3 at 2.92 and Kimi K3 at 2.94. These are vendor-run results rather than independent evaluations.
The company also says Hy4 participated in optimizing its own training methods, data strategies, evaluations and low-level operators. Work on operator fusion and communications reportedly improved end-to-end inference throughput by 31.8% from Tencent’s baseline. The model card nevertheless identifies excessive reasoning and repeated self-verification as known weaknesses.
Hy4 is available through Tencent products, Tencent Cloud TokenHub and OpenRouter. Tencent lists API prices of $0.834 per million input tokens, $2.501 per million output tokens and $0.042 per million cached tokens.
Why it matters
Hy4 adds another genuinely large Chinese open-weight model to a rapidly tightening field. Its combination of long context, agent-oriented training and downloadable weights gives researchers and infrastructure providers something concrete to inspect and deploy. The central uncertainty is efficiency: 49 billion active parameters are economical relative to the full model, but operating a 770-billion-parameter checkpoint remains a substantial hardware commitment.