baidu/ERNIE-5.5-Flash-Base

main boardmodel

ERNIE 5.5 Flash is an efficient sparse Mixture-of-Experts (MoE) language model with 280B total parameters and 14B activated per token, pre-trained from scratch on 33.5T tokens with a training context of up to 128K; fine-tuning and inference support a 1M-token context. Mixture-of-experts, 286B total, 14B active (512 experts, 10 per token). Context: 1M tokens in config.json; the model card says it was trained at 131,072 tokens. License: ernie-5.5-flash-base-model-license-v1.0. Vendor-reported: "At a comparable KV-cache budget, pairing HCA with MLA outperforms pairing it with Compressed Sparse Attention (CSA)." Vendor-reported: "Each MoE layer has 512 routed experts (top-10) plus 1 shared expert, operating in a 2048-d latent space so that expert dispatch/combine happens at half the hidden width."

Vendor-reported: At a comparable KV-cache budget, pairing HCA with MLA outperforms pairing it with Compressed Sparse Attention (CSA).
Vendor-reported: Each MoE layer has 512 routed experts (top-10) plus 1 shared expert, operating in a 2048-d latent space so that expert dispatch/combine happens at half the hidden width.

Org: baidu
Released: 2026-10-21
Status: main board
License: ernie-5.5-flash-base-model-license-v1.0 (restricted)
Parameters: 286.2B total, 14.0B active
Architecture: MoE · 512 experts, 10/token · Ernie5_5_MoeForCausalLM
Context: 1,048,576 tokens (config.json); trained at 131,072 tokens (model card)
Task: text-generation · text
Signal: 317 HF likes · 3914 downloads · 17 X likes · 3 X post(s)

Runs on

estimate from params; need = weights + FP16 KV cache for 8K tokens + 1GB. A sizing calculation, not tested.

Q4 ~173.1 GB (estimated size): 2× DGX Spark (256GB), Mac 256GB, CPU only, 256GB RAM (slow)
Q8 ~305.5 GB (estimated size): Mac 512GB
FP16 ~573.9 GB (estimated size): no listed device

baidu/ERNIE-5.5-Flash-Base

Runs on

Links