baidu/ERNIE-5.5-Flash-Base-BF16
main boardmodel
ERNIE 5.5 Flash is an efficient sparse Mixture-of-Experts (MoE) language model with 280B total parameters and 14B activated per token, pre-trained from scratch on 33.5T tokens with a training context of up to 128K; fine-tuning and inference support a 1M-token context. Mixture-of-experts, 286B total, 14B active (512 experts, 10 per token). Context: 1M tokens in config.json; the model card says it was trained at 131,072 tokens. License: ernie-5.5-flash-base-model-license-v1.0. Vendor-reported: "At a comparable KV-cache budget, pairing HCA with MLA outperforms pairing it with Compressed Sparse Attention (CSA)." Vendor-reported: "Each MoE layer has 512 routed experts (top-10) plus 1 shared expert, operating in a 2048-d latent space so that expert dispatch/combine happens at half the hidden width."
- Vendor-reported: At a comparable KV-cache budget, pairing HCA with MLA outperforms pairing it with Compressed Sparse Attention (CSA).
- Vendor-reported: Each MoE layer has 512 routed experts (top-10) plus 1 shared expert, operating in a 2048-d latent space so that expert dispatch/combine happens at half the hidden width.
- Org
- baidu
- Released
- 2026-10-21
- Status
- main board
- License
- ernie-5.5-flash-base-model-license-v1.0 (restricted)
- Parameters
- 286.2B total, 14.0B active
- Architecture
- MoE · 512 experts, 10/token · Ernie5_5_MoeForCausalLM
- Context
- 1,048,576 tokens (config.json); trained at 131,072 tokens (model card)
- Task
- text-generation · text
- Signal
- 392 HF likes · 1388 downloads · 17 X likes · 3 X post(s)
Runs on
mixed: real files + estimate; need = weights + FP16 KV cache for 8K tokens + 1GB. A sizing calculation, not tested.
- Q4 ~173.1 GB (estimated size): 2× DGX Spark (256GB), Mac 256GB, CPU only, 256GB RAM (slow)
- Q8 ~305.5 GB (estimated size): Mac 512GB
- FP16 ~574.3 GB: no listed device
Links
First published here 2026-10-21 · updated 2026-11-06 · markdown · JSON