Baidu has officially open-sourced its latest base model, ERNIE 5.5 Flash Base. The model utilizes a Mixture of Experts (MoE) architecture with 280 billion total parameters and 14 billion active parameters per token.
ERNIE 5.5 Flash Base was pre-trained on 33.5 trillion tokens with a context length of up to 128K during training. It supports extended contexts of up to 1 million tokens during fine-tuning and inference phases.
