Nous Research has open-sourced Lighthouse Attention, a novel pretraining mechanism that significantly accelerates computation on B200 GPUs. This approach achieves a 17x speed increase when processing 512K-length text compared to traditional methods. Additionally, it offers a 1.4x to 1.7x training acceleration for 98K-length sequences. Lighthouse Attention employs a two-stage process, initially scanning compressed text summaries to select key segments, which are then processed using FlashAttention. This method avoids the need for low-level coding and additional training objectives, overcoming limitations of previous acceleration techniques. Tests on a 530-million-parameter model trained on 50 billion tokens demonstrated reduced training times while maintaining or exceeding baseline performance.