In its current state, this model has not yet been finetuned on long-context data. Do not use this model without performing additional pretraining, as performance will likely suffer.

Similar to the other publicly available LSG models, this model was created using the LSG conversion script found here:
https://github.com/ccdv-ai/convert_checkpoint_to_lsg