GitHub repository: microsoft/LongRoPE
Primary language: Python
Description: LongRoPE is a novel method that can extends the context window of pre-trained LLMs to an impressive 2048k tokens.