CHANGELOG.md

version： v20240321

• train 17B tokens with 256k context window
• recall of "Needle in A HayStack": 99.0%

version： v20240318

• train 12B tokens with 256k context window
• recall of "Needle in A HayStack": 97.5%

version: initial

• train 6B tokens with 256k context window
• recall of "Needle in A HayStack": 87.1%