## version： v20240321
- train 17B tokens with 256k context window
- recall of "Needle in A HayStack": 99.0%  ![](./images/v20240321.png)

## version： v20240318
- train 12B tokens with 256k context window
- recall of "Needle in A HayStack": 97.5%  ![](./images/v20240318.png)

## version: initial 
- train 6B tokens with 256k context window
- recall of "Needle in A HayStack": 87.1%  ![](./images/initail.png)