simarora commited on
Commit
c720ef0
1 Parent(s): c05ef2d

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -2
README.md CHANGED
@@ -8,8 +8,9 @@ language:
8
 
9
  This model is pretrained Based model.
10
 
11
- As a quality reference, we include a pretrained Mamba model provided here: https://huggingface.co/hazyresearch/mamba-1b
12
- Both checkpoints are pretrained on 10Bn tokens of the Pile in the exact same data order using next token prediction.
 
13
 
14
 
15
  ### Model Sources
 
8
 
9
  This model is pretrained Based model.
10
 
11
+ As a quality reference, we include a pretrained Mamba model provided here: https://huggingface.co/hazyresearch/mamba-1b, and a pretrained Attention (Llama architecture) model provided here: https://huggingface.co/hazyresearch/attn-1b
12
+
13
+ All three checkpoints are pretrained on 10Bn tokens of the Pile in the exact same data order using next token prediction.
14
 
15
 
16
  ### Model Sources