efederici commited on
Commit
fa0590e
1 Parent(s): 3f94072

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -12,7 +12,7 @@ language:
12
 
13
  # ipt-350m
14
 
15
- ipt-350m is a decoder-style transformer pretrained from scratch on ~13B tokens of Italian text.
16
 
17
  It uses a modified transformer architecture optimized for efficient training and inference. Positional embeddings are replaced with Attention with Linear Biases ([ALiBi](https://arxiv.org/abs/2108.12409)).
18
 
 
12
 
13
  # ipt-350m
14
 
15
+ ipt-350m is a decoder-style transformer pretrained from scratch on ~13B tokens of Italian text (wip: trained on unfiltered oscar).
16
 
17
  It uses a modified transformer architecture optimized for efficient training and inference. Positional embeddings are replaced with Attention with Linear Biases ([ALiBi](https://arxiv.org/abs/2108.12409)).
18