7 contributors

History: 62 commits

amank

Updated code to work with streaming version

139e10d almost 3 years ago

.vscode
Updated code to work with streaming version almost 3 years ago
roberta_mc4_sentence_piece
Updated code to work with streaming version almost 3 years ago
.gitattributes

737 Bytes

Saving weights and logs of epoch 2 almost 3 years ago
.gitignore

12 Bytes

Updated code to work with streaming version almost 3 years ago
README.md

6.79 kB

Update README.md almost 3 years ago
config.json

692 Bytes

mc4 epoch14 exported torch config & tokenizer almost 3 years ago
create_config.py

147 Bytes

All set to train almost 3 years ago
flax_to_torch.py

793 Bytes

Convert latest model to PyTorch almost 3 years ago
merges.txt

1.21 MB

Pytorch export fix almost 3 years ago
run.sh

562 Bytes

Updated code to have different seed and reduced lr almost 3 years ago
run_mlm_flax.py

30.6 kB

Updated code to work with streaming version almost 3 years ago
run_mlm_flax_old.py

28.7 kB

Updated code to work with streaming version almost 3 years ago
run_mlm_flax_stream.py
26.9 kB

Updated code to work with streaming version almost 3 years ago
run_stream.sh
633 Bytes

Updated code to work with streaming version almost 3 years ago
special_tokens_map.json

239 Bytes

Pytorch export fix almost 3 years ago
tokenizer.json

2.86 MB

mc4 epoch14 exported torch config & tokenizer almost 3 years ago
tokenizer_config.json

279 Bytes

Pytorch export fix almost 3 years ago
train_tokenizer.py

1.08 kB

Updated code to work with streaming version almost 3 years ago
utils.py
1.22 kB

Updated code to work with streaming version almost 3 years ago
vocab.json

1.55 MB

Pytorch export fix almost 3 years ago