7 contributors

History: 18 commits

versae

Adding sampling to mc4

3f09f56 over 1 year ago

mc4
Adding sampling to mc4 over 1 year ago
.gitattributes

690 Bytes initial commit over 1 year ago
.gitignore

1.84 kB Initial test with BETO's corpus over 1 year ago
README.md

1.84 kB Adding checkpointing, wandb, and new mlm script over 1 year ago
config.json

619 Bytes Initial test with BETO's corpus over 1 year ago
config.py

247 Bytes Scripts for perplexity sampling and fixes over 1 year ago
convert.py

458 Bytes Scripts for perplexity sampling and fixes over 1 year ago
flax_model.msgpack

712 MB
LFS
Adding correct models 10k steps over 1 year ago
get_embeddings_and_perplexity.py

1.53 kB Add script to generate dataset of embeddings and perplexities. Add script to generate t-SNE plot for embedding and perplexity visualization. over 1 year ago
perplexity.py

751 Bytes Adding checkpointing, wandb, and new mlm script over 1 year ago
pytorch_model.bin
Detected Pickle imports (4)
- "torch.FloatStorage",
- "torch.LongStorage",
- "torch._utils._rebuild_tensor_v2",
- "collections.OrderedDict"
What is a pickle import?
1.42 GB
LFS
Adding correct models 10k steps over 1 year ago
run.sh

978 Bytes Updating run script over 1 year ago
run_mlm_flax.py

30 kB Adding sampling to mc4 over 1 year ago
run_mlm_flax_stream.py

28 kB Adding sampling to mc4 over 1 year ago
run_stream.sh

928 Bytes Adding sampling to mc4 over 1 year ago
tokenizer.json

1.45 MB Initial test with BETO's corpus over 1 year ago
tokens.py

649 Bytes Scripts for perplexity sampling and fixes over 1 year ago
tokens.py.orig

899 Bytes Adjust batch size for extrating tokens over 1 year ago
tsne_plot.py

3.02 kB Remove unused imports over 1 year ago

Detected Pickle imports (4)