contrastive pretraining (pious-haze-45)
3987233
Andrew Green
commited on
Commit History
Reset to last good pretrained version
84f48a4
Andrew Green
commited on
re-finetuned on deduplicated similarity data
579981a
Andrew Green
commited on
Prototype after contrastive learning first pass. Unlikely to be good due to data duplication
519ad17
Andrew Green
commited on
Updated pretrained model on 4 epochs (58,500 steps)
7341c83
Andrew Green
commited on
Updated model with better pretraining. 30% MLM rate and shuffled source data
c65927d
Andrew Green
commited on
Upload model
10e91d9
verified
Upload model
cd68b5d
verified
Upload model
101372f
verified
Upload model
79da6c6
verified
Upload model
9472c5e
verified
Upload model
b49ae26
verified
Update README.md
53e89e4
verified
Move MLM readme into this repo.
bc15b88
verified
Should be a base model with no MLM head
cb63d3a
Andrew Green
commited on