KennethEnevoldsen's picture
Initial commit
c7392f2
|
raw
history blame
No virus
597 Bytes
Trained using SimCSE with:
```
CUDA_VISIBLE_DEVICES=0 python train.py \
--train_file data/dfm_paragraphs.txt \
--model_name_or_path chcaa/dfm-encoder-large-v1 \
--output_dir result/dfm-sentence-encoder-medium-v4 \
--num_train_epochs 1 \
--per_device_train_batch_size 128 \
--learning_rate 1e-5 \
--max_seq_length 32 \
--evaluation_strategy steps \
--metric_for_best_model stsb_spearman \
--load_best_model_at_end \
--pooler_type cls \
--mlp_only_train \
--do_mlm \
--overwrite_output_dir \
--temp 0.05 \
--do_train \
--fp16
```