|
Trained using SimCSE with: |
|
|
|
``` |
|
CUDA_VISIBLE_DEVICES=0 python train.py \ |
|
--train_file data/dfm_paragraphs.txt \ |
|
--model_name_or_path chcaa/dfm-encoder-large-v1 \ |
|
--output_dir result/dfm-sentence-encoder-medium-v4 \ |
|
--num_train_epochs 1 \ |
|
--per_device_train_batch_size 128 \ |
|
--learning_rate 1e-5 \ |
|
--max_seq_length 32 \ |
|
--evaluation_strategy steps \ |
|
--metric_for_best_model stsb_spearman \ |
|
--load_best_model_at_end \ |
|
--pooler_type cls \ |
|
--mlp_only_train \ |
|
--do_mlm \ |
|
--overwrite_output_dir \ |
|
--temp 0.05 \ |
|
--do_train \ |
|
--fp16 |
|
``` |