Crystina ZHANG commited on
Commit
44ad610
2 Parent(s): 82d68e7 0c9678e

Merge branch 'main' of https://huggingface.co/crystina-z/monoXLMR.pft-msmarco

Browse files
Files changed (1) hide show
  1. README.md +30 -0
README.md ADDED
@@ -0,0 +1,30 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ datasets:
3
+ - Tevatron/msmarco-passage
4
+ ---
5
+ Trained with Tevatron `reranker` branch;
6
+
7
+ script:
8
+ ```
9
+ epoch=3
10
+ bs=32
11
+ gradient_accumulation_steps=8
12
+ real_bs=$(( $bs / $gradient_accumulation_steps ))
13
+
14
+ CUDA_VISIBLE_DEVICES=0 python examples/reranker/reranker_train.py \
15
+ --output_dir reranker_xlmr.bs-$bs.epoch-$epoch \
16
+ --model_name_or_path xlm-roberta-large \
17
+ --save_steps 20000 \
18
+ --dataset_name Tevatron/msmarco-passage \
19
+ --fp16 \
20
+ --per_device_train_batch_size $real_bs \
21
+ --gradient_accumulation_steps $gradient_accumulation_steps \
22
+ --train_n_passages 8 \
23
+ --learning_rate 5e-6 \
24
+ --q_max_len 16 \
25
+ --p_max_len 128 \
26
+ --num_train_epochs $epoch \
27
+ --logging_steps 500 \
28
+ --dataloader_num_workers 4 \
29
+ --overwrite_output_dir
30
+ ```