haining commited on
Commit
7423c06
1 Parent(s): 7c565f1

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -123,7 +123,7 @@ We finetuned the base model (flan-t5-large) on multiple relevant tasks with stan
123
  | Total | Challenge-proportional Mixing | n/a | 263,400 |
124
 
125
 
126
- - Multi-instruction pretuning: In the stage, we first created a task mixture using "challenge-proportional mixing" method. In a seperate pilot studie, for each task, we finetuned it on a base model and observed the number of samples when validation loss starts to rise. We mixed the samples of each task proportional to its optimal number of samples. A corpus is exhausted before upsampling if the number of total samples is smaller than its optimal number. We finetune with the task mixture (263,400 samples) with the aforementioned template.
127
 
128
  - fine-tuning: In this stage, we continued finetuning the checkpoint solely with the Scientific Abstract-Significance corpus till optimal validation loss was observed.
129
 
 
123
  | Total | Challenge-proportional Mixing | n/a | 263,400 |
124
 
125
 
126
+ - Multi-instruction pretuning: In the stage, we first created a task mixture using "challenge-proportional mixing" method. In a separate pilot study, for each task, we finetuned it on a base model and observed the number of samples when validation loss starts to rise. We mixed the samples of each task proportional to its optimal number of samples. A corpus is exhausted before upsampling if the number of total samples is smaller than its optimal number. We finetune with the task mixture (263,400 samples) with the aforementioned template.
127
 
128
  - fine-tuning: In this stage, we continued finetuning the checkpoint solely with the Scientific Abstract-Significance corpus till optimal validation loss was observed.
129