haining commited on
Commit
7203ecd
1 Parent(s): 3ef0559

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -121,7 +121,7 @@ We finetuned the base model (flan-t5-large) on multiple relevant tasks with stan
121
  | Recontextualization | Editor Abstract | "contextualize: " | 2,200 |
122
  | Simplification | Wiki Auto | "simplify: " | 57,000 |
123
  | Summarization | CNN/DailyMail | "summarize: " | 165,000 |
124
- | Total | Challenge-proportional Mixture | n/a | 263,400 |
125
 
126
 
127
  - Multi-instruction tuning: In the stage, we first created a task mixture using "challenge-proportional mixing" method. In a seperate pilot studie, for each task, we finetuned it on a base model and observed the number of samples when validation loss starts to rise. We mixed the samples of each task proportional to its optimal number of samples. A corpus is exhausted before upsampling if the number of total samples is smaller than its optimal number. We finetune with the task mixture (263,400 samples) with the aforementioned template.
 
121
  | Recontextualization | Editor Abstract | "contextualize: " | 2,200 |
122
  | Simplification | Wiki Auto | "simplify: " | 57,000 |
123
  | Summarization | CNN/DailyMail | "summarize: " | 165,000 |
124
+ | Total | Challenge-proportional Mixing | n/a | 263,400 |
125
 
126
 
127
  - Multi-instruction tuning: In the stage, we first created a task mixture using "challenge-proportional mixing" method. In a seperate pilot studie, for each task, we finetuned it on a base model and observed the number of samples when validation loss starts to rise. We mixed the samples of each task proportional to its optimal number of samples. A corpus is exhausted before upsampling if the number of total samples is smaller than its optimal number. We finetune with the task mixture (263,400 samples) with the aforementioned template.