Update README.md

README.md

@@ -7,7 +7,7 @@ datasets:
 language:
 - en
 ---
-# Instruction Pre-Training: Language Models are Supervised Multitask Learners
+# Instruction Pre-Training: Language Models are Supervised Multitask Learners (EMNLP 2024)
 This repo contains the **general models pre-trained from scratch** (on 100B tokens) in our paper [Instruction Pre-Training: Language Models are Supervised Multitask Learners](https://huggingface.co/papers/2406.14491).
 
 We explore supervised multitask pre-training by proposing ***Instruction Pre-Training***, a framework that scalably augments massive raw corpora with instruction-response pairs to pre-train language models. The instruction-response pairs are generated by an efficient instruction synthesizer built on open-source models. In our experiments, we synthesize 200M instruction-response pairs covering 40+ task categories to verify the effectiveness of *Instruction Pre-Training*. Instruction Pre-Training* outperforms *Vanilla Pre-training* in both general pre-training from scratch and domain-adaptive continual pre-training. **In pre-training from scratch, *Instruction Pre-Training* not only improves pre-trained base models but also benefits more from further instruction tuning.** In continual pre-training, *Instruction Pre-Training* enables Llama3-8B to be comparable to or even outperform Llama3-70B.
@@ -18,6 +18,8 @@ We explore supervised multitask pre-training by proposing ***Instruction Pre-Tra
 
 
 **************************** **Updates** ****************************
+* 2024/9/20: Our paper has been accepted by EMNLP 2024 main conference🎉
+* 2024/9/11: Updated [FAQ on continual pre-training from Llama3](https://huggingface.co/instruction-pretrain/instruction-synthesizer)
 * 2024/8/29: Updated [guidelines](https://huggingface.co/instruction-pretrain/medicine-Llama3-8B) on evaluating any 🤗Huggingface models on the domain-specific tasks
 * 2024/7/31: Updated pre-training suggestions in the `Advanced Usage` section of [instruction-synthesizer](https://huggingface.co/instruction-pretrain/instruction-synthesizer)
 * 2024/7/15: We scaled up the pre-trained tokens from 100B to 250B, with the number of synthesized instruction-response pairs reaching 500M. The performance trend on downstream tasks throughout the pre-training process:
@@ -77,7 +79,7 @@ accelerate launch -m lm_eval --model hf \
 ## Citation
 If you find our work helpful, please cite us:
 
-Instruction Pre-Training
+[Instruction Pre-Training](https://huggingface.co/papers/2406.14491) (EMNLP 2024)
 ```bibtex
 @article{cheng2024instruction,
   title={Instruction Pre-Training: Language Models are Supervised Multitask Learners},
@@ -87,7 +89,7 @@ Instruction Pre-Training
 }
 ```
 
-[Adapt LLM to Domains](https://huggingface.co/papers/2309.09530)
+[Adapt LLM to Domains](https://huggingface.co/papers/2309.09530)（ICLR 2024）
 ```bibtex
 @inproceedings{
 cheng2024adapting,