Update README.md

README.md

@@ -7,10 +7,10 @@ language:
 ---
 This is a model released for our paper: [REBEL: Reinforcement Learning via Regressing Relative Rewards](https://arxiv.org/abs/2404.16767). 
 
-# REBEL-Llama-3
+# REBEL-Llama-3-epoch_2
 
 This model is developed with REBEL based on [Meta-Llama-3-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct) with [FsfairX-LLaMA3-RM-v0.1](https://huggingface.co/sfairXC/FsfairX-LLaMA3-RM-v0.1) as the reward model and [UltraFeedback](https://huggingface.co/datasets/openbmb/UltraFeedback) dataset.
-The training code is available at https://github.com/ZhaolinGao/REBEL. This is the checkpoint that achieves the highest AlpacaEval 2.0 scores.
+The training code is available at https://github.com/ZhaolinGao/REBEL. We collect online generations during each iteration with a batch size of 32.
 
 ### Links to Other Model
 
@@ -18,13 +18,22 @@ The training code is available at https://github.com/ZhaolinGao/REBEL. This is t
 
 [REBEL-Llama-3](https://huggingface.co/Cornell-AGI/REBEL-Llama-3)
 
-### AlpacaEval 2.0 Evaluations
+[REBEL-Llama-3-Armo-iter_1](https://huggingface.co/Cornell-AGI/REBEL-Llama-3-Armo-iter_1)
 
-| Model | AlpacaEval 2.0<br>LC Win Rate | AlpacaEval 2.0<br>Win Rate |
-| :--------: | :--------: |  :--------: |
-| REBEL-OpenChat-3.5| 17.3 | 12.8 |
-| REBEL-Llama-3 | 30.1 | 32.6 |
-| REBEL-Llama-3-epoch_2| 31.33 | 34.22 |
+[REBEL-Llama-3-Armo-iter_2](https://huggingface.co/Cornell-AGI/REBEL-Llama-3-Armo-iter_2)
+
+[REBEL-Llama-3-Armo-iter_3](https://huggingface.co/Cornell-AGI/REBEL-Llama-3-Armo-iter_3)
+
+### Evaluations
+
+| Model | AlpacaEval 2.0<br>LC Win Rate | AlpacaEval 2.0<br>Win Rate | MT-Bench<br>Average | MMLU<br>(5-shot) | GSM8K<br>(5-shot) |
+| :--------: | :--------: |   :--------: | :--------: |  :--------: | :--------: |
+| REBEL-OpenChat-3.5| 17.3 | 12.8 | 8.06 | 63.7 | 68.8 |
+| REBEL-Llama-3 | 30.1 | 32.6 | 8.16 | 65.8 | 75.6 |
+| REBEL-Llama-3-epoch_2| 31.3 | 34.2 | 7.83 | 65.4 | 75.4 |
+| REBEL-Llama-3-Armo-iter_1| 48.3 | 41.8 | 8.13 | 66.3 | 75.8 |
+| REBEL-Llama-3-Armo-iter_2| 50.0 | 48.5 | 8.07 | 65.9 | 75.4 |
+| REBEL-Llama-3-Armo-iter_3| 49.7 | 48.1 | 8.01 | 66.0 | 75.7 |
 
 ## Citation
 Please cite our paper if you use this model in your own work:
@@ -37,11 +46,4 @@ Please cite our paper if you use this model in your own work:
       archivePrefix={arXiv},
       primaryClass={cs.LG}
 }
-```
-
-
-
-
-
-
-
+```