Create README.md

README.md

@@ -0,0 +1,36 @@
+---
+datasets:
+- natolambert/skywork-preferences-80k-v0.1-cleaned
+- allenai/preference-test-sets
+---
+
+# MetaMetrics-RM-v1.0
+
++ **Authors** [Genta Indra Winata](https://gentawinata.com/), [David Anugraha](https://weixiongust.github.io/WeiXiongUST/index.html), [Lucky Susanto](https://tengyangxie.github.io/), [Garry Kuwanto](https://hanzhaoml.github.io/), [Derry Tanti Wijaya](https://tongzhang-ml.org/)
++ **Paper**: https://arxiv.org/abs/2406.12845
++ **Model**: [meta-metrics/MetaMetrics-RM-v1.0](https://huggingface.co/meta-metrics/MetaMetrics-RM-v1.0)
++ **Dataset**:
+  - [natolambert/skywork-preferences-80k-v0.1-cleaned](https://huggingface.co/datasets/natolambert/skywork-preferences-80k-v0.1-cleaned)
+  - [allenai/preference-test-sets](https://huggingface.co/datasets/allenai/preference-test-sets)
++ **Code Repository:** https://github.com/meta-metrics/metametrics
+
+## RewardBench LeaderBoard
+
+ | Model  | Score | Chat | Chat Hard | Safety | Reasoning |
+|:--------------------------------------------------------------------------------|:-----------------------------------------------------------------------|:-----:|:-----|:----------|:-------|:----------|:-----------------------|:------------------------|
+  | nvidia/Llama-3.1-Nemotron-70B-Reward     | **94.1** | 97.5 | 85.7 | **95.1** | 98.1 |
+  | meta-metrics/MetaMetrics-RM-v1.0         | 93.5 | **98.9** | 86.2 | 90.7 | **98.2** |
+  | SF-Foundation/TextEval-Llama3.1-70B      | 93.5 | 94.1 | **90.1**  | 93.2 | 96.4 |
+  | RLHFlow/ArmoRM-Llama3-8B-v0.1            | 90.4 | 96.9 | 76.8 | 90.5 | 97.3 |
+
+## Citation
+
+If you find this work useful for your research, please consider citing:
+```
+@article{winata2024metametrics,
+  title={MetaMetrics: Calibrating Metrics For Generation Tasks Using Human Preferences},
+  author={Winata, Genta Indra and Anugraha, David and Susanto, Lucky and Kuwanto, Garry and Wijaya, Derry Tanti},
+  journal={arXiv preprint arXiv:2410.02381},
+  year={2024}
+}
+```