File size: 1,815 Bytes
556657e b220808 556657e b220808 556657e b220808 556657e b220808 556657e b220808 556657e b220808 556657e b220808 556657e |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 |
TITLE = '<h1 align="center" id="space-title">Open Multilingual Reasoning Leaderboard</h1>'
INTRO_TEXT = f"""
## About
This leaderboard tracks and ranks the reasoning performance of the leading, most advanced multilingual reasoning LLMs on three multilingual mathematical reasoning benchmarks. Each benchmark contains 10 languages: Bengali, Swedish, Thailand, Chinese, Japan, Russian, French, Spanish, German and English.
Both multilingual and language-specific LLMs are welcome in this leaderboard.
## Benchmarks
- <a href="https://huggingface.co/datasets/Mathoctopus/MSVAMP" target="_blank"> MSVAMP </a>
- <a href="https://huggingface.co/datasets/juletxara/mgsm" target="_blank"> MGSM </a>
- <a href="https://huggingface.co/datasets/kevinpro/MNumGLUESub" target="_blank"> MNumGLUESub </a>
"""
HOW_TO = f"""
## How to list your model performance on this leaderboard:
Run the evaluation of your model using this repo: <a href="https://github.com/NJUNLP/MAPO" target="_blank">https://github.com/NJUNLP/MAPO</a>.
And then, push the evaluation log and make a pull request.
"""
# CREDIT = f"""
# ## Credit
# To make this website, we use the following resources:
# - Datasets (AI2_ARC, HellaSwag, MMLU, TruthfulQA)
# - Funding and GPU access (Adobe Research)
# - Evaluation code (EleutherAI's lm_evaluation_harness repo)
# - Leaderboard code (Huggingface4's open_llm_leaderboard repo)
# """
CITATION = f"""
## Citation
```
@misc{{she2024mapo,
title={{MAPO: Advancing Multilingual Reasoning through Multilingual Alignment-as-Preference Optimization}},
author={{Shuaijie She and Wei Zou and Shujian Huang and Wenhao Zhu and Xiang Liu and Xiang Geng and Jiajun Chen}},
year={{2024}},
eprint={{2401.06838}},
archivePrefix={{arXiv}},
primaryClass={{cs.CL}}
}}
```
"""
|