File size: 3,598 Bytes
a19b91b 6c8298b a19b91b 6c8298b a19b91b 6c8298b a19b91b 6c8298b a19b91b 6c8298b |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 |
---
license: apache-2.0
metrics:
- accuracy
pipeline_tag: text-generation
tags:
- multilingual
---
## MAPO: Advancing Multilingual Reasoning through Multilingual Alignment-as-Preference Optimization
**🔥Our paper**
https://arxiv.org/abs/2401.06838
**🔥Github Project**
https://github.com/NJUNLP/MAPO
**🔥Open Multilingual Reasoning Leaderboard**
https://huggingface.co/spaces/kevinpro/Open-Multilingual-Reasoning-Leaderboard
## Benchmarks
| System | [MSVAMP](https://huggingface.co/datasets/Mathoctopus/MSVAMP) | [MGSM](https://huggingface.co/datasets/juletxara/mgsm) | [MNumGLUESub](https://huggingface.co/datasets/Mathoctopus/MSVAMP) |
| ------ | :-------------------------------------------------------------: | :------------------------------------------------------: | :-------------------------------------------------------------: |
| GPT-3.5-Turbo | 46.6 | 42.2 | 49.4 |
| [MAmmoTH 7B](https://huggingface.co/TIGER-Lab/MAmmoTH-7B) | 26.3 | 21.3 | 24.2 |
| [WizardMath 7B](https://huggingface.co/WizardLM/WizardMath-7B-V1.1) | 32.5 | 23.0 | 28.7 |
| [MetaMath 7B](https://huggingface.co/meta-math/MetaMath-7B-V1.0) | 46.2 | 37.0 | 43.2 |
| [QAlign 7B](https://huggingface.co/Wenhao97/QAlign-MetaMathQA-7B) | 57.2 | 49.6 | - |
| [MathOctopus 7B](https://huggingface.co/Mathoctopus/Parallel_7B) | 41.2 | 39.5 | 37.1 |
| **[+ MAPO-DPO(ours)🔥](https://huggingface.co/kevinpro/MathOctopus-MAPO-DPO-7B)** | **57.4** | **41.6** | **50.4** |
| [MetaMathOctopus 7B](https://huggingface.co/kevinpro/MetaMathOctopus-7B) | 53.0 | 45.5 | 39.2 |
| **[+ MAPO-DPO(ours) 👑](https://huggingface.co/kevinpro/MetaMathOctopus-MAPO-DPO-7B)** | **64.7** | **51.6** | **52.9** |
| [MistralMathOctopus 7B](https://huggingface.co/kevinpro/MistralMathOctopus-7B) | 59.0 | 58.0 | 56.8 |
| **[+ MAPO-DPO(ours) 👑](https://huggingface.co/kevinpro/MistralMathOctopus-MAPO-DPO-7B)** | **74.6** | **67.3** | **70.0** |
| System | [MSVAMP](https://huggingface.co/datasets/Mathoctopus/MSVAMP) | [MGSM](https://huggingface.co/datasets/juletxara/mgsm) | [MNumGLUESub](https://huggingface.co/datasets/Mathoctopus/MSVAMP) |
| ------ | :-------------------------------------------------------------: | :------------------------------------------------------: | :-------------------------------------------------------------: |
| GPT-3.5-Turbo | 46.6 | 42.2 | 49.4 |
| [MAmmoTH 13B](https://huggingface.co/TIGER-Lab/MAmmoTH-13B) | 38.6 | 28.9 | 29.5 |
| [WizardMath 13B](https://huggingface.co/WizardLM/WizardMath-13B-V1.1) | 35.7 | 28.3 | 29.0 |
| [MetaMath 13B](https://huggingface.co/meta-math/MetaMath-13B-V1.0) | 46.2 | 43.9 | 43.3 |
| [QAlign 13B](https://huggingface.co/Wenhao97/QAlign-MetaMathQA-13B) | 62.6 | 57.1 | - |
| [MathOctopus 13B](https://huggingface.co/Mathoctopus/Parallel_13B) | 51.8 | 46.0 | 40.3 |
| **[+ MAPO-DPO(ours)🔥](https://huggingface.co/kevinpro/MathOctopus-MAPO-DPO-13B)** | **60.1** | **48.5** | **53.8** |
| [MetaMathOctopus 13B](https://huggingface.co/kevinpro/MetaMathOctopus-13B) | 56.3 | 51.4 | 49.5 |
| **[+ MAPO-DPO(ours) 👑](https://huggingface.co/kevinpro/MetaMathOctopus-MAPO-DPO-13B)** | **67.0** | **58.0** | **59.8** |
## Citation
If you find this model helpful, feel free to cite our paper:
```
@misc{she2024mapo,
title={MAPO: Advancing Multilingual Reasoning through Multilingual Alignment-as-Preference Optimization},
author={Shuaijie She and Wei Zou and Shujian Huang and Wenhao Zhu and Xiang Liu and Xiang Geng and Jiajun Chen},
year={2024},
eprint={2401.06838},
archivePrefix={arXiv},
primaryClass={cs.CL}
}
``` |