Update README.md

README.md

@@ -1,3 +1,15 @@
----
-license: cc-by-nc-4.0
----
+---
+license: cc-by-nc-4.0
+datasets:
+- fdqerq22ds/MathScaleQA-2M
+---
+## Overview
+This is a reproduced MathScale-Mistral model by finetuning the [Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1) on our reproduced [MathScaleQA-2M](https://huggingface.co/datasets/fdqerq22ds/MathScaleQA-2M) dataset, following the hyperparameters in their [original paper](https://arxiv.org/abs/2403.02884) to ensure the effectiveness of our reproduction.
+
+## Reproduction Details
+Fortunately, the reproduction was smooth, and we managed to match the reported performance metrics when evaluating on their [MWPBench](https://github.com/microsoft/unilm/tree/master/mathscale/MWPBench). Below, we present a comparison between the performance of their official model and our reproduced model:
+
+| Model                         | GSM8K | MATH | CollegeMath | TAL  | Math23k | Ape210k | GaokaoBench-Math | AGIE-Gaokao-Math | AGIE-SAT-Math | AGIE-MATH | MicroAverage | MacroAverage |
+|-------------------------------|-------|------|-------------|------|---------|---------|------------------|------------------|---------------|-----------|--------------|--------------|
+| Official MathScale-Mistral    | 74.8  | 35.2 | 21.8        | 39.9 | 64.4    | 46.0    | 21.4             | 14.3             | 57.8          | 32.9      | 38.7         | 40.8         |
+| [Reproduced MathScale-Mistral](https://huggingface.co/fdqerq22ds/MathScale-Mistral)  | 74.0  | 34.5 | 22.0        | 39.6 | 61.7    | 45.1    | 21.6             | 15.5             | 56.8          | 34.4      | 38.3         | 40.5         |