File size: 1,345 Bytes
b91b3c5 82a0565 b91b3c5 82a0565 b91b3c5 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 |
---
license: llama2
datasets:
- meta-math/MetaMathQA
---
## Model Details
metamath-llemma-34b is Q-Lora fine-tuned on the MetaMathQA datasets and based on the powerful llemma-34b model.
## Installation
```
pip install transformers==4.35.0
pip install torch==2.0.1
pip install sentencepiece==0.1.99
pip install tokenizers==0.13.3
pip install accelerate==0.21.0
pip install bitsandbytes==0.40.0
pip install vllm
pip install fraction
pip install protobuf
```
## Model Usage
prompting template:
'''
"Below is an instruction that describes a task. "
"Write a response that appropriately completes the request.\n\n"
"### Instruction:\n{instruction}\n\n### Response: Let's think step by step."
'''
where you need to use your query question to replace the {instruction}
## Experiments
| Model | GSM8k Pass@1 | MATH Pass@1 |
|---------------------|--------------|-------------|
| MetaMath-7B | 66.5 | 19.8 |
| MetaMath-13B | 72.3 | 22.4 |
| MetaMath-70B | **82.3** | 26.6 |
| MetaMath-Llemma-7B | 69.2 | 30.0 |
| MetaMath-Mistral-7B | 77.7 | 28.2 |
| 🔥 metamath-llemma-34B | 76.1 | **31.4** |
*Model card adapted from [MetaMath-70B-V1.0](https://huggingface.co/meta-math/MetaMath-70B-V1.0/blob/main/README.md)* |