Longhui98 commited on
Commit
ae184d2
1 Parent(s): 2ea0355

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -12,7 +12,7 @@ https://meta-math.github.io/
12
 
13
  MetaMath-Mistral-7B is fully fine-tuned on the MetaMathQA datasets and based on the very strong Mistral-7B model. It is glad to see using MetaMathQA datasets and change the base model from llama-2-7B to Mistral-7b can boost the GSM8K performance from 66.5 to **77.7**.
14
 
15
- For everyone who wants to fine-tune Mistral-7B, I would suggest using a smaller learning rate(usually 1/5 to 1/10 of the lr for LlaMa-2-7B) and staying other training args unchanged.
16
  More training details and scripts can be seen at https://github.com/meta-math/MetaMath
17
 
18
  ## Installation
 
12
 
13
  MetaMath-Mistral-7B is fully fine-tuned on the MetaMathQA datasets and based on the very strong Mistral-7B model. It is glad to see using MetaMathQA datasets and change the base model from llama-2-7B to Mistral-7b can boost the GSM8K performance from 66.5 to **77.7**.
14
 
15
+ To fine-tune Mistral-7B, I would suggest using a smaller learning rate (usually 1/5 to 1/10 of the lr for LlaMa-2-7B) and staying other training args unchanged.
16
  More training details and scripts can be seen at https://github.com/meta-math/MetaMath
17
 
18
  ## Installation