Qwen

/

Qwen1.5-72B

like 58

Follow

Qwen 4.83k

I saw in your example for finetuning you use 3e-4 for 7b

https://github.com/QwenLM/Qwen1.5/blob/3360e0c775319b9986ba6a9f644ed52404518f3a/examples/sft/finetune.sh#L95

I wanted to ask, what is recommended learning rate for each size? 72b, 14b, 7b, 4b, 1.8b. 0.5b