Why need get_mup_param_groups instead of default one in Huggingface?

#18

by sanqiang - opened Aug 13, 2023

Aug 13, 2023

I think it is from Tensor Programs V paper.
Just curious how to train it: Deepspeed with custom optimizer will raise exception.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment