Where can I find a script to fine tune Bloom?
#129
by
mishavee
- opened
Where can I find a script to fine tune Bloom?
Its not that simple. If you want to finetune BLOOM-176B. You have to use Megatron-DeepSpeed for distributed training
what is it?
Some people were successful from a quick read: https://huggingface.co/bigscience/bloom/discussions/46#6354316beca5a823e3b95128
Cross-posting here from the other thread, here's a guide used to finetune BLOOMZ: https://github.com/bigscience-workshop/xmtf#bloomz