mosaicml/mpt-30b · Instructions for finetuning?

Hey there,

Thanks for the amazing work as always! I want to finetune the mpt-30b and its instruct variant on a custom dataset. Is there a reference script / notebook available that shows an example of how to do so? I tried looking around but couldn't find one.

I did try to finetune it using a vanilla huggingface Trainer with qlora and that didn't seem to work unfortunately. It gave a weird "mpt models do not support gradient checkpointing" error.

p.s. @abhi-mosaic @sam-mosaic is this something that you could help out with?