UFNLP

/

gatortron-medium

like 22

Follow

University of Florida NLP Group 23

These are questions for the transformer team I think. If they have not implemented to support these features for Megatron, then you need to raise the issues to the transformer repository. @shanchen

it seems other people did this. it didn't work w/ MegatronBertForSequenceClassification
https://huggingface.co/mosaicml/mpt-7b/discussions/45/files