ValueError: MegatronBertForSequenceClassification does not support `device_map='auto'` yet.
#1
by
shanchen
- opened
is int8 support available?
These are questions for the transformer team I think. If they have not implemented to support these features for Megatron, then you need to raise the issues to the transformer repository. @shanchen
it seems other people did this. it didn't work w/ MegatronBertForSequenceClassification
https://huggingface.co/mosaicml/mpt-7b/discussions/45/files