Flash Attn Import Errors (https://github.com/huggingface/transformers/issues/28459)
#1
by
rgandikota
- opened
Problem
When we try to use this module for a fine-tuning project on our cluster, there is an issue with the check for flash_attn.
Workaround
A workaround used by phi-1.5 is to wrap the imports in try-except block. We can implement a similar one here till there is a proper fix from the transformers library( Please refer to the issue above for an expanded context)
Phi-1.5 code:
https://huggingface.co/microsoft/phi-1_5/blob/main/modeling_phi.py#L50
Hi @rgandikota ! Sounds good, would you like to open a PR in this repo to add the workaround you proposed?
Sure @regisss . Will create a PR. Thanks.