microsoft/Phi-3.5-mini-instruct not working with FA2 due to position_ids
#27
by
BramVanroy
- opened
It seems not possible to use this model with any flash attention mechanism. SDPA triggers a "not supported" error and FA2 leads to errors with position_ids as reported here: https://github.com/huggingface/transformers/issues/35274