ValueError: JinaCLIPModel does not support Flash Attention 2.0 yet
code:
self.emb_model = AutoModel.from_pretrained(
model_path,
trust_remote_code=True,
torch_dtype = torch.float16,
attn_implementation="flash_attention_2"
)
ValueError: JinaCLIPModel does not support Flash Attention 2.0 yet. Please request to add support where the model is hosted, on its model hub page: https://huggingface.co/xxxx/jina-clip-v2/discussions/new or in the Transformers GitHub repo: https://github.com/huggingface/transformers/issues/new
Hey
@kelvennn
, thanks for reaching out! We use flash attention 2 by default in the text tower and xformers in the vision tower. See https://huggingface.co/jinaai/jina-clip-v2/blob/main/config.json#L31 and https://huggingface.co/jinaai/jina-clip-v2/blob/main/config.json#L63. I am afraid the attn_implementation option is not supported but flash attention 2.0 is enabled by default