google

/

gemma-7b-it

like 1.11k

I am getting an error just trying to run this model, can anyone help me identify what this issue is and how to fix it. I am using this command in (databricks environment) just to load the model:

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

tokenizer = AutoTokenizer.from_pretrained("google/gemma-7b-it") # google/gemma-2b-it
model = AutoModelForCausalLM.from_pretrained("google/gemma-7b-it", device_map="auto", torch_dtype=torch.bfloat16)

However when it is trying to load I recieve this error: Failed to import transformers.models.gemma.modeling_gemma because of the following error (look up to see its traceback):
/databricks/python/lib/python3.10/site-packages/flash_attn_2_cuda.cpython-310-x86_64-linux-gnu.so: undefined symbol: _ZN2at4_ops9_pad_enum4callERKNS_6TensorEN3c108ArrayRefINS5_6SymIntEEElNS5_8optionalIdEE

I am installing the libraries: transformers 4.38.1, accelerate 0.23.0, torch2.2.1+cu121

Any help to understand what this error means exactly and why it is called or if konw how to fix. I cannot seem to find anything to help