Qlora Fine tuning error
#4
by
TinyPixel
- opened
TypeError Traceback (most recent call last)
in <cell line: 1>()
----> 1 trainer.train()
14 frames
/usr/local/lib/python3.10/dist-packages/accelerate/hooks.py in new_forward(module, *args, **kwargs)
163 output = module._old_forward(*args, **kwargs)
164 else:
--> 165 output = module._old_forward(*args, **kwargs)
166 return module._hf_hook.post_forward(module, output)
167
TypeError: OLMoForCausalLM.forward() got an unexpected keyword argument 'inputs_embeds'
Hi, it looks like you are trying to call model.forward with the inputs_embeds keyword argument. OLMo currently does not support passing input embeddings in directly with this argument.