clibrain

/

Llama-2-ft-instruct-es

like 18

Hi, @eduiqe . Have you tried to upgrade bitsandbytes to the highest version?

Hi, thanks for the response.
I´m using this versions of libraries:
torch == 1.11.0 and 2.0.0
transformers == 4.31.0
bitsandbytes == 0.41.0 (latest)
and I am running this code with cpu instead of cuda, which shouldn´t affect the result (?)

Is there anything else I can try to make the completion work?
Thanks again!

same error here

Same

Thanks for your feedback. Working on fixing it

Same error here. Using load_in_8bit and with auto device map:
model = AutoModelForCausalLM.from_pretrained(model_id, trust_remote_code=True, load_in_8bit = True, device_map='auto')

Response to all inputs is:
ed 10c30c30c30c300c30c30c30c30c30c30c30c30c30c30c30c30c30c30c30c30c30c30c30c30c30c30c30c30c30c30c30c30c30c30c30c30c30c30c30c30-slash.

Hi, guys!
Problem solved here: https://huggingface.co/clibrain/Llama-2-7b-ft-instruct-es