exllama requires that pad_token_id be specified in config.json
#2
by
mike-ravkine
- opened
Hi @TheBloke ,
exllama crashes unless pad_token_id is set in config.json: https://github.com/turboderp/exllama/blob/master/model.py#L52
I have filled in the value of the [PAD] token here as per https://huggingface.co/TheBloke/WizardCoder-Guanaco-15B-V1.1-GPTQ/blob/main/special_tokens_map.json#L25
This issue is likely unique to the GPTQ quant but would affect some of the other branches that have exllama support I think.
Thanks as always,
--Mike
But hang on, this is not a Llama model. So Exllama won't support it anyway, will it?
@TheBloke Ahh you're right they've added Llama2 but for some reason still holding out on BigCode support. This indeed wont help much until then..
mike-ravkine
changed pull request status to
closed