Update config.json
#2
by
Pinkgu1
- opened
In github README.md, the user has to update these keys in config.json before running this model.
"attn_logit_softcapping": null,
"final_logit_softcapping": null,
Can we update the config.json to reflect this instead so that the user can just pull from huggingface without needed to patch.
Pinkgu1
changed discussion status to
closed
Hi @Pinkgu1 ,
My apologies, the instruction from the README is outdated and we have removed the section about setting the softcapping keys to null.
This was originally a workaround for when softcapping was not widely adopted, this patch should not be required with the current version of vllm.
Thank you