Make model vllm compatible
#4
by
vrdn23
- opened
The model had an un-used layer of final_logits_bias which was full of zeroes that was causing vLLM to fail to load the model. Removing that has no impact on quality which I've verified. Also regenerated the new onnx models and moved them to the onnx folder.
vrdn23
changed pull request status to
open
I've verified it results match with the new onnx models.
vrdn23
changed pull request status to
merged