I had issues hosting the model on vllm, adding this solved the issue.

Edit: I think I may have used the wrong model. (should have gone to -chat instead)

IJSDerbashi changed pull request status to closed

Sign up or log in to comment