Implementation is not working in EC2
#38
by
mohankrishnan
- opened
Hi there,
Currently, I tried to deploy the Cohere model in AWS EC2 with (p3.16x large model) but it doesn't produce the output for the given question. When I click enter on the same question, the model loads again.
Even when I load the quantized one with 4bit, it comes with some random accelerate and bitsandbytes error
What is the issue and how to fix this.