What library was used to quantize the model?

by KirillR - opened Apr 6, 2024

Apr 6, 2024

Did you use AutoGPTQ to quantize the model or something else? I'm trying to replicate this myself but AutoGPTQ doesn't support cohere

Owner Apr 6, 2024

Please see #1

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

Your need to confirm your account before you can post a new comment.

· Sign up or log in to comment