quantized models？ This 4b model is a little bit too big for personal use

by NathanNi - opened 27 days ago

Are there any quantized models (e.g., int8 or int4) available for this release/version?

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment