quantization won't work on mac silicon devices

#55

by sryadava - opened Mar 1

Mar 1

Hi,

For the example of A script to perform Supervised Fine-Tuning (SFT) on UltraChat dataset using QLoRA , the quantization won't work on mac silicon devices.
If that is expected can we update the code accordingly that helps us to do enough modifications to the code.

ybelkada

Google org Mar 4

Hi @sryadava
Yes it is expected, bitsandbytes is not supported on MacOS devices

sryadava

Mar 4

@ybelkada ,

Then we have to modify the code and update it for mac users. Can I raise a pull request for the same?

ybelkada

Google org Mar 4

yes feel free to do so and ping me for a review!

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment