bigcode

/

starcoder

like 2.71k

https://github.com/ravenscroftj/turbopilot

Yes. Try https://github.com/bigcode-project/starcoder.cpp

you can, depending on the memory of your Mac. In my case, I can load and do inference for StarCoder2-3B on my M1 32GB, without quantization. Last time I checked, bitsandbytes (which is used to do the 8 or 4-bit quantization) is not available on Apple M chips