Post
🚨 Now you can run Starcoder- 2 models locally on your Mac M1 Pro Apple Silicon with 16GB memory! 🧑🏽💻 ⚡️✨
Below is the UX with Twinny extension using bigcode/starcoder2-3b for FIM and codellama/CodeLlama-7b-Instruct-hf for chat. Dev tools is showing the prompt being sent to ollama server.
Starcoder-2 is now supported in
For more details, please go through the following tweet thread: https://x.com/sourab_m/status/1764583139798823235?s=20
Below is the UX with Twinny extension using bigcode/starcoder2-3b for FIM and codellama/CodeLlama-7b-Instruct-hf for chat. Dev tools is showing the prompt being sent to ollama server.
Starcoder-2 is now supported in
llama.cpp https://github.com/ggerganov/llama.cpp/pull/5795!cd llama.cpp
python convert-hf-to-gguf.py ../starcoder2-3b/ --outfile models/starcoder2-3b.gguf --outtype "f16"
./quantize models/starcoder2-3b.gguf models/starcoder2-3b-Q4_K_M.gguf Q4_K_MFor more details, please go through the following tweet thread: https://x.com/sourab_m/status/1764583139798823235?s=20