kpi/fasthtml-llama3.2-3b-instruct-unsloth
Updated
•
10
kPaul
kpi
AI & ML interests
None yet
Recent Activity
updated
a model
about 1 month ago
kpi/fasthtml-llama3.2-3b-instruct-unsloth
published
a model
about 1 month ago
kpi/fasthtml-llama3.2-3b-instruct-unsloth
Organizations
None yet
kpi's activity
updated
a
model
about 1 month ago
published
a
model
about 1 month ago
reacted to
smangrul's
post with 🔥
12 months ago
Post
3643
Unlocking the Power of locally running Llama-3 8B Model Agents with Chat-UI! 🔥🚀✨
I'm thrilled to share my hackathon-style side project:
1. Finetuning Llama-8B for function calling using PEFT QLoRA as the instruct Llama-3 model doesn't support this. The colab notebook for it is here: https://lnkd.in/ggJMzqh2. 🛠️
2. Finetuned model along with the 4-bit quants here: https://lnkd.in/gNpFKY6V ✨
3. Clone Hugging Face https://lnkd.in/gKBKuUBQ and make it compatible for function calling by building upon the PR https://lnkd.in/gnqFuAd4 for my model and local inferencing usecase using Ollama. This was a steep learning curve wherein I stayed awake the whole night to get it working. 💪🏽
4. Above, I used SerpAPI for web browsing and Mongo DB Atlas free tier for persistence of conversations and assistant configs. 🔎
5. More work is required to switch between using tools and responding directly wherein I see the model breaks. 🧐
How cool is this wherein we are approaching experience akin to ChatGPT while using local hosted agent model running on your laptop! 💻
I'm thrilled to share my hackathon-style side project:
1. Finetuning Llama-8B for function calling using PEFT QLoRA as the instruct Llama-3 model doesn't support this. The colab notebook for it is here: https://lnkd.in/ggJMzqh2. 🛠️
2. Finetuned model along with the 4-bit quants here: https://lnkd.in/gNpFKY6V ✨
3. Clone Hugging Face https://lnkd.in/gKBKuUBQ and make it compatible for function calling by building upon the PR https://lnkd.in/gnqFuAd4 for my model and local inferencing usecase using Ollama. This was a steep learning curve wherein I stayed awake the whole night to get it working. 💪🏽
4. Above, I used SerpAPI for web browsing and Mongo DB Atlas free tier for persistence of conversations and assistant configs. 🔎
5. More work is required to switch between using tools and responding directly wherein I see the model breaks. 🧐
How cool is this wherein we are approaching experience akin to ChatGPT while using local hosted agent model running on your laptop! 💻
- 1 reply
updated
a
model
12 months ago
is INSTRUCTOR embeddings compatible with LLAMA2?
1
#16 opened over 1 year ago
by
iAFuisMe1234
Simple colab notebook
4
#20 opened over 1 year ago
by
matt1452