Marco Valenti's picture

2 17

Marco Valenti

valentimarco

·

valentimarco

AI & ML interests

Beginner ML

Recent Activity

liked a model about 1 month ago

DeepMount00/Lexora-Medium-7B

Reacted to onekq's post with 🔥 about 1 month ago

I'm now working on finetuning of coding models. If you are GPU-hungry like me, you will find quantized models very helpful. But quantization for finetuning and inference are different and incompatible. So I made two collections here. Inference (GGUF, via Ollama, CPU is enough) https://huggingface.co/collections/onekq-ai/ollama-ready-coding-models-67118c3cfa1af2cf04a926d6 Finetuning (Bitsandbytes, QLora, GPU is needed) https://huggingface.co/collections/onekq-ai/qlora-ready-coding-models-67118771ce001b8f4cf946b2 For quantization, the inference models are far more popular on HF than finetuning models. I use https://huggingface.co/QuantFactory to generate inference models (GGUF), and there are a few other choices. But there hasn't been such a service for finetuning models. DIY isn't too hard though. I made a few myself and you can find the script in the model cards. If the original model is small enough, you can even do it on a free T4 (available via Google Colab). If you know a (small) coding model worthy of quantization, please let me know and I'd love to add it to the collections.

upvoted a collection about 1 month ago

Ollama-ready Coding Models

View all activity

Organizations

None yet

models

None public yet

datasets

None public yet