Train custom AI models with the trainer API and adapt them to ๐ค
โข
24
LAin
not-lain
AI & ML interests
custom AI models with HF integration, multimodal rag and open-source software
Articles
Organizations
not-lain's activity
nice work @alvdansen , really loved it โค๏ธ
posted
an
update
8 days ago
Post
1662
It is with great pleasure I inform you that huggingface's ModelHubMixin reached 200+ models on the hub ๐ฅณ
ModelHubMixin is a class developed by HF to integrate AI models with the hub with ease and it comes with 3 methods :
* save_pretrained
* from_pretrained
* push_to_hub
Shoutout to @nielsr , @Wauplin and everyone else on HF for their awesome work ๐ค
If you are not familiar with ModelHubMixin and you are looking for extra resources you might consider :
* docs: https://huggingface.co/docs/huggingface_hub/main/en/package_reference/mixins
๐blog about training models with the trainer API and using ModelHubMixin: https://huggingface.co/blog/not-lain/trainer-api-and-mixin-classes
๐GitHub repo with pip integration: https://github.com/not-lain/PyTorchModelHubMixin-template
๐basic guide: https://huggingface.co/posts/not-lain/884273241241808
ModelHubMixin is a class developed by HF to integrate AI models with the hub with ease and it comes with 3 methods :
* save_pretrained
* from_pretrained
* push_to_hub
Shoutout to @nielsr , @Wauplin and everyone else on HF for their awesome work ๐ค
If you are not familiar with ModelHubMixin and you are looking for extra resources you might consider :
* docs: https://huggingface.co/docs/huggingface_hub/main/en/package_reference/mixins
๐blog about training models with the trainer API and using ModelHubMixin: https://huggingface.co/blog/not-lain/trainer-api-and-mixin-classes
๐GitHub repo with pip integration: https://github.com/not-lain/PyTorchModelHubMixin-template
๐basic guide: https://huggingface.co/posts/not-lain/884273241241808
@m-ric
really nice video, also I really loved the blogpost.
Thanks a lot ๐ค
replied to
their
post
12 days ago
Thanks @Taylor658 ๐ค
posted
an
update
13 days ago
Post
1704
I will be delivering an introductory coding session this Sunday 7Pm gmt+1 time about huggingface, if you are new to HF and don't know where to begin, you are welcome to join us ๐ค
๐Place: huggingface discord server
๐Link : https://discord.gg/hugging-face-879548962464493619?event=1245406127668203541
๐Place: huggingface discord server
๐Link : https://discord.gg/hugging-face-879548962464493619?event=1245406127668203541
-
- 2 replies
@lunarflu don't forget HF unwrapped too, similar to what GitHub, Reddit and other websites do, would be fun to have
replied to
hakunamatata1997's
post
21 days ago
@hakunamatata1997 , some people (including me) sometimes think that an AI is unfit for public use (for example failing the toxicity test) therefore, it is not recommended to keep them.
posted
an
update
25 days ago
Post
1476
If you're a researcher or developing your own model ๐ you might need to take a look at huggingface's ModelHubMixin classes.
They are used to seamlessly integrate your AI model with huggingface and to save/ load your model easily ๐
1๏ธโฃ make sure you're using the appropriate library version
2๏ธโฃ inherit from the appropriate class
4๏ธโฃ push the model to the hub (or use save_pretrained method to save locally)
5๏ธโฃ Load and initialize the model from the hub using the original class
They are used to seamlessly integrate your AI model with huggingface and to save/ load your model easily ๐
1๏ธโฃ make sure you're using the appropriate library version
pip install -qU "huggingface_hub>=0.22"2๏ธโฃ inherit from the appropriate class
from huggingface_hub import PyTorchModelHubMixin
from torch import nn
class MyModel(nn.Module,PyTorchModelHubMixin):
def __init__(self, a, b):
super().__init__()
self.layer = nn.Linear(a,b)
def forward(self,inputs):
return self.layer(inputs)
first_model = MyModel(3,1)4๏ธโฃ push the model to the hub (or use save_pretrained method to save locally)
first_model.push_to_hub("not-lain/test")5๏ธโฃ Load and initialize the model from the hub using the original class
pretrained_model = MyModel.from_pretrained("not-lain/test")
posted
an
update
28 days ago
Post
1082
I'm looking for open-source image embedding models for RAG applications and/or multimodel embedding models if they exist in the first place.
if you have any extra resources about using, creating, or finetuning them feel free to share them below ๐ค
if you have any extra resources about using, creating, or finetuning them feel free to share them below ๐ค
reel
replied to
their
post
about 1 month ago
thank you sooooo much โค๏ธโค๏ธ
posted
an
update
about 1 month ago
Post
1183
๐ฅณcelebrating 5K readers in one of my blog posts ๐ฅณ
I came back with another one this time ๐ค
in this blog you will learn ๐ :
* How to train custom AI models with the trainer API ๐
* integrate your AI models with HF using the mixin classes ๐ฅ
happy reading everyone ๐ค
๐link: https://huggingface.co/blog/not-lain/trainer-api-and-mixin-classes
I came back with another one this time ๐ค
in this blog you will learn ๐ :
* How to train custom AI models with the trainer API ๐
* integrate your AI models with HF using the mixin classes ๐ฅ
happy reading everyone ๐ค
๐link: https://huggingface.co/blog/not-lain/trainer-api-and-mixin-classes
-
- 2 replies
I see you're using the image slider component too, nice work โค๏ธโค๏ธ
Nice work @abhishek โจ
posted
an
update
about 1 month ago
Post
1742
๐ just reached 3K+ readers on this blog post about RAG using only HF๐ค related tools in just a little over 1 week from publishing.
๐the most interesting thing about it is that you can use the FAISS index in the datasets library to retrieve your most similar documents.
๐https://huggingface.co/blog/not-lain/rag-chatbot-using-llama3
Happy reading everyone โจ
๐the most interesting thing about it is that you can use the FAISS index in the datasets library to retrieve your most similar documents.
๐https://huggingface.co/blog/not-lain/rag-chatbot-using-llama3
Happy reading everyone โจ
replied to
VictorSanh's
post
about 1 month ago
Did anyone say HF?
literally on ๐ฅ
Nice work ๐ฅณ๐ฅณ
replied to
HeshamHaroon's
post
about 2 months ago
nope it's not a bug, the posts feature is now open for everyone
posted
an
update
about 2 months ago
Post
2256
๐ฅ **transformers** 4.40.0 is out ๐ฅ
โ๏ธ you can now push your custom pipelines easily to ๐ค, allowing people to easily use your model in a more friendly, unified way.
๐ค I already updated my blog to match the new feature https://huggingface.co/blog/not-lain/custom-architectures-with-huggingface.
๐A list of some repos that have custom pipelines :
* briaai/RMBG-1.4
* p1atdev/siglip-tagger-test-3
* sgugger/test-dynamic-pipeline
โ๏ธ you can now push your custom pipelines easily to ๐ค, allowing people to easily use your model in a more friendly, unified way.
๐ค I already updated my blog to match the new feature https://huggingface.co/blog/not-lain/custom-architectures-with-huggingface.
๐A list of some repos that have custom pipelines :
* briaai/RMBG-1.4
* p1atdev/siglip-tagger-test-3
* sgugger/test-dynamic-pipeline
x3 download speed in HfFileSystem = ๐ฅ๐
I also really loved using the PyTorchModelHubMixin class ๐ค๐ค
try adding labels parameter to the forward function in my version of of the RMBG-1.4 model in here https://huggingface.co/not-lain/CustomCodeForRMBG/blob/main/briarmbg.py#L392, this should allow the model to be finetuned easily using the trainer API.
find out more about the labels parameter in this documentation :
https://huggingface.co/docs/transformers/custom_models#writing-a-custom-model
it should look somethind like this :
any open contributions are welcome.
Made a little contribution and it is now easier than ever to use the model via the transformers library
all you have to do is use the following code to access the model :
checkout more details in this pull request : https://huggingface.co/briaai/RMBG-1.4/discussions/9
or like make it a badge in their profiles if they exceed 100 contributions, could be a nice motive for people to make them engage more in the hub
maybe ๐งโโ๏ธ๐งโโ๏ธ for open sorcerers out there
leaving this for future readers, huggingface has a really nice documentation about RAG which i highly recommend reading if you're interested about using them. It incorporates clear examples and neatly formatted code :
https://huggingface.co/docs/transformers/model_doc/rag
we're actually an anime/meme group and i made this community for AI + anime related stuff \(๏ฟฃ๏ธถ๏ฟฃ*\))
@Tonic
we just reached 300000 members yerterday on FB ๐ฅณ๐ฅณ