Article
Timm ❤️ Transformers: Use any timm model with transformers
•
30
IndustryAI
AI4Industry
AI & ML interests
None yet
Recent Activity
upvoted
an
article
2 days ago
Timm ❤️ Transformers: Use any timm model with transformers
commented on
a paper
about 1 month ago
MolParser: End-to-end Visual Recognition of Molecule Structures in the
Wild
upvoted
a
paper
about 1 month ago
MolParser: End-to-end Visual Recognition of Molecule Structures in the
Wild
Organizations
None yet
AI4Industry's activity
upvoted
an
article
2 days ago
commented
a
paper
about 1 month ago
upvoted
a
paper
about 1 month ago
updated
a
dataset
3 months ago
liked
a
dataset
3 months ago
upvoted
a
collection
4 months ago
reacted to
rwightman's
post with ❤️
4 months ago
Post
2529
A 'small' MobileNet-V4 update, I just pushed weights for the smallest model I've trained in the series, a 0.5 width multiplier version of the MobileNet-V4 Conv Small.
Now you may look at this and say hey, why is this impressive? 64.8% top-1 and 2.2M params? MobileNetV3-Small 0.75, and MobileNet-V2 0.5 are both fewer params (at ~2M) and over 65% top-1, what gives? Well this is where MobileNet-V4 differs from the previous versions of the model family, it trades off (gives up) a little parameter efficiency for some computational efficiency.
So, let's look at the speed. On a 4090 w/ torchcompile
* 98K img/sec - timm/mobilenetv4_conv_small_050.e3000_r224_in1k
* 58K img/sec - timm/mobilenetv3_small_075.lamb_in1k
* 37K img/sec - timm/mobilenetv2_050.lamb_in1k
And there you go, if you have a need for speed, MNV4 is the better option.
Now you may look at this and say hey, why is this impressive? 64.8% top-1 and 2.2M params? MobileNetV3-Small 0.75, and MobileNet-V2 0.5 are both fewer params (at ~2M) and over 65% top-1, what gives? Well this is where MobileNet-V4 differs from the previous versions of the model family, it trades off (gives up) a little parameter efficiency for some computational efficiency.
So, let's look at the speed. On a 4090 w/ torchcompile
* 98K img/sec - timm/mobilenetv4_conv_small_050.e3000_r224_in1k
* 58K img/sec - timm/mobilenetv3_small_075.lamb_in1k
* 37K img/sec - timm/mobilenetv2_050.lamb_in1k
And there you go, if you have a need for speed, MNV4 is the better option.
upvoted
a
collection
4 months ago
reacted to
merve's
post with 👍
4 months ago
Post
3848
If you have documents that do not only have text and you're doing retrieval or RAG (using OCR and LLMs), give it up and give ColPali and vision language models a try 🤗
Why? Documents consist of multiple modalities: layout, table, text, chart, images. Document processing pipelines often consist of multiple models and they're immensely brittle and slow. 🥲
How? ColPali is a ColBERT-like document retrieval model built on PaliGemma, it operates over image patches directly, and indexing takes far less time with more accuracy. You can use it for retrieval, and if you want to do retrieval augmented generation, find the closest document, and do not process it, give it directly to a VLM like Qwen2-VL (as image input) and give your text query. 🤝
This is much faster + you do not lose out on any information + much easier to maintain too! 🥳
Multimodal RAG merve/multimodal-rag-66d97602e781122aae0a5139 💬
Document AI (made it way before, for folks who want structured input/output and can fine-tune a model) merve/awesome-document-ai-65ef1cdc2e97ef9cc85c898e 📖
Why? Documents consist of multiple modalities: layout, table, text, chart, images. Document processing pipelines often consist of multiple models and they're immensely brittle and slow. 🥲
How? ColPali is a ColBERT-like document retrieval model built on PaliGemma, it operates over image patches directly, and indexing takes far less time with more accuracy. You can use it for retrieval, and if you want to do retrieval augmented generation, find the closest document, and do not process it, give it directly to a VLM like Qwen2-VL (as image input) and give your text query. 🤝
This is much faster + you do not lose out on any information + much easier to maintain too! 🥳
Multimodal RAG merve/multimodal-rag-66d97602e781122aae0a5139 💬
Document AI (made it way before, for folks who want structured input/output and can fine-tune a model) merve/awesome-document-ai-65ef1cdc2e97ef9cc85c898e 📖
- 2 replies
liked
a
dataset
5 months ago
upvoted
a
paper
6 months ago
liked
a
dataset
6 months ago
upvoted
an
article
6 months ago
Article
MobileNet Baselines
By
•
•
23
upvoted
a
collection
6 months ago
liked
a
model
6 months ago
liked
3
models
8 months ago
upvoted
a
collection
8 months ago
liked
a
model
9 months ago
