Adem Boukhris

AdemB

AI & ML interests

None yet

Recent Activity

liked a model 24 days ago

thwin27/Aria-sequential_mlp-bnb_nf4

liked a model about 2 months ago

rhymes-ai/Aria

View all activity

Organizations

None yet

AdemB's activity

liked a model 24 days ago

thwin27/Aria-sequential_mlp-bnb_nf4

Image-Text-to-Text • Updated Oct 23 • 451 • 10

liked a model about 2 months ago

rhymes-ai/Aria

Image-Text-to-Text • Updated 1 day ago • 13.6k • 588

liked a model 3 months ago

AgentPublic/Faust

Text Generation • Updated Nov 6, 2023 • 9 • 2

liked a model 4 months ago

openbmb/MiniCPM-V-2_6

Image-Text-to-Text • Updated 18 days ago • 92.8k • 842

upvoted 2 collections 4 months ago

Enhancer

Collection

Restore and Scaler Tools • 7 items • Updated 1 day ago • 2

Llama 3.1

Collection

This collection hosts the transformers and original repos of the Llama 3.1, Llama Guard 3 and Prompt Guard models • 11 items • Updated Sep 25 • 630

reacted to yushun0410's post with 🚀❤️ 5 months ago

Post

4611

Hi Huggingfacers!

Thrilled to introduce Adam-mini, an optimizer that achieves on-par or better performance than AdamW with 45% to 50% less memory footprint. Adam-mini can also achieve 49.5% higher throughput than AdamW on Llama2-7B pre-training.

The design of Adam-mini is inspired by certain Hessian structures we observed on Transformers.

Feel free to try it out! Try switching to Adam-mini with the same hyperparams of AdamW, it would work with only half memory. Hope Adam-mini can help save time, cost, and energy in your tasks!

Paper: "Adam-mini: Use Fewer Learning Rates To Gain More" https://arxiv.org/abs/2406.16793

Code: https://github.com/zyushun/Adam-mini