28 22 181

Kaizhao Liang

kz919

https://kyleliang919.github.io/

AI & ML interests

Multimodal foundational model

Recent Activity

liked a Space 4 days ago

sambanovasystems/QwQ-32B-preview

updated a Space 4 days ago

sambanovasystems/QwQ-32B-preview

upvoted a paper 8 days ago

On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes

View all activity

Organizations

kz919's activity

liked a Space 4 days ago

Running

🔍

On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes

Paper • 2306.13649 • Published Jun 23, 2023 • 17

liked a Space 8 days ago

Running

194

⚡

meta-llama/Llama-3.3-70B-Instruct

Text Generation • Updated 8 days ago • 213k • • 1.15k

liked a Space 12 days ago

Running

137

🐢

Qwen2.5 Coder Artifacts

liked a Space 12 days ago

Running on Zero

1.58k

🏢

TRELLIS

Scalable and Versatile 3D Generation from images

liked a dataset 12 days ago

amphora/QwQ-LongCoT-130K

Viewer • Updated 13 days ago • 133k • 888 • 70

liked a model 14 days ago

Qwen/Qwen2.5-0.5B-Instruct

Text Generation • Updated Sep 25 • 517k • • 143

reacted to rwightman's post with 🔥🚀 14 days ago

Post

1265

There's a new timm release, v 1.0.12, with a focus on optimizers. The optimizer factory has been refactored, there's now a timm.optim.list_optimizers() and new way to register optimizers and their attributes. As always you can use an timm optimizer like a torch one, just replace torch.optim with timm.optim

New optimizers include:
* AdafactorBigVision - adfactorbv
* ADOPT - adopt / adoptw (decoupled decay)
* MARS - mars
* LaProp - laprop
* Cautious Optimizers - a modification to all of the above, prefix with c as well as cadamw, cnadamw, csgdw, clamb, crmsproptf

I shared some caution comparisons in this model repo: rwightman/timm-optim-caution

For details, references, see the code: https://github.com/huggingface/pytorch-image-models/tree/main/timm/optim