🐕Small-Doges
Collection
Doge family of small language models!
•
18 items
•
Updated
•
6
AlbertShi
AlbertShi
AI & ML interests
None yet
Recent Activity
upvoted
a
collection
23 days ago
🐕Small-Doges
liked
a model
23 days ago
SmallDoge/Doge-160M
liked
a model
23 days ago
SmallDoge/Doge-320M
Organizations
AlbertShi's activity
upvoted
a
collection
23 days ago
liked
3
models
23 days ago
liked
a
dataset
about 1 month ago
reacted to
JingzeShi's
post with 🚀
about 1 month ago
Post
4731
We distill a more accurate and concise dataset from DeepSeek R1, and also provide a distillation pipeline code repository.🤗
Dataset: SmallDoge/SmallThoughts
Code: https://github.com/SmallDoges/small-thoughts
Dataset: SmallDoge/SmallThoughts
Code: https://github.com/SmallDoges/small-thoughts
liked
a
model
2 months ago
upvoted
a
paper
3 months ago
liked
4
models
3 months ago
upvoted
a
collection
3 months ago
liked
3
models
3 months ago
reacted to
JingzeShi's
post with 👍🤯👀
3 months ago
Post
2078
Only a single RTX 4090 running model pre-training is really slow, even for small language models!!! (https://huggingface.co/collections/JingzeShi/doge-slm-677fd879f8c4fd0f43e05458)
- 2 replies
reacted to
JingzeShi's
post with 🔥
3 months ago
Post
1720
🤩warmup -> stable -> decay leanring rate scheduler:
😎use the Stable Phase CheckPoints to Continue Training the model on Any New Dataset without spikes of the training!!!
SmallDoge/Doge-20M-checkpoint
SmallDoge/Doge-60M-checkpoint
😎use the Stable Phase CheckPoints to Continue Training the model on Any New Dataset without spikes of the training!!!
SmallDoge/Doge-20M-checkpoint
SmallDoge/Doge-60M-checkpoint
-
-
- 4 replies