Update README.md
Browse files
README.md
CHANGED
@@ -8,5 +8,5 @@ base_model:
|
|
8 |
---
|
9 |
Model currently under training. Official publish will be on (DD/MM/YYYY) 23/02/2025.
|
10 |
SaplingDream is a 0.5B parameter small GPT based on [Qwen/Qwen2.5-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct) finetuned on reasoning datasets with very high caution to ensure a high-quality sapling model—hence "SaplingDream".
|
11 |
-
The base model is finetuned using SGD to ensure better generalisation, in combination with the lr scheduler with a starting lr of 1e-4. Better safe than sorry, we hope the model picks up on not only the tokens but also on how to actually reason through a problem.
|
12 |
We're using [open-thoughts/OpenThoughts-114k](https://huggingface.co/datasets/open-thoughts/OpenThoughts-114k) and [prithivMLmods/Deepthink-Reasoning-Ins](https://huggingface.co/datasets/prithivMLmods/Deepthink-Reasoning-Ins) for training through an entire epoch.
|
|
|
8 |
---
|
9 |
Model currently under training. Official publish will be on (DD/MM/YYYY) 23/02/2025.
|
10 |
SaplingDream is a 0.5B parameter small GPT based on [Qwen/Qwen2.5-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct) finetuned on reasoning datasets with very high caution to ensure a high-quality sapling model—hence "SaplingDream".
|
11 |
+
The base model is finetuned using SGD to ensure better generalisation, in combination with the lr scheduler "Polynomial" with a starting lr of 1e-4. Better safe than sorry, we hope the model picks up on not only the tokens but also on how to actually reason through a problem.
|
12 |
We're using [open-thoughts/OpenThoughts-114k](https://huggingface.co/datasets/open-thoughts/OpenThoughts-114k) and [prithivMLmods/Deepthink-Reasoning-Ins](https://huggingface.co/datasets/prithivMLmods/Deepthink-Reasoning-Ins) for training through an entire epoch.
|