Update README.md
Browse files
README.md
CHANGED
@@ -13,7 +13,7 @@ base_model:
|
|
13 |
## Model summary
|
14 |
|
15 |
This model is part of the 📐 [FineMath](https://huggingface.co/datasets/HuggingFaceTB/finemath) ablations, we continue pretraining [Llama-3.2-3B](https://huggingface.co/meta-llama/Llama-3.2-3B) base on different math datasets for 60B tokens.
|
16 |
-
The model has 3.21B parameters and 4096 context length. It was trained on 60B tokens from [FineWeb-Edu](https://huggingface.co/datasets/HuggingFaceFW/fineweb-edu), tokenized using `llama3` tokenizer.
|
17 |
|
18 |
- **License**: Apache-2
|
19 |
- **Languages**: English
|
|
|
13 |
## Model summary
|
14 |
|
15 |
This model is part of the 📐 [FineMath](https://huggingface.co/datasets/HuggingFaceTB/finemath) ablations, we continue pretraining [Llama-3.2-3B](https://huggingface.co/meta-llama/Llama-3.2-3B) base on different math datasets for 60B tokens.
|
16 |
+
The model has 3.21B parameters and 4096 context length. It was trained on **60B tokens** from [FineWeb-Edu](https://huggingface.co/datasets/HuggingFaceFW/fineweb-edu), tokenized using `llama3` tokenizer.
|
17 |
|
18 |
- **License**: Apache-2
|
19 |
- **Languages**: English
|