Update README.md
Browse files
README.md
CHANGED
|
@@ -14,6 +14,19 @@ tags: []
|
|
| 14 |
### Model Description
|
| 15 |
|
| 16 |
<!-- Provide a longer summary of what this model is. -->
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 17 |
|
| 18 |
This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
|
| 19 |
|
|
@@ -102,7 +115,8 @@ Use the code below to get started with the model.
|
|
| 102 |
|
| 103 |
## Evaluation
|
| 104 |
|
| 105 |
-
|
|
|
|
| 106 |
|
| 107 |
### Testing Data, Factors & Metrics
|
| 108 |
|
|
|
|
| 14 |
### Model Description
|
| 15 |
|
| 16 |
<!-- Provide a longer summary of what this model is. -->
|
| 17 |
+
CLASS-IT is a 140M parameter language model based on the LLaMA architecture.
|
| 18 |
+
|
| 19 |
+
The model is first pre-trained for 8 epochs on a cleaned version of the BabyLM Challenge strict track dataset.
|
| 20 |
+
After pre-training, the model is instruction-tuned on two additional datasets (8.7M words total) for 10 epochs :
|
| 21 |
+
|
| 22 |
+
- a conversational dataset derived from Switchboard, and
|
| 23 |
+
|
| 24 |
+
- an educational dataset based on an augmented version of Simple English Wikipedia (to be released soon).
|
| 25 |
+
|
| 26 |
+
The resulting model is designed to support interactive and educational use cases, combining general language understanding with dialog and explanatory capabilities.
|
| 27 |
+
|
| 28 |
+
|
| 29 |
+
|
| 30 |
|
| 31 |
This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
|
| 32 |
|
|
|
|
| 115 |
|
| 116 |
## Evaluation
|
| 117 |
|
| 118 |
+
The model has been submitted to the 2025 BabyLM Challenge – Interaction Track:
|
| 119 |
+
https://huggingface.co/spaces/BabyLM-community/babylm-leaderboard-2025-all-tasks
|
| 120 |
|
| 121 |
### Testing Data, Factors & Metrics
|
| 122 |
|