nicholasKluge
commited on
Commit
•
183333a
1
Parent(s):
3a5db13
Update README.md
Browse files
README.md
CHANGED
@@ -48,7 +48,7 @@ Also, TeenyTinyLlama models were trained by leveraging [scaling laws](https://ar
|
|
48 |
## Details
|
49 |
|
50 |
- **Architecture:** a Transformer-based model pre-trained via causal language modeling
|
51 |
-
- **Size:** 162,417,408
|
52 |
- **Context length:** 2048 tokens
|
53 |
- **Dataset:** [Portuguese-Corpus-v3](https://huggingface.co/datasets/nicholasKluge/portuguese-corpus-v3) (6.2B tokens)
|
54 |
- **Language:** Portuguese
|
@@ -159,12 +159,12 @@ for i, completion in enumerate(completions):
|
|
159 |
|
160 |
## Evaluations
|
161 |
|
162 |
-
| Steps | Evaluation Loss | Perplexity | Total Energy Consumption | Emissions
|
163 |
-
|
164 |
-
| 100.000 | 3.19 | 24.52 | 3.75 kWh | 1.28
|
165 |
-
| 200.000 | 3.02 | 20.58 | 7.51 kWh | 2.56
|
166 |
-
| 300.000 | 2.83 | 16.98 | 11.25 kWh | 3.84
|
167 |
-
| 400.000 | 2.79 | 16.41 | 14.52 kWh | 5.11
|
168 |
|
169 |
## Benchmarks
|
170 |
|
|
|
48 |
## Details
|
49 |
|
50 |
- **Architecture:** a Transformer-based model pre-trained via causal language modeling
|
51 |
+
- **Size:** 162,417,408 parameters
|
52 |
- **Context length:** 2048 tokens
|
53 |
- **Dataset:** [Portuguese-Corpus-v3](https://huggingface.co/datasets/nicholasKluge/portuguese-corpus-v3) (6.2B tokens)
|
54 |
- **Language:** Portuguese
|
|
|
159 |
|
160 |
## Evaluations
|
161 |
|
162 |
+
| Steps | Evaluation Loss | Perplexity | Total Energy Consumption | Emissions |
|
163 |
+
|---------|-----------------|------------|--------------------------|--------------|
|
164 |
+
| 100.000 | 3.19 | 24.52 | 3.75 kWh | 1.28 KgCO2eq |
|
165 |
+
| 200.000 | 3.02 | 20.58 | 7.51 kWh | 2.56 KgCO2eq |
|
166 |
+
| 300.000 | 2.83 | 16.98 | 11.25 kWh | 3.84 KgCO2eq |
|
167 |
+
| 400.000 | 2.79 | 16.41 | 14.52 kWh | 5.11 KgCO2eq |
|
168 |
|
169 |
## Benchmarks
|
170 |
|