Update README.md
Browse files
README.md
CHANGED
|
@@ -12,7 +12,7 @@ The model was trained with a learning rate of 1e-4, with a warmup of 1024 steps,
|
|
| 12 |
|
| 13 |
The resulting model achieves a puplexity of 339.38, making it competative with Cerebras-590m with only 21% of the parameters, and much better than the original GPT-2 which scores 491.57!
|
| 14 |
|
| 15 |
-
(metric explanation here: https://twitter.com/aicrumb/status/1650350363898265601 , tldr it's a joke
|
| 16 |
|
| 17 |
*(from GPT-2 model card)*
|
| 18 |
|
|
|
|
| 12 |
|
| 13 |
The resulting model achieves a puplexity of 339.38, making it competative with Cerebras-590m with only 21% of the parameters, and much better than the original GPT-2 which scores 491.57!
|
| 14 |
|
| 15 |
+
(metric explanation here: https://twitter.com/aicrumb/status/1650350363898265601 , tldr it's a joke but only kind of)
|
| 16 |
|
| 17 |
*(from GPT-2 model card)*
|
| 18 |
|