Update README.md
Browse files
README.md
CHANGED
|
@@ -2,4 +2,6 @@
|
|
| 2 |
license: apache-2.0
|
| 3 |
---
|
| 4 |
|
| 5 |
-
StarCoder finetuned on 20k AlpacaCode Dataset.
|
|
|
|
|
|
|
|
|
| 2 |
license: apache-2.0
|
| 3 |
---
|
| 4 |
|
| 5 |
+
StarCoder finetuned on 20k AlpacaCode Dataset.
|
| 6 |
+
|
| 7 |
+
The StarCoder models are 15.5B parameter models trained on 80+ programming languages from The Stack (v1.2), with opt-out requests excluded. The model uses Multi Query Attention, a context window of 8192 tokens, and was trained using the Fill-in-the-Middle objective on 1 trillion tokens.
|