Update README.md
Browse files
README.md
CHANGED
|
@@ -23,7 +23,7 @@ tags:
|
|
| 23 |
Filiberto 124M Instruct is only 124 million parameters. It can run easily on CPU or provide correction at scale on GPUs (>10k tokens/seconds).
|
| 24 |
|
| 25 |
## Training
|
| 26 |
-
The pre-trained included a collection of individual verses and their correction taken from the TEXORO corpus, totalling ~5 million tokens.
|
| 27 |
|
| 28 |
Pre-training ran on 5 epochs with levanter (500 steps total, each processing 1024 sequences of 512 tokens) on a TPUv4-32 for 15 minutes.
|
| 29 |
|
|
|
|
| 23 |
Filiberto 124M Instruct is only 124 million parameters. It can run easily on CPU or provide correction at scale on GPUs (>10k tokens/seconds).
|
| 24 |
|
| 25 |
## Training
|
| 26 |
+
The pre-trained included a collection of individual verses and their correction taken from the [TEXORO](https://etso.es/texoro) corpus, via a collaboration with [ETSO](https://etso.es/), totalling ~5 million tokens.
|
| 27 |
|
| 28 |
Pre-training ran on 5 epochs with levanter (500 steps total, each processing 1024 sequences of 512 tokens) on a TPUv4-32 for 15 minutes.
|
| 29 |
|