Commit
•
9face7b
1
Parent(s):
9b55035
update model
Browse files- README.md +1 -1
- assets/train_loss.png +0 -0
- model.safetensors +1 -1
README.md
CHANGED
|
@@ -22,7 +22,7 @@ with an interface for interpretability and control.
|
|
| 22 |
This model was trained on the [OpenWebText](https://huggingface.co/datasets/openwebtext) corpus.
|
| 23 |
## Training Procedure
|
| 24 |
|
| 25 |
-
This model was trained for
|
| 26 |
|
| 27 |
# Environmental Impact
|
| 28 |
|
|
|
|
| 22 |
This model was trained on the [OpenWebText](https://huggingface.co/datasets/openwebtext) corpus.
|
| 23 |
## Training Procedure
|
| 24 |
|
| 25 |
+
This model was trained for 450k gradient steps and cosine decaying learning rate from 1e-4 to zero, with a linear warmup of 5k steps.
|
| 26 |
|
| 27 |
# Environmental Impact
|
| 28 |
|
assets/train_loss.png
CHANGED
|
|
model.safetensors
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 5666995936
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:95cc71c2988c836e4822f848b0a0f201131404e32ae6c09015b191571e2ab648
|
| 3 |
size 5666995936
|