yhavinga commited on
Commit
cd84b33
1 Parent(s): 6f7461f

Saving weights and logs at step 320000

Browse files
README.md CHANGED
@@ -14,6 +14,8 @@ datasets:
14
  ---
15
  # GPT2-Medium pre-trained on cleaned Dutch mC4 🇳🇱
16
 
 
 
17
  Dataset:
18
 
19
  * [mC4 NL Cleaned](https://huggingface.co/datasets/yhavinga/mc4_nl_cleaned)
@@ -26,7 +28,7 @@ Tokenizer:
26
 
27
  Training details:
28
 
29
- * Trained for 280k steps (30 dec 2021)
30
  * Block size: 512
31
  * Optimizer: adam, lr 8e-4, beta1 0.9, beta2 0.98
32
  * Warmup steps: 5000
 
14
  ---
15
  # GPT2-Medium pre-trained on cleaned Dutch mC4 🇳🇱
16
 
17
+ Training is not finished!
18
+
19
  Dataset:
20
 
21
  * [mC4 NL Cleaned](https://huggingface.co/datasets/yhavinga/mc4_nl_cleaned)
 
28
 
29
  Training details:
30
 
31
+ * Trained for 320K of 520K steps (31 dec 2021)
32
  * Block size: 512
33
  * Optimizer: adam, lr 8e-4, beta1 0.9, beta2 0.98
34
  * Warmup steps: 5000
flax_model.msgpack CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:d2bc942466bedf81fea88c9bbeaaafa7dfb2fec485a78c89c52705b841a2bf0a
3
  size 1419302302
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a5b7d6e0a2d7b5a04dbf348d2196ad0fc3797612d0a846685cea7685fb916543
3
  size 1419302302
runs/events.out.tfevents.1640332964.t1v-n-f9cfcc28-w-0.384322.0.v2 CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:c60bb6a82a55ceae1859f8fc81e83b0c19ee72e64de5ecdc95e012746328f4c6
3
- size 43681985
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:78a17fcd88365f6bcb23b316b06024faf91960c419d1cab9d39b055a0ae3fcb8
3
+ size 49426035