cahya commited on
Commit
c555954
β€’
1 Parent(s): f5d3959

Saving weights and logs of step 40

Browse files
events.out.tfevents.1625840127.t1v-n-528d9406-w-0.245719.3.v2 CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:2e0a75fadea524b6337311333d1421579b055ac7cfbaccaa2e412a2281cc6456
3
- size 460
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:64a1f53cff36f1ceef50b134d2b7ac7ac032f23128f8144fad2f113039c49027
3
+ size 600
flax_model.msgpack CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:e3adb083f796adedcab232ab5df60b012de3f9c74d538b151cd193c51beb278d
3
  size 1419302302
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:42da66777ec7c8efce01daf35cfa7c98bfa70b0dc0f0bff82a13bead069b2f75
3
  size 1419302302
wandb/run-20210709_141445-2k8cnty2/files/output.log CHANGED
@@ -464,4 +464,14 @@ Training...: 58%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ
464
 
465
  Evaluating...: 40%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ | 2/5 [00:05<00:07, 2.46s/it]
466
 
 
 
 
 
 
 
 
 
 
 
467
  tcmalloc: large alloc 2426904576 bytes == 0x56396efea000 @ 0x7faff8cbc680 0x7faff8cdcbdd 0x7faca7dcf20d 0x7faca7ddd340 0x7faca7ddce87 0x7faca7ddce87 0x7faca7ddce87 0x7faca7dd8bd3 0x7faca7dd91fe 0x56364a10452a 0x56364a0cb254 0x56364a11c9c9 0x56364a11d85f 0x56364a0cacc7 0x56364a11d768 0x56364a0cae14 0x56364a11d768 0x56364a0cae14 0x56364a11d066 0x56364a1284c6 0x56364a0cacc7 0x56364a11d132 0x56364a11d7fb 0x56364a0cae14 0x56364a11c9c9 0x56364a17c383 0x56364a17c8d2 0x56364a18c982 0x56364a18e429 0x56364a18e60f 0x56364a18ea09
 
464
 
465
  Evaluating...: 40%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ | 2/5 [00:05<00:07, 2.46s/it]
466
 
467
+ tcmalloc: large alloc 2426904576 bytes == 0x56396efea000 @ 0x7faff8cbc680 0x7faff8cdcbdd 0x7faca7dcf20d 0x7faca7ddd340 0x7faca7ddce87 0x7faca7ddce87 0x7faca7ddce87 0x7faca7dd8bd3 0x7faca7dd91fe 0x56364a10452a 0x56364a0cb254 0x56364a11c9c9 0x56364a11d85f 0x56364a0cacc7 0x56364a11d768 0x56364a0cae14 0x56364a11d768 0x56364a0cae14 0x56364a11d066 0x56364a1284c6 0x56364a0cacc7 0x56364a11d132 0x56364a11d7fb 0x56364a0cae14 0x56364a11c9c9 0x56364a17c383 0x56364a17c8d2 0x56364a18c982 0x56364a18e429 0x56364a18e60f 0x56364a18ea09
468
+ Model weights saved in /home/cahya/Work/flax-community/gpt2-medium-indonesian/flax_model.msgpack
469
+ Model pushed to the hub in this commit: https://huggingface.co/flax-community/gpt2-medium-indonesian/commit/f5d39590d9c85082eb6e239e93942d2dd86cbe3e
470
+
471
+
472
+ Training...: 77%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ | 40/52 [05:00<00:21, 1.76s/it]
473
+
474
+ Evaluating...: 20%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ | 1/5 [00:05<00:22, 5.72s/it]
475
+
476
+
477
  tcmalloc: large alloc 2426904576 bytes == 0x56396efea000 @ 0x7faff8cbc680 0x7faff8cdcbdd 0x7faca7dcf20d 0x7faca7ddd340 0x7faca7ddce87 0x7faca7ddce87 0x7faca7ddce87 0x7faca7dd8bd3 0x7faca7dd91fe 0x56364a10452a 0x56364a0cb254 0x56364a11c9c9 0x56364a11d85f 0x56364a0cacc7 0x56364a11d768 0x56364a0cae14 0x56364a11d768 0x56364a0cae14 0x56364a11d066 0x56364a1284c6 0x56364a0cacc7 0x56364a11d132 0x56364a11d7fb 0x56364a0cae14 0x56364a11c9c9 0x56364a17c383 0x56364a17c8d2 0x56364a18c982 0x56364a18e429 0x56364a18e60f 0x56364a18ea09
wandb/run-20210709_141445-2k8cnty2/files/wandb-summary.json CHANGED
@@ -1 +1 @@
1
- {"global_step": 20, "_timestamp": 1625840258.4632, "eval_loss": 9.361381530761719, "eval_perplexity": 11630.4453125, "_step": 1}
 
1
+ {"global_step": 30, "_timestamp": 1625840355.097392, "eval_loss": 8.853381156921387, "eval_perplexity": 6998.01025390625, "_step": 2}
wandb/run-20210709_141445-2k8cnty2/logs/debug-internal.log CHANGED
@@ -141,3 +141,30 @@
141
  2021-07-09 14:19:35,410 INFO Thread-8 :246776 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/cahya/Work/flax-community/gpt2-medium-indonesian/wandb/run-20210709_141445-2k8cnty2/files/output.log
142
  2021-07-09 14:19:37,819 DEBUG HandlerThread:246776 [handler.py:handle_request():124] handle_request: stop_status
143
  2021-07-09 14:19:37,819 DEBUG SenderThread:246776 [sender.py:send_request():193] send_request: stop_status
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
141
  2021-07-09 14:19:35,410 INFO Thread-8 :246776 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/cahya/Work/flax-community/gpt2-medium-indonesian/wandb/run-20210709_141445-2k8cnty2/files/output.log
142
  2021-07-09 14:19:37,819 DEBUG HandlerThread:246776 [handler.py:handle_request():124] handle_request: stop_status
143
  2021-07-09 14:19:37,819 DEBUG SenderThread:246776 [sender.py:send_request():193] send_request: stop_status
144
+ 2021-07-09 14:19:46,259 DEBUG SenderThread:246776 [sender.py:send():179] send: stats
145
+ 2021-07-09 14:19:52,949 DEBUG HandlerThread:246776 [handler.py:handle_request():124] handle_request: stop_status
146
+ 2021-07-09 14:19:52,949 DEBUG SenderThread:246776 [sender.py:send_request():193] send_request: stop_status
147
+ 2021-07-09 14:20:08,078 DEBUG HandlerThread:246776 [handler.py:handle_request():124] handle_request: stop_status
148
+ 2021-07-09 14:20:08,078 DEBUG SenderThread:246776 [sender.py:send_request():193] send_request: stop_status
149
+ 2021-07-09 14:20:16,335 DEBUG SenderThread:246776 [sender.py:send():179] send: stats
150
+ 2021-07-09 14:20:23,209 DEBUG HandlerThread:246776 [handler.py:handle_request():124] handle_request: stop_status
151
+ 2021-07-09 14:20:23,209 DEBUG SenderThread:246776 [sender.py:send_request():193] send_request: stop_status
152
+ 2021-07-09 14:20:27,430 INFO Thread-8 :246776 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/cahya/Work/flax-community/gpt2-medium-indonesian/wandb/run-20210709_141445-2k8cnty2/files/output.log
153
+ 2021-07-09 14:20:29,431 INFO Thread-8 :246776 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/cahya/Work/flax-community/gpt2-medium-indonesian/wandb/run-20210709_141445-2k8cnty2/files/output.log
154
+ 2021-07-09 14:20:31,432 INFO Thread-8 :246776 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/cahya/Work/flax-community/gpt2-medium-indonesian/wandb/run-20210709_141445-2k8cnty2/files/output.log
155
+ 2021-07-09 14:20:33,433 INFO Thread-8 :246776 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/cahya/Work/flax-community/gpt2-medium-indonesian/wandb/run-20210709_141445-2k8cnty2/files/output.log
156
+ 2021-07-09 14:20:37,434 INFO Thread-8 :246776 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/cahya/Work/flax-community/gpt2-medium-indonesian/wandb/run-20210709_141445-2k8cnty2/files/output.log
157
+ 2021-07-09 14:20:37,965 DEBUG SenderThread:246776 [sender.py:send():179] send: history
158
+ 2021-07-09 14:20:37,965 DEBUG SenderThread:246776 [sender.py:send():179] send: summary
159
+ 2021-07-09 14:20:37,966 INFO SenderThread:246776 [sender.py:_save_file():841] saving file wandb-summary.json with policy end
160
+ 2021-07-09 14:20:38,340 DEBUG HandlerThread:246776 [handler.py:handle_request():124] handle_request: stop_status
161
+ 2021-07-09 14:20:38,340 DEBUG SenderThread:246776 [sender.py:send_request():193] send_request: stop_status
162
+ 2021-07-09 14:20:38,435 INFO Thread-8 :246776 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/cahya/Work/flax-community/gpt2-medium-indonesian/wandb/run-20210709_141445-2k8cnty2/files/events.out.tfevents.1625840127.t1v-n-528d9406-w-0.245719.3.v2
163
+ 2021-07-09 14:20:38,435 INFO Thread-8 :246776 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/cahya/Work/flax-community/gpt2-medium-indonesian/wandb/run-20210709_141445-2k8cnty2/files/wandb-summary.json
164
+ 2021-07-09 14:20:38,913 INFO Thread-17 :246776 [upload_job.py:push():137] Uploaded file /tmp/tmp4f0a6i2mwandb/i981d3j1-events.out.tfevents.1625840127.t1v-n-528d9406-w-0.245719.3.v2
165
+ 2021-07-09 14:20:39,435 INFO Thread-8 :246776 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/cahya/Work/flax-community/gpt2-medium-indonesian/wandb/run-20210709_141445-2k8cnty2/files/output.log
166
+ 2021-07-09 14:20:43,437 INFO Thread-8 :246776 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/cahya/Work/flax-community/gpt2-medium-indonesian/wandb/run-20210709_141445-2k8cnty2/files/output.log
167
+ 2021-07-09 14:20:46,411 DEBUG SenderThread:246776 [sender.py:send():179] send: stats
168
+ 2021-07-09 14:20:53,491 DEBUG HandlerThread:246776 [handler.py:handle_request():124] handle_request: stop_status
169
+ 2021-07-09 14:20:53,491 DEBUG SenderThread:246776 [sender.py:send_request():193] send_request: stop_status
170
+ 2021-07-09 14:20:57,442 INFO Thread-8 :246776 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/cahya/Work/flax-community/gpt2-medium-indonesian/wandb/run-20210709_141445-2k8cnty2/files/output.log
wandb/run-20210709_141445-2k8cnty2/run-2k8cnty2.wandb CHANGED
Binary files a/wandb/run-20210709_141445-2k8cnty2/run-2k8cnty2.wandb and b/wandb/run-20210709_141445-2k8cnty2/run-2k8cnty2.wandb differ