Hugging Face's logo

flax-community
/
pino-bigbird-roberta-base

Fill-Mask
Transformers
PyTorch
JAX
TensorBoard
Dutch
big_bird
Inference Endpoints
Model card Files Metrics Community
2
pino-bigbird-roberta-base / wandb /run-20210714_183304-3d4rorca /logs /debug-internal.log
dat
Saving weights and logs at step 1252
f291f93
raw history blame contribute delete
No virus
26.7 kB
2021-07-14 18:33:05,441 INFO MainThread:551470 [internal.py:wandb_internal():88] W&B internal server running at pid: 551470, started at: 2021-07-14 18:33:05.441281
2021-07-14 18:33:05,443 DEBUG HandlerThread:551470 [handler.py:handle_request():124] handle_request: check_version
2021-07-14 18:33:05,443 INFO WriterThread:551470 [datastore.py:open_for_write():80] open: /home/dat/pino-roberta-base/wandb/run-20210714_183304-3d4rorca/run-3d4rorca.wandb
2021-07-14 18:33:05,444 DEBUG SenderThread:551470 [sender.py:send():179] send: header
2021-07-14 18:33:05,444 DEBUG SenderThread:551470 [sender.py:send_request():193] send_request: check_version
2021-07-14 18:33:05,481 DEBUG SenderThread:551470 [sender.py:send():179] send: run
2021-07-14 18:33:05,658 INFO SenderThread:551470 [dir_watcher.py:__init__():168] watching files in: /home/dat/pino-roberta-base/wandb/run-20210714_183304-3d4rorca/files
2021-07-14 18:33:05,658 INFO SenderThread:551470 [sender.py:_start_run_threads():716] run started: 3d4rorca with start time 1626287584
2021-07-14 18:33:05,658 DEBUG SenderThread:551470 [sender.py:send():179] send: summary
2021-07-14 18:33:05,659 INFO SenderThread:551470 [sender.py:_save_file():841] saving file wandb-summary.json with policy end
2021-07-14 18:33:05,659 DEBUG HandlerThread:551470 [handler.py:handle_request():124] handle_request: run_start
2021-07-14 18:33:06,662 INFO Thread-8 :551470 [dir_watcher.py:_on_file_created():216] file/dir created: /home/dat/pino-roberta-base/wandb/run-20210714_183304-3d4rorca/files/wandb-summary.json
2021-07-14 18:33:06,772 DEBUG HandlerThread:551470 [meta.py:__init__():39] meta init
2021-07-14 18:33:06,772 DEBUG HandlerThread:551470 [meta.py:__init__():53] meta init done
2021-07-14 18:33:06,772 DEBUG HandlerThread:551470 [meta.py:probe():210] probe
2021-07-14 18:33:06,773 DEBUG HandlerThread:551470 [meta.py:_setup_git():200] setup git
2021-07-14 18:33:06,799 DEBUG HandlerThread:551470 [meta.py:_setup_git():207] setup git done
2021-07-14 18:33:06,799 DEBUG HandlerThread:551470 [meta.py:_save_pip():57] save pip
2021-07-14 18:33:06,799 DEBUG HandlerThread:551470 [meta.py:_save_pip():71] save pip done
2021-07-14 18:33:06,799 DEBUG HandlerThread:551470 [meta.py:probe():252] probe done
2021-07-14 18:33:06,802 DEBUG SenderThread:551470 [sender.py:send():179] send: files
2021-07-14 18:33:06,802 INFO SenderThread:551470 [sender.py:_save_file():841] saving file wandb-metadata.json with policy now
2021-07-14 18:33:06,808 DEBUG HandlerThread:551470 [handler.py:handle_request():124] handle_request: stop_status
2021-07-14 18:33:06,808 DEBUG SenderThread:551470 [sender.py:send_request():193] send_request: stop_status
2021-07-14 18:33:06,939 DEBUG SenderThread:551470 [sender.py:send():179] send: config
2021-07-14 18:33:06,940 DEBUG SenderThread:551470 [sender.py:send():179] send: config
2021-07-14 18:33:06,940 DEBUG SenderThread:551470 [sender.py:send():179] send: config
2021-07-14 18:33:07,267 INFO Thread-11 :551470 [upload_job.py:push():137] Uploaded file /tmp/tmpwkna8gakwandb/1910yhst-wandb-metadata.json
2021-07-14 18:33:07,661 INFO Thread-8 :551470 [dir_watcher.py:_on_file_created():216] file/dir created: /home/dat/pino-roberta-base/wandb/run-20210714_183304-3d4rorca/files/output.log
2021-07-14 18:33:07,661 INFO Thread-8 :551470 [dir_watcher.py:_on_file_created():216] file/dir created: /home/dat/pino-roberta-base/wandb/run-20210714_183304-3d4rorca/files/requirements.txt
2021-07-14 18:33:07,661 INFO Thread-8 :551470 [dir_watcher.py:_on_file_created():216] file/dir created: /home/dat/pino-roberta-base/wandb/run-20210714_183304-3d4rorca/files/wandb-metadata.json
2021-07-14 18:33:21,941 DEBUG HandlerThread:551470 [handler.py:handle_request():124] handle_request: stop_status
2021-07-14 18:33:21,941 DEBUG SenderThread:551470 [sender.py:send_request():193] send_request: stop_status
2021-07-14 18:33:25,667 INFO Thread-8 :551470 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_183304-3d4rorca/files/output.log
2021-07-14 18:33:34,849 DEBUG SenderThread:551470 [sender.py:send():179] send: stats
2021-07-14 18:33:36,671 INFO Thread-8 :551470 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_183304-3d4rorca/files/config.yaml
2021-07-14 18:33:37,073 DEBUG HandlerThread:551470 [handler.py:handle_request():124] handle_request: stop_status
2021-07-14 18:33:37,073 DEBUG SenderThread:551470 [sender.py:send_request():193] send_request: stop_status
2021-07-14 18:33:52,207 DEBUG HandlerThread:551470 [handler.py:handle_request():124] handle_request: stop_status
2021-07-14 18:33:52,207 DEBUG SenderThread:551470 [sender.py:send_request():193] send_request: stop_status
2021-07-14 18:34:04,916 DEBUG SenderThread:551470 [sender.py:send():179] send: stats
2021-07-14 18:34:07,343 DEBUG HandlerThread:551470 [handler.py:handle_request():124] handle_request: stop_status
2021-07-14 18:34:07,343 DEBUG SenderThread:551470 [sender.py:send_request():193] send_request: stop_status
2021-07-14 18:34:22,477 DEBUG HandlerThread:551470 [handler.py:handle_request():124] handle_request: stop_status
2021-07-14 18:34:22,477 DEBUG SenderThread:551470 [sender.py:send_request():193] send_request: stop_status
2021-07-14 18:34:34,976 DEBUG SenderThread:551470 [sender.py:send():179] send: stats
2021-07-14 18:34:37,613 DEBUG HandlerThread:551470 [handler.py:handle_request():124] handle_request: stop_status
2021-07-14 18:34:37,613 DEBUG SenderThread:551470 [sender.py:send_request():193] send_request: stop_status
2021-07-14 18:34:52,747 DEBUG HandlerThread:551470 [handler.py:handle_request():124] handle_request: stop_status
2021-07-14 18:34:52,747 DEBUG SenderThread:551470 [sender.py:send_request():193] send_request: stop_status
2021-07-14 18:35:05,044 DEBUG SenderThread:551470 [sender.py:send():179] send: stats
2021-07-14 18:35:07,878 DEBUG HandlerThread:551470 [handler.py:handle_request():124] handle_request: stop_status
2021-07-14 18:35:07,879 DEBUG SenderThread:551470 [sender.py:send_request():193] send_request: stop_status
2021-07-14 18:35:23,013 DEBUG HandlerThread:551470 [handler.py:handle_request():124] handle_request: stop_status
2021-07-14 18:35:23,013 DEBUG SenderThread:551470 [sender.py:send_request():193] send_request: stop_status
2021-07-14 18:35:35,110 DEBUG SenderThread:551470 [sender.py:send():179] send: stats
2021-07-14 18:35:38,147 DEBUG HandlerThread:551470 [handler.py:handle_request():124] handle_request: stop_status
2021-07-14 18:35:38,148 DEBUG SenderThread:551470 [sender.py:send_request():193] send_request: stop_status
2021-07-14 18:35:53,282 DEBUG HandlerThread:551470 [handler.py:handle_request():124] handle_request: stop_status
2021-07-14 18:35:53,282 DEBUG SenderThread:551470 [sender.py:send_request():193] send_request: stop_status
2021-07-14 18:36:05,181 DEBUG SenderThread:551470 [sender.py:send():179] send: stats
2021-07-14 18:36:08,416 DEBUG HandlerThread:551470 [handler.py:handle_request():124] handle_request: stop_status
2021-07-14 18:36:08,417 DEBUG SenderThread:551470 [sender.py:send_request():193] send_request: stop_status
2021-07-14 18:36:23,549 DEBUG HandlerThread:551470 [handler.py:handle_request():124] handle_request: stop_status
2021-07-14 18:36:23,549 DEBUG SenderThread:551470 [sender.py:send_request():193] send_request: stop_status
2021-07-14 18:36:35,251 DEBUG SenderThread:551470 [sender.py:send():179] send: stats
2021-07-14 18:36:38,681 DEBUG HandlerThread:551470 [handler.py:handle_request():124] handle_request: stop_status
2021-07-14 18:36:38,681 DEBUG SenderThread:551470 [sender.py:send_request():193] send_request: stop_status
2021-07-14 18:36:53,817 DEBUG HandlerThread:551470 [handler.py:handle_request():124] handle_request: stop_status
2021-07-14 18:36:53,817 DEBUG SenderThread:551470 [sender.py:send_request():193] send_request: stop_status
2021-07-14 18:37:05,320 DEBUG SenderThread:551470 [sender.py:send():179] send: stats
2021-07-14 18:37:08,952 DEBUG HandlerThread:551470 [handler.py:handle_request():124] handle_request: stop_status
2021-07-14 18:37:08,953 DEBUG SenderThread:551470 [sender.py:send_request():193] send_request: stop_status
2021-07-14 18:37:24,087 DEBUG HandlerThread:551470 [handler.py:handle_request():124] handle_request: stop_status
2021-07-14 18:37:24,087 DEBUG SenderThread:551470 [sender.py:send_request():193] send_request: stop_status
2021-07-14 18:37:35,392 DEBUG SenderThread:551470 [sender.py:send():179] send: stats
2021-07-14 18:37:39,221 DEBUG HandlerThread:551470 [handler.py:handle_request():124] handle_request: stop_status
2021-07-14 18:37:39,222 DEBUG SenderThread:551470 [sender.py:send_request():193] send_request: stop_status
2021-07-14 18:37:54,352 DEBUG HandlerThread:551470 [handler.py:handle_request():124] handle_request: stop_status
2021-07-14 18:37:54,353 DEBUG SenderThread:551470 [sender.py:send_request():193] send_request: stop_status
2021-07-14 18:38:05,464 DEBUG SenderThread:551470 [sender.py:send():179] send: stats
2021-07-14 18:38:09,485 DEBUG HandlerThread:551470 [handler.py:handle_request():124] handle_request: stop_status
2021-07-14 18:38:09,485 DEBUG SenderThread:551470 [sender.py:send_request():193] send_request: stop_status
2021-07-14 18:38:24,626 DEBUG HandlerThread:551470 [handler.py:handle_request():124] handle_request: stop_status
2021-07-14 18:38:24,626 DEBUG SenderThread:551470 [sender.py:send_request():193] send_request: stop_status
2021-07-14 18:38:35,535 DEBUG SenderThread:551470 [sender.py:send():179] send: stats
2021-07-14 18:38:39,761 DEBUG HandlerThread:551470 [handler.py:handle_request():124] handle_request: stop_status
2021-07-14 18:38:39,761 DEBUG SenderThread:551470 [sender.py:send_request():193] send_request: stop_status
2021-07-14 18:38:54,893 DEBUG HandlerThread:551470 [handler.py:handle_request():124] handle_request: stop_status
2021-07-14 18:38:54,893 DEBUG SenderThread:551470 [sender.py:send_request():193] send_request: stop_status
2021-07-14 18:39:05,597 DEBUG SenderThread:551470 [sender.py:send():179] send: stats
2021-07-14 18:39:10,025 DEBUG HandlerThread:551470 [handler.py:handle_request():124] handle_request: stop_status
2021-07-14 18:39:10,026 DEBUG SenderThread:551470 [sender.py:send_request():193] send_request: stop_status
2021-07-14 18:39:25,164 DEBUG HandlerThread:551470 [handler.py:handle_request():124] handle_request: stop_status
2021-07-14 18:39:25,164 DEBUG SenderThread:551470 [sender.py:send_request():193] send_request: stop_status
2021-07-14 18:39:35,672 DEBUG SenderThread:551470 [sender.py:send():179] send: stats
2021-07-14 18:39:40,296 DEBUG HandlerThread:551470 [handler.py:handle_request():124] handle_request: stop_status
2021-07-14 18:39:40,297 DEBUG SenderThread:551470 [sender.py:send_request():193] send_request: stop_status
2021-07-14 18:39:55,430 DEBUG HandlerThread:551470 [handler.py:handle_request():124] handle_request: stop_status
2021-07-14 18:39:55,430 DEBUG SenderThread:551470 [sender.py:send_request():193] send_request: stop_status
2021-07-14 18:40:05,746 DEBUG SenderThread:551470 [sender.py:send():179] send: stats
2021-07-14 18:40:10,563 DEBUG HandlerThread:551470 [handler.py:handle_request():124] handle_request: stop_status
2021-07-14 18:40:10,564 DEBUG SenderThread:551470 [sender.py:send_request():193] send_request: stop_status
2021-07-14 18:40:25,697 DEBUG HandlerThread:551470 [handler.py:handle_request():124] handle_request: stop_status
2021-07-14 18:40:25,698 DEBUG SenderThread:551470 [sender.py:send_request():193] send_request: stop_status
2021-07-14 18:40:35,821 DEBUG SenderThread:551470 [sender.py:send():179] send: stats
2021-07-14 18:40:40,834 DEBUG HandlerThread:551470 [handler.py:handle_request():124] handle_request: stop_status
2021-07-14 18:40:40,834 DEBUG SenderThread:551470 [sender.py:send_request():193] send_request: stop_status
2021-07-14 18:40:55,966 DEBUG HandlerThread:551470 [handler.py:handle_request():124] handle_request: stop_status
2021-07-14 18:40:55,967 DEBUG SenderThread:551470 [sender.py:send_request():193] send_request: stop_status
2021-07-14 18:41:05,888 DEBUG SenderThread:551470 [sender.py:send():179] send: stats
2021-07-14 18:41:11,114 DEBUG HandlerThread:551470 [handler.py:handle_request():124] handle_request: stop_status
2021-07-14 18:41:11,114 DEBUG SenderThread:551470 [sender.py:send_request():193] send_request: stop_status
2021-07-14 18:41:11,814 INFO Thread-8 :551470 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_183304-3d4rorca/files/output.log
2021-07-14 18:41:26,276 DEBUG HandlerThread:551470 [handler.py:handle_request():124] handle_request: stop_status
2021-07-14 18:41:26,277 DEBUG SenderThread:551470 [sender.py:send_request():193] send_request: stop_status
2021-07-14 18:41:35,966 DEBUG SenderThread:551470 [sender.py:send():179] send: stats
2021-07-14 18:41:41,417 DEBUG HandlerThread:551470 [handler.py:handle_request():124] handle_request: stop_status
2021-07-14 18:41:41,418 DEBUG SenderThread:551470 [sender.py:send_request():193] send_request: stop_status
2021-07-14 18:41:56,548 DEBUG HandlerThread:551470 [handler.py:handle_request():124] handle_request: stop_status
2021-07-14 18:41:56,548 DEBUG SenderThread:551470 [sender.py:send_request():193] send_request: stop_status
2021-07-14 18:42:06,047 DEBUG SenderThread:551470 [sender.py:send():179] send: stats
2021-07-14 18:42:11,683 DEBUG HandlerThread:551470 [handler.py:handle_request():124] handle_request: stop_status
2021-07-14 18:42:11,684 DEBUG SenderThread:551470 [sender.py:send_request():193] send_request: stop_status
2021-07-14 18:42:26,818 DEBUG HandlerThread:551470 [handler.py:handle_request():124] handle_request: stop_status
2021-07-14 18:42:26,818 DEBUG SenderThread:551470 [sender.py:send_request():193] send_request: stop_status
2021-07-14 18:42:36,121 DEBUG SenderThread:551470 [sender.py:send():179] send: stats
2021-07-14 18:42:41,950 DEBUG HandlerThread:551470 [handler.py:handle_request():124] handle_request: stop_status
2021-07-14 18:42:41,951 DEBUG SenderThread:551470 [sender.py:send_request():193] send_request: stop_status
2021-07-14 18:42:51,640 DEBUG SenderThread:551470 [sender.py:send():179] send: telemetry
2021-07-14 18:42:51,641 DEBUG SenderThread:551470 [sender.py:send():179] send: exit
2021-07-14 18:42:51,641 INFO SenderThread:551470 [sender.py:send_exit():287] handling exit code: 1
2021-07-14 18:42:51,641 INFO SenderThread:551470 [sender.py:send_exit():295] send defer
2021-07-14 18:42:51,641 DEBUG HandlerThread:551470 [handler.py:handle_request():124] handle_request: poll_exit
2021-07-14 18:42:51,642 DEBUG HandlerThread:551470 [handler.py:handle_request():124] handle_request: defer
2021-07-14 18:42:51,642 INFO HandlerThread:551470 [handler.py:handle_request_defer():141] handle defer: 0
2021-07-14 18:42:51,642 DEBUG SenderThread:551470 [sender.py:send_request():193] send_request: poll_exit
2021-07-14 18:42:51,642 DEBUG SenderThread:551470 [sender.py:send_request():193] send_request: defer
2021-07-14 18:42:51,642 INFO SenderThread:551470 [sender.py:send_request_defer():304] handle sender defer: 0
2021-07-14 18:42:51,642 INFO SenderThread:551470 [sender.py:transition_state():308] send defer: 1
2021-07-14 18:42:51,643 DEBUG HandlerThread:551470 [handler.py:handle_request():124] handle_request: defer
2021-07-14 18:42:51,643 INFO HandlerThread:551470 [handler.py:handle_request_defer():141] handle defer: 1
2021-07-14 18:42:51,659 DEBUG SenderThread:551470 [sender.py:send_request():193] send_request: defer
2021-07-14 18:42:51,660 INFO SenderThread:551470 [sender.py:send_request_defer():304] handle sender defer: 1
2021-07-14 18:42:51,660 INFO SenderThread:551470 [sender.py:transition_state():308] send defer: 2
2021-07-14 18:42:51,660 DEBUG SenderThread:551470 [sender.py:send():179] send: stats
2021-07-14 18:42:51,660 DEBUG HandlerThread:551470 [handler.py:handle_request():124] handle_request: defer
2021-07-14 18:42:51,660 INFO HandlerThread:551470 [handler.py:handle_request_defer():141] handle defer: 2
2021-07-14 18:42:51,660 DEBUG SenderThread:551470 [sender.py:send_request():193] send_request: defer
2021-07-14 18:42:51,660 INFO SenderThread:551470 [sender.py:send_request_defer():304] handle sender defer: 2
2021-07-14 18:42:51,660 INFO SenderThread:551470 [sender.py:transition_state():308] send defer: 3
2021-07-14 18:42:51,661 DEBUG HandlerThread:551470 [handler.py:handle_request():124] handle_request: defer
2021-07-14 18:42:51,661 INFO HandlerThread:551470 [handler.py:handle_request_defer():141] handle defer: 3
2021-07-14 18:42:51,661 DEBUG SenderThread:551470 [sender.py:send():179] send: summary
2021-07-14 18:42:51,661 INFO SenderThread:551470 [sender.py:_save_file():841] saving file wandb-summary.json with policy end
2021-07-14 18:42:51,661 DEBUG SenderThread:551470 [sender.py:send_request():193] send_request: defer
2021-07-14 18:42:51,661 INFO SenderThread:551470 [sender.py:send_request_defer():304] handle sender defer: 3
2021-07-14 18:42:51,661 INFO SenderThread:551470 [sender.py:transition_state():308] send defer: 4
2021-07-14 18:42:51,662 DEBUG HandlerThread:551470 [handler.py:handle_request():124] handle_request: defer
2021-07-14 18:42:51,662 INFO HandlerThread:551470 [handler.py:handle_request_defer():141] handle defer: 4
2021-07-14 18:42:51,662 DEBUG SenderThread:551470 [sender.py:send_request():193] send_request: defer
2021-07-14 18:42:51,662 INFO SenderThread:551470 [sender.py:send_request_defer():304] handle sender defer: 4
2021-07-14 18:42:51,744 DEBUG HandlerThread:551470 [handler.py:handle_request():124] handle_request: poll_exit
2021-07-14 18:42:51,838 INFO SenderThread:551470 [sender.py:transition_state():308] send defer: 5
2021-07-14 18:42:51,838 DEBUG SenderThread:551470 [sender.py:send_request():193] send_request: poll_exit
2021-07-14 18:42:51,838 DEBUG HandlerThread:551470 [handler.py:handle_request():124] handle_request: defer
2021-07-14 18:42:51,838 INFO HandlerThread:551470 [handler.py:handle_request_defer():141] handle defer: 5
2021-07-14 18:42:51,839 DEBUG SenderThread:551470 [sender.py:send_request():193] send_request: defer
2021-07-14 18:42:51,839 INFO SenderThread:551470 [sender.py:send_request_defer():304] handle sender defer: 5
2021-07-14 18:42:51,839 INFO SenderThread:551470 [dir_watcher.py:finish():282] shutting down directory watcher
2021-07-14 18:42:51,854 INFO Thread-8 :551470 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_183304-3d4rorca/files/config.yaml
2021-07-14 18:42:51,855 INFO SenderThread:551470 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_183304-3d4rorca/files/output.log
2021-07-14 18:42:51,855 INFO SenderThread:551470 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_183304-3d4rorca/files/wandb-summary.json
2021-07-14 18:42:51,855 INFO SenderThread:551470 [dir_watcher.py:finish():312] scan: /home/dat/pino-roberta-base/wandb/run-20210714_183304-3d4rorca/files
2021-07-14 18:42:51,855 INFO SenderThread:551470 [dir_watcher.py:finish():318] scan save: /home/dat/pino-roberta-base/wandb/run-20210714_183304-3d4rorca/files/requirements.txt requirements.txt
2021-07-14 18:42:51,855 INFO SenderThread:551470 [dir_watcher.py:finish():318] scan save: /home/dat/pino-roberta-base/wandb/run-20210714_183304-3d4rorca/files/output.log output.log
2021-07-14 18:42:51,856 INFO SenderThread:551470 [dir_watcher.py:finish():318] scan save: /home/dat/pino-roberta-base/wandb/run-20210714_183304-3d4rorca/files/wandb-metadata.json wandb-metadata.json
2021-07-14 18:42:51,856 INFO SenderThread:551470 [dir_watcher.py:finish():318] scan save: /home/dat/pino-roberta-base/wandb/run-20210714_183304-3d4rorca/files/config.yaml config.yaml
2021-07-14 18:42:51,856 INFO SenderThread:551470 [dir_watcher.py:finish():318] scan save: /home/dat/pino-roberta-base/wandb/run-20210714_183304-3d4rorca/files/wandb-summary.json wandb-summary.json
2021-07-14 18:42:51,856 INFO SenderThread:551470 [sender.py:transition_state():308] send defer: 6
2021-07-14 18:42:51,862 DEBUG HandlerThread:551470 [handler.py:handle_request():124] handle_request: defer
2021-07-14 18:42:51,862 INFO HandlerThread:551470 [handler.py:handle_request_defer():141] handle defer: 6
2021-07-14 18:42:51,865 DEBUG SenderThread:551470 [sender.py:send_request():193] send_request: defer
2021-07-14 18:42:51,865 INFO SenderThread:551470 [sender.py:send_request_defer():304] handle sender defer: 6
2021-07-14 18:42:51,865 INFO SenderThread:551470 [file_pusher.py:finish():177] shutting down file pusher
2021-07-14 18:42:51,939 DEBUG HandlerThread:551470 [handler.py:handle_request():124] handle_request: poll_exit
2021-07-14 18:42:51,940 DEBUG SenderThread:551470 [sender.py:send_request():193] send_request: poll_exit
2021-07-14 18:42:52,041 DEBUG HandlerThread:551470 [handler.py:handle_request():124] handle_request: poll_exit
2021-07-14 18:42:52,041 DEBUG SenderThread:551470 [sender.py:send_request():193] send_request: poll_exit
2021-07-14 18:42:52,143 DEBUG HandlerThread:551470 [handler.py:handle_request():124] handle_request: poll_exit
2021-07-14 18:42:52,143 DEBUG SenderThread:551470 [sender.py:send_request():193] send_request: poll_exit
2021-07-14 18:42:52,245 DEBUG HandlerThread:551470 [handler.py:handle_request():124] handle_request: poll_exit
2021-07-14 18:42:52,245 DEBUG SenderThread:551470 [sender.py:send_request():193] send_request: poll_exit
2021-07-14 18:42:52,308 INFO Thread-12 :551470 [upload_job.py:push():137] Uploaded file /home/dat/pino-roberta-base/wandb/run-20210714_183304-3d4rorca/files/requirements.txt
2021-07-14 18:42:52,325 INFO Thread-13 :551470 [upload_job.py:push():137] Uploaded file /home/dat/pino-roberta-base/wandb/run-20210714_183304-3d4rorca/files/output.log
2021-07-14 18:42:52,341 INFO Thread-15 :551470 [upload_job.py:push():137] Uploaded file /home/dat/pino-roberta-base/wandb/run-20210714_183304-3d4rorca/files/wandb-summary.json
2021-07-14 18:42:52,346 DEBUG HandlerThread:551470 [handler.py:handle_request():124] handle_request: poll_exit
2021-07-14 18:42:52,346 DEBUG SenderThread:551470 [sender.py:send_request():193] send_request: poll_exit
2021-07-14 18:42:52,361 INFO Thread-14 :551470 [upload_job.py:push():137] Uploaded file /home/dat/pino-roberta-base/wandb/run-20210714_183304-3d4rorca/files/config.yaml
2021-07-14 18:42:52,448 DEBUG HandlerThread:551470 [handler.py:handle_request():124] handle_request: poll_exit
2021-07-14 18:42:52,448 DEBUG SenderThread:551470 [sender.py:send_request():193] send_request: poll_exit
2021-07-14 18:42:52,549 DEBUG HandlerThread:551470 [handler.py:handle_request():124] handle_request: poll_exit
2021-07-14 18:42:52,550 DEBUG SenderThread:551470 [sender.py:send_request():193] send_request: poll_exit
2021-07-14 18:42:52,562 INFO Thread-7 :551470 [sender.py:transition_state():308] send defer: 7
2021-07-14 18:42:52,562 DEBUG HandlerThread:551470 [handler.py:handle_request():124] handle_request: defer
2021-07-14 18:42:52,562 INFO HandlerThread:551470 [handler.py:handle_request_defer():141] handle defer: 7
2021-07-14 18:42:52,562 DEBUG SenderThread:551470 [sender.py:send_request():193] send_request: defer
2021-07-14 18:42:52,562 INFO SenderThread:551470 [sender.py:send_request_defer():304] handle sender defer: 7
2021-07-14 18:42:52,651 DEBUG HandlerThread:551470 [handler.py:handle_request():124] handle_request: poll_exit
2021-07-14 18:42:52,837 INFO SenderThread:551470 [sender.py:transition_state():308] send defer: 8
2021-07-14 18:42:52,837 DEBUG SenderThread:551470 [sender.py:send_request():193] send_request: poll_exit
2021-07-14 18:42:52,837 DEBUG HandlerThread:551470 [handler.py:handle_request():124] handle_request: defer
2021-07-14 18:42:52,837 INFO HandlerThread:551470 [handler.py:handle_request_defer():141] handle defer: 8
2021-07-14 18:42:52,838 DEBUG SenderThread:551470 [sender.py:send_request():193] send_request: defer
2021-07-14 18:42:52,838 INFO SenderThread:551470 [sender.py:send_request_defer():304] handle sender defer: 8
2021-07-14 18:42:52,838 INFO SenderThread:551470 [sender.py:transition_state():308] send defer: 9
2021-07-14 18:42:52,838 DEBUG SenderThread:551470 [sender.py:send():179] send: final
2021-07-14 18:42:52,838 DEBUG HandlerThread:551470 [handler.py:handle_request():124] handle_request: defer
2021-07-14 18:42:52,838 INFO HandlerThread:551470 [handler.py:handle_request_defer():141] handle defer: 9
2021-07-14 18:42:52,839 DEBUG SenderThread:551470 [sender.py:send():179] send: footer
2021-07-14 18:42:52,839 DEBUG SenderThread:551470 [sender.py:send_request():193] send_request: defer
2021-07-14 18:42:52,839 INFO SenderThread:551470 [sender.py:send_request_defer():304] handle sender defer: 9
2021-07-14 18:42:52,938 DEBUG HandlerThread:551470 [handler.py:handle_request():124] handle_request: poll_exit
2021-07-14 18:42:52,939 DEBUG SenderThread:551470 [sender.py:send_request():193] send_request: poll_exit
2021-07-14 18:42:52,939 INFO SenderThread:551470 [file_pusher.py:join():182] waiting for file pusher
2021-07-14 18:42:52,940 DEBUG HandlerThread:551470 [handler.py:handle_request():124] handle_request: get_summary
2021-07-14 18:42:52,941 DEBUG HandlerThread:551470 [handler.py:handle_request():124] handle_request: sampled_history
2021-07-14 18:42:52,941 DEBUG HandlerThread:551470 [handler.py:handle_request():124] handle_request: shutdown
2021-07-14 18:42:52,941 INFO HandlerThread:551470 [handler.py:finish():638] shutting down handler
2021-07-14 18:42:53,839 INFO WriterThread:551470 [datastore.py:close():288] close: /home/dat/pino-roberta-base/wandb/run-20210714_183304-3d4rorca/run-3d4rorca.wandb
2021-07-14 18:42:53,939 INFO SenderThread:551470 [sender.py:finish():945] shutting down sender
2021-07-14 18:42:53,939 INFO SenderThread:551470 [file_pusher.py:finish():177] shutting down file pusher
2021-07-14 18:42:53,939 INFO SenderThread:551470 [file_pusher.py:join():182] waiting for file pusher
2021-07-14 18:42:53,941 INFO MainThread:551470 [internal.py:handle_exit():78] Internal process exited