| 2021-07-14 22:58:21,337 INFO MainThread:601574 [internal.py:wandb_internal():88] W&B internal server running at pid: 601574, started at: 2021-07-14 22:58:21.336704 | |
| 2021-07-14 22:58:21,339 DEBUG HandlerThread:601574 [handler.py:handle_request():124] handle_request: check_version | |
| 2021-07-14 22:58:21,339 INFO WriterThread:601574 [datastore.py:open_for_write():80] open: /home/dat/pino-roberta-base/wandb/run-20210714_225820-1dpoijkp/run-1dpoijkp.wandb | |
| 2021-07-14 22:58:21,340 DEBUG SenderThread:601574 [sender.py:send():179] send: header | |
| 2021-07-14 22:58:21,340 DEBUG SenderThread:601574 [sender.py:send_request():193] send_request: check_version | |
| 2021-07-14 22:58:21,377 DEBUG SenderThread:601574 [sender.py:send():179] send: run | |
| 2021-07-14 22:58:21,602 INFO SenderThread:601574 [dir_watcher.py:__init__():168] watching files in: /home/dat/pino-roberta-base/wandb/run-20210714_225820-1dpoijkp/files | |
| 2021-07-14 22:58:21,602 INFO SenderThread:601574 [sender.py:_start_run_threads():716] run started: 1dpoijkp with start time 1626303500 | |
| 2021-07-14 22:58:21,602 DEBUG SenderThread:601574 [sender.py:send():179] send: summary | |
| 2021-07-14 22:58:21,603 DEBUG HandlerThread:601574 [handler.py:handle_request():124] handle_request: run_start | |
| 2021-07-14 22:58:21,603 INFO SenderThread:601574 [sender.py:_save_file():841] saving file wandb-summary.json with policy end | |
| 2021-07-14 22:58:22,605 INFO Thread-8 :601574 [dir_watcher.py:_on_file_created():216] file/dir created: /home/dat/pino-roberta-base/wandb/run-20210714_225820-1dpoijkp/files/wandb-summary.json | |
| 2021-07-14 22:58:22,701 DEBUG HandlerThread:601574 [meta.py:__init__():39] meta init | |
| 2021-07-14 22:58:22,701 DEBUG HandlerThread:601574 [meta.py:__init__():53] meta init done | |
| 2021-07-14 22:58:22,701 DEBUG HandlerThread:601574 [meta.py:probe():210] probe | |
| 2021-07-14 22:58:22,702 DEBUG HandlerThread:601574 [meta.py:_setup_git():200] setup git | |
| 2021-07-14 22:58:22,732 DEBUG HandlerThread:601574 [meta.py:_setup_git():207] setup git done | |
| 2021-07-14 22:58:22,732 DEBUG HandlerThread:601574 [meta.py:_save_pip():57] save pip | |
| 2021-07-14 22:58:22,732 DEBUG HandlerThread:601574 [meta.py:_save_pip():71] save pip done | |
| 2021-07-14 22:58:22,733 DEBUG HandlerThread:601574 [meta.py:probe():252] probe done | |
| 2021-07-14 22:58:22,736 DEBUG SenderThread:601574 [sender.py:send():179] send: files | |
| 2021-07-14 22:58:22,736 INFO SenderThread:601574 [sender.py:_save_file():841] saving file wandb-metadata.json with policy now | |
| 2021-07-14 22:58:22,742 DEBUG HandlerThread:601574 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 22:58:22,742 DEBUG SenderThread:601574 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 22:58:22,871 DEBUG SenderThread:601574 [sender.py:send():179] send: config | |
| 2021-07-14 22:58:22,872 DEBUG SenderThread:601574 [sender.py:send():179] send: config | |
| 2021-07-14 22:58:22,872 DEBUG SenderThread:601574 [sender.py:send():179] send: config | |
| 2021-07-14 22:58:23,214 INFO Thread-11 :601574 [upload_job.py:push():137] Uploaded file /tmp/tmpg5fs3m8gwandb/35jlequ6-wandb-metadata.json | |
| 2021-07-14 22:58:23,603 INFO Thread-8 :601574 [dir_watcher.py:_on_file_created():216] file/dir created: /home/dat/pino-roberta-base/wandb/run-20210714_225820-1dpoijkp/files/output.log | |
| 2021-07-14 22:58:23,604 INFO Thread-8 :601574 [dir_watcher.py:_on_file_created():216] file/dir created: /home/dat/pino-roberta-base/wandb/run-20210714_225820-1dpoijkp/files/requirements.txt | |
| 2021-07-14 22:58:23,604 INFO Thread-8 :601574 [dir_watcher.py:_on_file_created():216] file/dir created: /home/dat/pino-roberta-base/wandb/run-20210714_225820-1dpoijkp/files/wandb-metadata.json | |
| 2021-07-14 22:58:37,609 INFO Thread-8 :601574 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_225820-1dpoijkp/files/output.log | |
| 2021-07-14 22:58:37,873 DEBUG HandlerThread:601574 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 22:58:37,874 DEBUG SenderThread:601574 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 22:58:50,784 DEBUG SenderThread:601574 [sender.py:send():179] send: stats | |
| 2021-07-14 22:58:52,614 INFO Thread-8 :601574 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_225820-1dpoijkp/files/config.yaml | |
| 2021-07-14 22:58:53,006 DEBUG HandlerThread:601574 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 22:58:53,006 DEBUG SenderThread:601574 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 22:59:08,141 DEBUG HandlerThread:601574 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 22:59:08,141 DEBUG SenderThread:601574 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 22:59:20,866 DEBUG SenderThread:601574 [sender.py:send():179] send: stats | |
| 2021-07-14 22:59:23,276 DEBUG HandlerThread:601574 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 22:59:23,277 DEBUG SenderThread:601574 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 22:59:38,415 DEBUG HandlerThread:601574 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 22:59:38,415 DEBUG SenderThread:601574 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 22:59:50,943 DEBUG SenderThread:601574 [sender.py:send():179] send: stats | |
| 2021-07-14 22:59:53,547 DEBUG HandlerThread:601574 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 22:59:53,547 DEBUG SenderThread:601574 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 23:00:08,680 DEBUG HandlerThread:601574 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 23:00:08,680 DEBUG SenderThread:601574 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 23:00:21,020 DEBUG SenderThread:601574 [sender.py:send():179] send: stats | |
| 2021-07-14 23:00:23,810 DEBUG HandlerThread:601574 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 23:00:23,811 DEBUG SenderThread:601574 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 23:00:38,944 DEBUG HandlerThread:601574 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 23:00:38,945 DEBUG SenderThread:601574 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 23:00:51,098 DEBUG SenderThread:601574 [sender.py:send():179] send: stats | |
| 2021-07-14 23:00:54,080 DEBUG HandlerThread:601574 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 23:00:54,080 DEBUG SenderThread:601574 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 23:01:09,212 DEBUG HandlerThread:601574 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 23:01:09,212 DEBUG SenderThread:601574 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 23:01:21,176 DEBUG SenderThread:601574 [sender.py:send():179] send: stats | |
| 2021-07-14 23:01:24,345 DEBUG HandlerThread:601574 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 23:01:24,346 DEBUG SenderThread:601574 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 23:01:39,477 DEBUG HandlerThread:601574 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 23:01:39,478 DEBUG SenderThread:601574 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 23:01:51,254 DEBUG SenderThread:601574 [sender.py:send():179] send: stats | |
| 2021-07-14 23:01:54,612 DEBUG HandlerThread:601574 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 23:01:54,612 DEBUG SenderThread:601574 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 23:02:09,744 DEBUG HandlerThread:601574 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 23:02:09,744 DEBUG SenderThread:601574 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 23:02:21,332 DEBUG SenderThread:601574 [sender.py:send():179] send: stats | |
| 2021-07-14 23:02:24,932 DEBUG HandlerThread:601574 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 23:02:24,932 DEBUG SenderThread:601574 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 23:02:40,066 DEBUG HandlerThread:601574 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 23:02:40,067 DEBUG SenderThread:601574 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 23:02:51,409 DEBUG SenderThread:601574 [sender.py:send():179] send: stats | |
| 2021-07-14 23:02:55,209 DEBUG HandlerThread:601574 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 23:02:55,209 DEBUG SenderThread:601574 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 23:03:10,341 DEBUG HandlerThread:601574 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 23:03:10,341 DEBUG SenderThread:601574 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 23:03:21,484 DEBUG SenderThread:601574 [sender.py:send():179] send: stats | |
| 2021-07-14 23:03:25,483 DEBUG HandlerThread:601574 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 23:03:25,483 DEBUG SenderThread:601574 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 23:03:40,615 DEBUG HandlerThread:601574 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 23:03:40,615 DEBUG SenderThread:601574 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 23:03:51,547 DEBUG SenderThread:601574 [sender.py:send():179] send: stats | |
| 2021-07-14 23:03:56,377 DEBUG HandlerThread:601574 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 23:03:56,377 DEBUG SenderThread:601574 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 23:04:11,520 DEBUG HandlerThread:601574 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 23:04:11,520 DEBUG SenderThread:601574 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 23:04:21,610 DEBUG SenderThread:601574 [sender.py:send():179] send: stats | |
| 2021-07-14 23:04:26,652 DEBUG HandlerThread:601574 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 23:04:26,652 DEBUG SenderThread:601574 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 23:04:41,781 DEBUG HandlerThread:601574 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 23:04:41,781 DEBUG SenderThread:601574 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 23:04:51,676 DEBUG SenderThread:601574 [sender.py:send():179] send: stats | |
| 2021-07-14 23:04:56,913 DEBUG HandlerThread:601574 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 23:04:56,913 DEBUG SenderThread:601574 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 23:05:13,026 DEBUG HandlerThread:601574 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 23:05:13,027 DEBUG SenderThread:601574 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 23:05:21,744 DEBUG SenderThread:601574 [sender.py:send():179] send: stats | |
| 2021-07-14 23:05:28,237 DEBUG HandlerThread:601574 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 23:05:28,238 DEBUG SenderThread:601574 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 23:05:44,049 DEBUG HandlerThread:601574 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 23:05:44,049 DEBUG SenderThread:601574 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 23:05:51,809 DEBUG SenderThread:601574 [sender.py:send():179] send: stats | |
| 2021-07-14 23:05:59,179 DEBUG HandlerThread:601574 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 23:05:59,180 DEBUG SenderThread:601574 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 23:06:14,311 DEBUG HandlerThread:601574 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 23:06:14,311 DEBUG SenderThread:601574 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 23:06:21,764 INFO Thread-8 :601574 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_225820-1dpoijkp/files/output.log | |
| 2021-07-14 23:06:21,877 DEBUG SenderThread:601574 [sender.py:send():179] send: stats | |
| 2021-07-14 23:06:29,456 DEBUG HandlerThread:601574 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 23:06:29,457 DEBUG SenderThread:601574 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 23:06:44,749 DEBUG HandlerThread:601574 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 23:06:44,749 DEBUG SenderThread:601574 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 23:06:51,950 DEBUG SenderThread:601574 [sender.py:send():179] send: stats | |
| 2021-07-14 23:06:59,941 DEBUG HandlerThread:601574 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 23:06:59,941 DEBUG SenderThread:601574 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 23:07:15,073 DEBUG HandlerThread:601574 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 23:07:15,077 DEBUG SenderThread:601574 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 23:07:22,025 DEBUG SenderThread:601574 [sender.py:send():179] send: stats | |
| 2021-07-14 23:07:30,221 DEBUG HandlerThread:601574 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 23:07:30,221 DEBUG SenderThread:601574 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 23:07:45,363 DEBUG HandlerThread:601574 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 23:07:45,364 DEBUG SenderThread:601574 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 23:07:50,438 DEBUG SenderThread:601574 [sender.py:send():179] send: telemetry | |
| 2021-07-14 23:07:50,439 DEBUG HandlerThread:601574 [handler.py:handle_request():124] handle_request: poll_exit | |
| 2021-07-14 23:07:50,439 DEBUG SenderThread:601574 [sender.py:send():179] send: exit | |
| 2021-07-14 23:07:50,439 INFO SenderThread:601574 [sender.py:send_exit():287] handling exit code: 1 | |
| 2021-07-14 23:07:50,440 INFO SenderThread:601574 [sender.py:send_exit():295] send defer | |
| 2021-07-14 23:07:50,440 DEBUG SenderThread:601574 [sender.py:send_request():193] send_request: poll_exit | |
| 2021-07-14 23:07:50,441 DEBUG HandlerThread:601574 [handler.py:handle_request():124] handle_request: defer | |
| 2021-07-14 23:07:50,441 INFO HandlerThread:601574 [handler.py:handle_request_defer():141] handle defer: 0 | |
| 2021-07-14 23:07:50,441 DEBUG SenderThread:601574 [sender.py:send_request():193] send_request: defer | |
| 2021-07-14 23:07:50,441 INFO SenderThread:601574 [sender.py:send_request_defer():304] handle sender defer: 0 | |
| 2021-07-14 23:07:50,441 INFO SenderThread:601574 [sender.py:transition_state():308] send defer: 1 | |
| 2021-07-14 23:07:50,442 DEBUG HandlerThread:601574 [handler.py:handle_request():124] handle_request: defer | |
| 2021-07-14 23:07:50,442 INFO HandlerThread:601574 [handler.py:handle_request_defer():141] handle defer: 1 | |
| 2021-07-14 23:07:50,497 DEBUG SenderThread:601574 [sender.py:send_request():193] send_request: defer | |
| 2021-07-14 23:07:50,497 INFO SenderThread:601574 [sender.py:send_request_defer():304] handle sender defer: 1 | |
| 2021-07-14 23:07:50,497 INFO SenderThread:601574 [sender.py:transition_state():308] send defer: 2 | |
| 2021-07-14 23:07:50,497 DEBUG SenderThread:601574 [sender.py:send():179] send: stats | |
| 2021-07-14 23:07:50,498 DEBUG HandlerThread:601574 [handler.py:handle_request():124] handle_request: defer | |
| 2021-07-14 23:07:50,498 INFO HandlerThread:601574 [handler.py:handle_request_defer():141] handle defer: 2 | |
| 2021-07-14 23:07:50,498 DEBUG SenderThread:601574 [sender.py:send_request():193] send_request: defer | |
| 2021-07-14 23:07:50,498 INFO SenderThread:601574 [sender.py:send_request_defer():304] handle sender defer: 2 | |
| 2021-07-14 23:07:50,498 INFO SenderThread:601574 [sender.py:transition_state():308] send defer: 3 | |
| 2021-07-14 23:07:50,499 DEBUG HandlerThread:601574 [handler.py:handle_request():124] handle_request: defer | |
| 2021-07-14 23:07:50,499 INFO HandlerThread:601574 [handler.py:handle_request_defer():141] handle defer: 3 | |
| 2021-07-14 23:07:50,499 DEBUG SenderThread:601574 [sender.py:send():179] send: summary | |
| 2021-07-14 23:07:50,499 INFO SenderThread:601574 [sender.py:_save_file():841] saving file wandb-summary.json with policy end | |
| 2021-07-14 23:07:50,499 DEBUG SenderThread:601574 [sender.py:send_request():193] send_request: defer | |
| 2021-07-14 23:07:50,499 INFO SenderThread:601574 [sender.py:send_request_defer():304] handle sender defer: 3 | |
| 2021-07-14 23:07:50,499 INFO SenderThread:601574 [sender.py:transition_state():308] send defer: 4 | |
| 2021-07-14 23:07:50,500 DEBUG HandlerThread:601574 [handler.py:handle_request():124] handle_request: defer | |
| 2021-07-14 23:07:50,500 INFO HandlerThread:601574 [handler.py:handle_request_defer():141] handle defer: 4 | |
| 2021-07-14 23:07:50,500 DEBUG SenderThread:601574 [sender.py:send_request():193] send_request: defer | |
| 2021-07-14 23:07:50,500 INFO SenderThread:601574 [sender.py:send_request_defer():304] handle sender defer: 4 | |
| 2021-07-14 23:07:50,543 DEBUG HandlerThread:601574 [handler.py:handle_request():124] handle_request: poll_exit | |
| 2021-07-14 23:07:50,746 INFO SenderThread:601574 [sender.py:transition_state():308] send defer: 5 | |
| 2021-07-14 23:07:50,746 DEBUG SenderThread:601574 [sender.py:send_request():193] send_request: poll_exit | |
| 2021-07-14 23:07:50,747 DEBUG HandlerThread:601574 [handler.py:handle_request():124] handle_request: defer | |
| 2021-07-14 23:07:50,747 INFO HandlerThread:601574 [handler.py:handle_request_defer():141] handle defer: 5 | |
| 2021-07-14 23:07:50,747 DEBUG SenderThread:601574 [sender.py:send_request():193] send_request: defer | |
| 2021-07-14 23:07:50,747 INFO SenderThread:601574 [sender.py:send_request_defer():304] handle sender defer: 5 | |
| 2021-07-14 23:07:50,747 INFO SenderThread:601574 [dir_watcher.py:finish():282] shutting down directory watcher | |
| 2021-07-14 23:07:50,787 INFO SenderThread:601574 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_225820-1dpoijkp/files/output.log | |
| 2021-07-14 23:07:50,787 INFO SenderThread:601574 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_225820-1dpoijkp/files/config.yaml | |
| 2021-07-14 23:07:50,787 INFO SenderThread:601574 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_225820-1dpoijkp/files/wandb-summary.json | |
| 2021-07-14 23:07:50,788 INFO SenderThread:601574 [dir_watcher.py:finish():312] scan: /home/dat/pino-roberta-base/wandb/run-20210714_225820-1dpoijkp/files | |
| 2021-07-14 23:07:50,788 INFO SenderThread:601574 [dir_watcher.py:finish():318] scan save: /home/dat/pino-roberta-base/wandb/run-20210714_225820-1dpoijkp/files/requirements.txt requirements.txt | |
| 2021-07-14 23:07:50,788 INFO SenderThread:601574 [dir_watcher.py:finish():318] scan save: /home/dat/pino-roberta-base/wandb/run-20210714_225820-1dpoijkp/files/output.log output.log | |
| 2021-07-14 23:07:50,788 INFO SenderThread:601574 [dir_watcher.py:finish():318] scan save: /home/dat/pino-roberta-base/wandb/run-20210714_225820-1dpoijkp/files/wandb-metadata.json wandb-metadata.json | |
| 2021-07-14 23:07:50,788 INFO SenderThread:601574 [dir_watcher.py:finish():318] scan save: /home/dat/pino-roberta-base/wandb/run-20210714_225820-1dpoijkp/files/config.yaml config.yaml | |
| 2021-07-14 23:07:50,789 INFO SenderThread:601574 [dir_watcher.py:finish():318] scan save: /home/dat/pino-roberta-base/wandb/run-20210714_225820-1dpoijkp/files/wandb-summary.json wandb-summary.json | |
| 2021-07-14 23:07:50,792 INFO SenderThread:601574 [sender.py:transition_state():308] send defer: 6 | |
| 2021-07-14 23:07:50,792 DEBUG HandlerThread:601574 [handler.py:handle_request():124] handle_request: defer | |
| 2021-07-14 23:07:50,792 INFO HandlerThread:601574 [handler.py:handle_request_defer():141] handle defer: 6 | |
| 2021-07-14 23:07:50,795 DEBUG SenderThread:601574 [sender.py:send_request():193] send_request: defer | |
| 2021-07-14 23:07:50,795 INFO SenderThread:601574 [sender.py:send_request_defer():304] handle sender defer: 6 | |
| 2021-07-14 23:07:50,795 INFO SenderThread:601574 [file_pusher.py:finish():177] shutting down file pusher | |
| 2021-07-14 23:07:50,848 DEBUG HandlerThread:601574 [handler.py:handle_request():124] handle_request: poll_exit | |
| 2021-07-14 23:07:50,849 DEBUG SenderThread:601574 [sender.py:send_request():193] send_request: poll_exit | |
| 2021-07-14 23:07:50,951 DEBUG HandlerThread:601574 [handler.py:handle_request():124] handle_request: poll_exit | |
| 2021-07-14 23:07:50,951 DEBUG SenderThread:601574 [sender.py:send_request():193] send_request: poll_exit | |
| 2021-07-14 23:07:51,053 DEBUG HandlerThread:601574 [handler.py:handle_request():124] handle_request: poll_exit | |
| 2021-07-14 23:07:51,053 DEBUG SenderThread:601574 [sender.py:send_request():193] send_request: poll_exit | |
| 2021-07-14 23:07:51,155 DEBUG HandlerThread:601574 [handler.py:handle_request():124] handle_request: poll_exit | |
| 2021-07-14 23:07:51,155 DEBUG SenderThread:601574 [sender.py:send_request():193] send_request: poll_exit | |
| 2021-07-14 23:07:51,257 DEBUG HandlerThread:601574 [handler.py:handle_request():124] handle_request: poll_exit | |
| 2021-07-14 23:07:51,257 DEBUG SenderThread:601574 [sender.py:send_request():193] send_request: poll_exit | |
| 2021-07-14 23:07:51,284 INFO Thread-12 :601574 [upload_job.py:push():137] Uploaded file /home/dat/pino-roberta-base/wandb/run-20210714_225820-1dpoijkp/files/requirements.txt | |
| 2021-07-14 23:07:51,311 INFO Thread-14 :601574 [upload_job.py:push():137] Uploaded file /home/dat/pino-roberta-base/wandb/run-20210714_225820-1dpoijkp/files/config.yaml | |
| 2021-07-14 23:07:51,322 INFO Thread-13 :601574 [upload_job.py:push():137] Uploaded file /home/dat/pino-roberta-base/wandb/run-20210714_225820-1dpoijkp/files/output.log | |
| 2021-07-14 23:07:51,359 DEBUG HandlerThread:601574 [handler.py:handle_request():124] handle_request: poll_exit | |
| 2021-07-14 23:07:51,359 DEBUG SenderThread:601574 [sender.py:send_request():193] send_request: poll_exit | |
| 2021-07-14 23:07:51,461 DEBUG HandlerThread:601574 [handler.py:handle_request():124] handle_request: poll_exit | |
| 2021-07-14 23:07:51,461 DEBUG SenderThread:601574 [sender.py:send_request():193] send_request: poll_exit | |
| 2021-07-14 23:07:51,466 INFO Thread-15 :601574 [upload_job.py:push():137] Uploaded file /home/dat/pino-roberta-base/wandb/run-20210714_225820-1dpoijkp/files/wandb-summary.json | |
| 2021-07-14 23:07:51,563 DEBUG HandlerThread:601574 [handler.py:handle_request():124] handle_request: poll_exit | |
| 2021-07-14 23:07:51,563 DEBUG SenderThread:601574 [sender.py:send_request():193] send_request: poll_exit | |
| 2021-07-14 23:07:51,665 DEBUG HandlerThread:601574 [handler.py:handle_request():124] handle_request: poll_exit | |
| 2021-07-14 23:07:51,665 DEBUG SenderThread:601574 [sender.py:send_request():193] send_request: poll_exit | |
| 2021-07-14 23:07:51,666 INFO Thread-7 :601574 [sender.py:transition_state():308] send defer: 7 | |
| 2021-07-14 23:07:51,667 DEBUG HandlerThread:601574 [handler.py:handle_request():124] handle_request: defer | |
| 2021-07-14 23:07:51,667 INFO HandlerThread:601574 [handler.py:handle_request_defer():141] handle defer: 7 | |
| 2021-07-14 23:07:51,667 DEBUG SenderThread:601574 [sender.py:send_request():193] send_request: defer | |
| 2021-07-14 23:07:51,667 INFO SenderThread:601574 [sender.py:send_request_defer():304] handle sender defer: 7 | |
| 2021-07-14 23:07:51,766 DEBUG HandlerThread:601574 [handler.py:handle_request():124] handle_request: poll_exit | |
| 2021-07-14 23:07:51,941 INFO SenderThread:601574 [sender.py:transition_state():308] send defer: 8 | |
| 2021-07-14 23:07:51,941 DEBUG SenderThread:601574 [sender.py:send_request():193] send_request: poll_exit | |
| 2021-07-14 23:07:51,942 DEBUG HandlerThread:601574 [handler.py:handle_request():124] handle_request: defer | |
| 2021-07-14 23:07:51,942 INFO HandlerThread:601574 [handler.py:handle_request_defer():141] handle defer: 8 | |
| 2021-07-14 23:07:51,942 DEBUG SenderThread:601574 [sender.py:send_request():193] send_request: defer | |
| 2021-07-14 23:07:51,942 INFO SenderThread:601574 [sender.py:send_request_defer():304] handle sender defer: 8 | |
| 2021-07-14 23:07:51,942 INFO SenderThread:601574 [sender.py:transition_state():308] send defer: 9 | |
| 2021-07-14 23:07:51,942 DEBUG SenderThread:601574 [sender.py:send():179] send: final | |
| 2021-07-14 23:07:51,942 DEBUG SenderThread:601574 [sender.py:send():179] send: footer | |
| 2021-07-14 23:07:51,943 DEBUG HandlerThread:601574 [handler.py:handle_request():124] handle_request: defer | |
| 2021-07-14 23:07:51,943 INFO HandlerThread:601574 [handler.py:handle_request_defer():141] handle defer: 9 | |
| 2021-07-14 23:07:51,943 DEBUG SenderThread:601574 [sender.py:send_request():193] send_request: defer | |
| 2021-07-14 23:07:51,943 INFO SenderThread:601574 [sender.py:send_request_defer():304] handle sender defer: 9 | |
| 2021-07-14 23:07:52,043 DEBUG HandlerThread:601574 [handler.py:handle_request():124] handle_request: poll_exit | |
| 2021-07-14 23:07:52,043 DEBUG SenderThread:601574 [sender.py:send_request():193] send_request: poll_exit | |
| 2021-07-14 23:07:52,044 INFO SenderThread:601574 [file_pusher.py:join():182] waiting for file pusher | |
| 2021-07-14 23:07:52,045 DEBUG HandlerThread:601574 [handler.py:handle_request():124] handle_request: get_summary | |
| 2021-07-14 23:07:52,045 DEBUG HandlerThread:601574 [handler.py:handle_request():124] handle_request: sampled_history | |
| 2021-07-14 23:07:52,046 DEBUG HandlerThread:601574 [handler.py:handle_request():124] handle_request: shutdown | |
| 2021-07-14 23:07:52,046 INFO HandlerThread:601574 [handler.py:finish():638] shutting down handler | |
| 2021-07-14 23:07:52,943 INFO WriterThread:601574 [datastore.py:close():288] close: /home/dat/pino-roberta-base/wandb/run-20210714_225820-1dpoijkp/run-1dpoijkp.wandb | |
| 2021-07-14 23:07:53,044 INFO SenderThread:601574 [sender.py:finish():945] shutting down sender | |
| 2021-07-14 23:07:53,044 INFO SenderThread:601574 [file_pusher.py:finish():177] shutting down file pusher | |
| 2021-07-14 23:07:53,044 INFO SenderThread:601574 [file_pusher.py:join():182] waiting for file pusher | |
| 2021-07-14 23:07:53,047 INFO MainThread:601574 [internal.py:handle_exit():78] Internal process exited | |