| 2021-07-14 17:55:20,938 INFO MainThread:545287 [internal.py:wandb_internal():88] W&B internal server running at pid: 545287, started at: 2021-07-14 17:55:20.938091 | |
| 2021-07-14 17:55:20,940 DEBUG HandlerThread:545287 [handler.py:handle_request():124] handle_request: check_version | |
| 2021-07-14 17:55:20,941 INFO WriterThread:545287 [datastore.py:open_for_write():80] open: /home/dat/pino-roberta-base/wandb/run-20210714_175518-t843i2it/run-t843i2it.wandb | |
| 2021-07-14 17:55:20,942 DEBUG SenderThread:545287 [sender.py:send():179] send: header | |
| 2021-07-14 17:55:20,942 DEBUG SenderThread:545287 [sender.py:send_request():193] send_request: check_version | |
| 2021-07-14 17:55:20,985 DEBUG SenderThread:545287 [sender.py:send():179] send: run | |
| 2021-07-14 17:55:21,157 INFO SenderThread:545287 [dir_watcher.py:__init__():168] watching files in: /home/dat/pino-roberta-base/wandb/run-20210714_175518-t843i2it/files | |
| 2021-07-14 17:55:21,158 INFO SenderThread:545287 [sender.py:_start_run_threads():716] run started: t843i2it with start time 1626285318 | |
| 2021-07-14 17:55:21,158 DEBUG SenderThread:545287 [sender.py:send():179] send: summary | |
| 2021-07-14 17:55:21,158 DEBUG HandlerThread:545287 [handler.py:handle_request():124] handle_request: run_start | |
| 2021-07-14 17:55:21,159 INFO SenderThread:545287 [sender.py:_save_file():841] saving file wandb-summary.json with policy end | |
| 2021-07-14 17:55:24,792 INFO Thread-8 :545287 [dir_watcher.py:_on_file_created():216] file/dir created: /home/dat/pino-roberta-base/wandb/run-20210714_175518-t843i2it/files/wandb-summary.json | |
| 2021-07-14 17:55:26,883 INFO Thread-8 :545287 [dir_watcher.py:_on_file_created():216] file/dir created: /home/dat/pino-roberta-base/wandb/run-20210714_175518-t843i2it/files/output.log | |
| 2021-07-14 17:55:32,372 DEBUG HandlerThread:545287 [meta.py:__init__():39] meta init | |
| 2021-07-14 17:55:32,372 DEBUG HandlerThread:545287 [meta.py:__init__():53] meta init done | |
| 2021-07-14 17:55:32,372 DEBUG HandlerThread:545287 [meta.py:probe():210] probe | |
| 2021-07-14 17:55:32,373 DEBUG HandlerThread:545287 [meta.py:_setup_git():200] setup git | |
| 2021-07-14 17:55:32,402 DEBUG HandlerThread:545287 [meta.py:_setup_git():207] setup git done | |
| 2021-07-14 17:55:32,402 DEBUG HandlerThread:545287 [meta.py:_save_pip():57] save pip | |
| 2021-07-14 17:55:32,403 DEBUG HandlerThread:545287 [meta.py:_save_pip():71] save pip done | |
| 2021-07-14 17:55:32,403 DEBUG HandlerThread:545287 [meta.py:probe():252] probe done | |
| 2021-07-14 17:55:32,406 DEBUG HandlerThread:545287 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 17:55:32,406 DEBUG SenderThread:545287 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 17:55:32,544 DEBUG SenderThread:545287 [sender.py:send():179] send: config | |
| 2021-07-14 17:55:32,545 DEBUG SenderThread:545287 [sender.py:send():179] send: config | |
| 2021-07-14 17:55:32,545 DEBUG SenderThread:545287 [sender.py:send():179] send: config | |
| 2021-07-14 17:55:32,545 DEBUG SenderThread:545287 [sender.py:send():179] send: files | |
| 2021-07-14 17:55:32,545 INFO SenderThread:545287 [sender.py:_save_file():841] saving file wandb-metadata.json with policy now | |
| 2021-07-14 17:55:32,952 INFO Thread-8 :545287 [dir_watcher.py:_on_file_created():216] file/dir created: /home/dat/pino-roberta-base/wandb/run-20210714_175518-t843i2it/files/wandb-metadata.json | |
| 2021-07-14 17:55:32,953 INFO Thread-8 :545287 [dir_watcher.py:_on_file_created():216] file/dir created: /home/dat/pino-roberta-base/wandb/run-20210714_175518-t843i2it/files/requirements.txt | |
| 2021-07-14 17:55:33,181 INFO Thread-11 :545287 [upload_job.py:push():137] Uploaded file /tmp/tmpldoi9p0fwandb/1bn7qo44-wandb-metadata.json | |
| 2021-07-14 17:55:42,956 INFO Thread-8 :545287 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_175518-t843i2it/files/output.log | |
| 2021-07-14 17:55:44,957 INFO Thread-8 :545287 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_175518-t843i2it/files/output.log | |
| 2021-07-14 17:55:47,548 DEBUG HandlerThread:545287 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 17:55:47,549 DEBUG SenderThread:545287 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 17:55:51,960 INFO Thread-8 :545287 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_175518-t843i2it/files/config.yaml | |
| 2021-07-14 17:56:00,458 DEBUG SenderThread:545287 [sender.py:send():179] send: stats | |
| 2021-07-14 17:56:02,681 DEBUG HandlerThread:545287 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 17:56:02,681 DEBUG SenderThread:545287 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 17:56:17,816 DEBUG HandlerThread:545287 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 17:56:17,816 DEBUG SenderThread:545287 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 17:56:30,528 DEBUG SenderThread:545287 [sender.py:send():179] send: stats | |
| 2021-07-14 17:56:32,958 DEBUG HandlerThread:545287 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 17:56:32,959 DEBUG SenderThread:545287 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 17:56:48,094 DEBUG HandlerThread:545287 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 17:56:48,094 DEBUG SenderThread:545287 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 17:57:00,620 DEBUG SenderThread:545287 [sender.py:send():179] send: stats | |
| 2021-07-14 17:57:03,228 DEBUG HandlerThread:545287 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 17:57:03,229 DEBUG SenderThread:545287 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 17:57:18,364 DEBUG HandlerThread:545287 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 17:57:18,364 DEBUG SenderThread:545287 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 17:57:30,693 DEBUG SenderThread:545287 [sender.py:send():179] send: stats | |
| 2021-07-14 17:57:33,502 DEBUG HandlerThread:545287 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 17:57:33,502 DEBUG SenderThread:545287 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 17:57:48,640 DEBUG HandlerThread:545287 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 17:57:48,640 DEBUG SenderThread:545287 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 17:58:00,768 DEBUG SenderThread:545287 [sender.py:send():179] send: stats | |
| 2021-07-14 17:58:03,776 DEBUG HandlerThread:545287 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 17:58:03,776 DEBUG SenderThread:545287 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 17:58:18,928 DEBUG HandlerThread:545287 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 17:58:18,928 DEBUG SenderThread:545287 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 17:58:30,841 DEBUG SenderThread:545287 [sender.py:send():179] send: stats | |
| 2021-07-14 17:58:34,063 DEBUG HandlerThread:545287 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 17:58:34,063 DEBUG SenderThread:545287 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 17:58:49,196 DEBUG HandlerThread:545287 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 17:58:49,196 DEBUG SenderThread:545287 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 17:59:00,919 DEBUG SenderThread:545287 [sender.py:send():179] send: stats | |
| 2021-07-14 17:59:04,329 DEBUG HandlerThread:545287 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 17:59:04,330 DEBUG SenderThread:545287 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 17:59:19,466 DEBUG HandlerThread:545287 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 17:59:19,467 DEBUG SenderThread:545287 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 17:59:30,993 DEBUG SenderThread:545287 [sender.py:send():179] send: stats | |
| 2021-07-14 17:59:34,601 DEBUG HandlerThread:545287 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 17:59:34,601 DEBUG SenderThread:545287 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 17:59:49,736 DEBUG HandlerThread:545287 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 17:59:49,736 DEBUG SenderThread:545287 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 18:00:01,070 DEBUG SenderThread:545287 [sender.py:send():179] send: stats | |
| 2021-07-14 18:00:04,866 DEBUG HandlerThread:545287 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 18:00:04,867 DEBUG SenderThread:545287 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 18:00:20,000 DEBUG HandlerThread:545287 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 18:00:20,001 DEBUG SenderThread:545287 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 18:00:31,138 DEBUG SenderThread:545287 [sender.py:send():179] send: stats | |
| 2021-07-14 18:00:35,144 DEBUG HandlerThread:545287 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 18:00:35,144 DEBUG SenderThread:545287 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 18:00:50,279 DEBUG HandlerThread:545287 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 18:00:50,280 DEBUG SenderThread:545287 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 18:01:01,215 DEBUG SenderThread:545287 [sender.py:send():179] send: stats | |
| 2021-07-14 18:01:05,414 DEBUG HandlerThread:545287 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 18:01:05,414 DEBUG SenderThread:545287 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 18:01:20,564 DEBUG HandlerThread:545287 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 18:01:20,565 DEBUG SenderThread:545287 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 18:01:31,294 DEBUG SenderThread:545287 [sender.py:send():179] send: stats | |
| 2021-07-14 18:01:35,699 DEBUG HandlerThread:545287 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 18:01:35,699 DEBUG SenderThread:545287 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 18:01:50,829 DEBUG HandlerThread:545287 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 18:01:50,830 DEBUG SenderThread:545287 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 18:02:01,371 DEBUG SenderThread:545287 [sender.py:send():179] send: stats | |
| 2021-07-14 18:02:05,966 DEBUG HandlerThread:545287 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 18:02:05,967 DEBUG SenderThread:545287 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 18:02:21,099 DEBUG HandlerThread:545287 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 18:02:21,099 DEBUG SenderThread:545287 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 18:02:31,448 DEBUG SenderThread:545287 [sender.py:send():179] send: stats | |
| 2021-07-14 18:02:36,234 DEBUG HandlerThread:545287 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 18:02:36,235 DEBUG SenderThread:545287 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 18:02:51,370 DEBUG HandlerThread:545287 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 18:02:51,370 DEBUG SenderThread:545287 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 18:03:01,521 DEBUG SenderThread:545287 [sender.py:send():179] send: stats | |
| 2021-07-14 18:03:06,821 DEBUG HandlerThread:545287 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 18:03:06,821 DEBUG SenderThread:545287 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 18:03:22,036 DEBUG HandlerThread:545287 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 18:03:22,036 DEBUG SenderThread:545287 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 18:03:31,600 DEBUG SenderThread:545287 [sender.py:send():179] send: stats | |
| 2021-07-14 18:03:33,147 INFO Thread-8 :545287 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_175518-t843i2it/files/output.log | |
| 2021-07-14 18:03:37,189 DEBUG HandlerThread:545287 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 18:03:37,190 DEBUG SenderThread:545287 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 18:03:52,336 DEBUG HandlerThread:545287 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 18:03:52,337 DEBUG SenderThread:545287 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 18:04:01,678 DEBUG SenderThread:545287 [sender.py:send():179] send: stats | |
| 2021-07-14 18:04:07,470 DEBUG HandlerThread:545287 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 18:04:07,470 DEBUG SenderThread:545287 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 18:04:22,603 DEBUG HandlerThread:545287 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 18:04:22,604 DEBUG SenderThread:545287 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 18:04:31,760 DEBUG SenderThread:545287 [sender.py:send():179] send: stats | |
| 2021-07-14 18:04:37,735 DEBUG HandlerThread:545287 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 18:04:37,735 DEBUG SenderThread:545287 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 18:04:52,873 DEBUG HandlerThread:545287 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 18:04:52,873 DEBUG SenderThread:545287 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 18:05:01,843 DEBUG SenderThread:545287 [sender.py:send():179] send: stats | |
| 2021-07-14 18:05:08,009 DEBUG HandlerThread:545287 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 18:05:08,010 DEBUG SenderThread:545287 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 18:05:21,189 INFO Thread-8 :545287 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_175518-t843i2it/files/output.log | |
| 2021-07-14 18:05:22,751 DEBUG SenderThread:545287 [sender.py:send():179] send: telemetry | |
| 2021-07-14 18:05:22,752 DEBUG SenderThread:545287 [sender.py:send():179] send: exit | |
| 2021-07-14 18:05:22,752 INFO SenderThread:545287 [sender.py:send_exit():287] handling exit code: 1 | |
| 2021-07-14 18:05:22,753 INFO SenderThread:545287 [sender.py:send_exit():295] send defer | |
| 2021-07-14 18:05:22,753 DEBUG HandlerThread:545287 [handler.py:handle_request():124] handle_request: poll_exit | |
| 2021-07-14 18:05:22,754 DEBUG HandlerThread:545287 [handler.py:handle_request():124] handle_request: defer | |
| 2021-07-14 18:05:22,754 INFO HandlerThread:545287 [handler.py:handle_request_defer():141] handle defer: 0 | |
| 2021-07-14 18:05:22,754 DEBUG SenderThread:545287 [sender.py:send_request():193] send_request: poll_exit | |
| 2021-07-14 18:05:22,754 DEBUG SenderThread:545287 [sender.py:send_request():193] send_request: defer | |
| 2021-07-14 18:05:22,754 INFO SenderThread:545287 [sender.py:send_request_defer():304] handle sender defer: 0 | |
| 2021-07-14 18:05:22,754 INFO SenderThread:545287 [sender.py:transition_state():308] send defer: 1 | |
| 2021-07-14 18:05:22,755 DEBUG HandlerThread:545287 [handler.py:handle_request():124] handle_request: defer | |
| 2021-07-14 18:05:22,755 INFO HandlerThread:545287 [handler.py:handle_request_defer():141] handle defer: 1 | |
| 2021-07-14 18:05:22,795 DEBUG SenderThread:545287 [sender.py:send_request():193] send_request: defer | |
| 2021-07-14 18:05:22,795 INFO SenderThread:545287 [sender.py:send_request_defer():304] handle sender defer: 1 | |
| 2021-07-14 18:05:22,795 INFO SenderThread:545287 [sender.py:transition_state():308] send defer: 2 | |
| 2021-07-14 18:05:22,796 DEBUG SenderThread:545287 [sender.py:send():179] send: stats | |
| 2021-07-14 18:05:22,796 DEBUG HandlerThread:545287 [handler.py:handle_request():124] handle_request: defer | |
| 2021-07-14 18:05:22,796 INFO HandlerThread:545287 [handler.py:handle_request_defer():141] handle defer: 2 | |
| 2021-07-14 18:05:22,796 DEBUG SenderThread:545287 [sender.py:send_request():193] send_request: defer | |
| 2021-07-14 18:05:22,797 INFO SenderThread:545287 [sender.py:send_request_defer():304] handle sender defer: 2 | |
| 2021-07-14 18:05:22,797 INFO SenderThread:545287 [sender.py:transition_state():308] send defer: 3 | |
| 2021-07-14 18:05:22,797 DEBUG HandlerThread:545287 [handler.py:handle_request():124] handle_request: defer | |
| 2021-07-14 18:05:22,797 INFO HandlerThread:545287 [handler.py:handle_request_defer():141] handle defer: 3 | |
| 2021-07-14 18:05:22,797 DEBUG SenderThread:545287 [sender.py:send():179] send: summary | |
| 2021-07-14 18:05:22,797 INFO SenderThread:545287 [sender.py:_save_file():841] saving file wandb-summary.json with policy end | |
| 2021-07-14 18:05:22,798 DEBUG SenderThread:545287 [sender.py:send_request():193] send_request: defer | |
| 2021-07-14 18:05:22,798 INFO SenderThread:545287 [sender.py:send_request_defer():304] handle sender defer: 3 | |
| 2021-07-14 18:05:22,798 INFO SenderThread:545287 [sender.py:transition_state():308] send defer: 4 | |
| 2021-07-14 18:05:22,798 DEBUG HandlerThread:545287 [handler.py:handle_request():124] handle_request: defer | |
| 2021-07-14 18:05:22,798 INFO HandlerThread:545287 [handler.py:handle_request_defer():141] handle defer: 4 | |
| 2021-07-14 18:05:22,798 DEBUG SenderThread:545287 [sender.py:send_request():193] send_request: defer | |
| 2021-07-14 18:05:22,798 INFO SenderThread:545287 [sender.py:send_request_defer():304] handle sender defer: 4 | |
| 2021-07-14 18:05:22,856 DEBUG HandlerThread:545287 [handler.py:handle_request():124] handle_request: poll_exit | |
| 2021-07-14 18:05:22,985 INFO SenderThread:545287 [sender.py:transition_state():308] send defer: 5 | |
| 2021-07-14 18:05:22,985 DEBUG SenderThread:545287 [sender.py:send_request():193] send_request: poll_exit | |
| 2021-07-14 18:05:22,986 DEBUG HandlerThread:545287 [handler.py:handle_request():124] handle_request: defer | |
| 2021-07-14 18:05:22,986 INFO HandlerThread:545287 [handler.py:handle_request_defer():141] handle defer: 5 | |
| 2021-07-14 18:05:22,986 DEBUG SenderThread:545287 [sender.py:send_request():193] send_request: defer | |
| 2021-07-14 18:05:22,986 INFO SenderThread:545287 [sender.py:send_request_defer():304] handle sender defer: 5 | |
| 2021-07-14 18:05:22,986 INFO SenderThread:545287 [dir_watcher.py:finish():282] shutting down directory watcher | |
| 2021-07-14 18:05:23,087 DEBUG HandlerThread:545287 [handler.py:handle_request():124] handle_request: poll_exit | |
| 2021-07-14 18:05:23,190 INFO Thread-8 :545287 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_175518-t843i2it/files/output.log | |
| 2021-07-14 18:05:23,190 INFO SenderThread:545287 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_175518-t843i2it/files/wandb-summary.json | |
| 2021-07-14 18:05:23,190 INFO SenderThread:545287 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_175518-t843i2it/files/config.yaml | |
| 2021-07-14 18:05:23,191 INFO SenderThread:545287 [dir_watcher.py:finish():312] scan: /home/dat/pino-roberta-base/wandb/run-20210714_175518-t843i2it/files | |
| 2021-07-14 18:05:23,191 INFO SenderThread:545287 [dir_watcher.py:finish():318] scan save: /home/dat/pino-roberta-base/wandb/run-20210714_175518-t843i2it/files/requirements.txt requirements.txt | |
| 2021-07-14 18:05:23,191 INFO SenderThread:545287 [dir_watcher.py:finish():318] scan save: /home/dat/pino-roberta-base/wandb/run-20210714_175518-t843i2it/files/output.log output.log | |
| 2021-07-14 18:05:23,195 INFO SenderThread:545287 [dir_watcher.py:finish():318] scan save: /home/dat/pino-roberta-base/wandb/run-20210714_175518-t843i2it/files/wandb-metadata.json wandb-metadata.json | |
| 2021-07-14 18:05:23,195 INFO SenderThread:545287 [dir_watcher.py:finish():318] scan save: /home/dat/pino-roberta-base/wandb/run-20210714_175518-t843i2it/files/config.yaml config.yaml | |
| 2021-07-14 18:05:23,195 INFO SenderThread:545287 [dir_watcher.py:finish():318] scan save: /home/dat/pino-roberta-base/wandb/run-20210714_175518-t843i2it/files/wandb-summary.json wandb-summary.json | |
| 2021-07-14 18:05:23,195 INFO SenderThread:545287 [sender.py:transition_state():308] send defer: 6 | |
| 2021-07-14 18:05:23,195 DEBUG SenderThread:545287 [sender.py:send_request():193] send_request: poll_exit | |
| 2021-07-14 18:05:23,196 DEBUG HandlerThread:545287 [handler.py:handle_request():124] handle_request: defer | |
| 2021-07-14 18:05:23,196 INFO HandlerThread:545287 [handler.py:handle_request_defer():141] handle defer: 6 | |
| 2021-07-14 18:05:23,197 DEBUG SenderThread:545287 [sender.py:send_request():193] send_request: defer | |
| 2021-07-14 18:05:23,197 INFO SenderThread:545287 [sender.py:send_request_defer():304] handle sender defer: 6 | |
| 2021-07-14 18:05:23,197 INFO SenderThread:545287 [file_pusher.py:finish():177] shutting down file pusher | |
| 2021-07-14 18:05:23,299 DEBUG HandlerThread:545287 [handler.py:handle_request():124] handle_request: poll_exit | |
| 2021-07-14 18:05:23,299 DEBUG SenderThread:545287 [sender.py:send_request():193] send_request: poll_exit | |
| 2021-07-14 18:05:23,401 DEBUG HandlerThread:545287 [handler.py:handle_request():124] handle_request: poll_exit | |
| 2021-07-14 18:05:23,401 DEBUG SenderThread:545287 [sender.py:send_request():193] send_request: poll_exit | |
| 2021-07-14 18:05:23,503 DEBUG HandlerThread:545287 [handler.py:handle_request():124] handle_request: poll_exit | |
| 2021-07-14 18:05:23,503 DEBUG SenderThread:545287 [sender.py:send_request():193] send_request: poll_exit | |
| 2021-07-14 18:05:23,605 DEBUG HandlerThread:545287 [handler.py:handle_request():124] handle_request: poll_exit | |
| 2021-07-14 18:05:23,605 DEBUG SenderThread:545287 [sender.py:send_request():193] send_request: poll_exit | |
| 2021-07-14 18:05:23,648 INFO Thread-12 :545287 [upload_job.py:push():137] Uploaded file /home/dat/pino-roberta-base/wandb/run-20210714_175518-t843i2it/files/requirements.txt | |
| 2021-07-14 18:05:23,655 INFO Thread-13 :545287 [upload_job.py:push():137] Uploaded file /home/dat/pino-roberta-base/wandb/run-20210714_175518-t843i2it/files/output.log | |
| 2021-07-14 18:05:23,658 INFO Thread-15 :545287 [upload_job.py:push():137] Uploaded file /home/dat/pino-roberta-base/wandb/run-20210714_175518-t843i2it/files/wandb-summary.json | |
| 2021-07-14 18:05:23,661 INFO Thread-14 :545287 [upload_job.py:push():137] Uploaded file /home/dat/pino-roberta-base/wandb/run-20210714_175518-t843i2it/files/config.yaml | |
| 2021-07-14 18:05:23,708 DEBUG HandlerThread:545287 [handler.py:handle_request():124] handle_request: poll_exit | |
| 2021-07-14 18:05:23,708 DEBUG SenderThread:545287 [sender.py:send_request():193] send_request: poll_exit | |
| 2021-07-14 18:05:23,810 DEBUG HandlerThread:545287 [handler.py:handle_request():124] handle_request: poll_exit | |
| 2021-07-14 18:05:23,810 DEBUG SenderThread:545287 [sender.py:send_request():193] send_request: poll_exit | |
| 2021-07-14 18:05:23,861 INFO Thread-7 :545287 [sender.py:transition_state():308] send defer: 7 | |
| 2021-07-14 18:05:23,862 DEBUG HandlerThread:545287 [handler.py:handle_request():124] handle_request: defer | |
| 2021-07-14 18:05:23,862 INFO HandlerThread:545287 [handler.py:handle_request_defer():141] handle defer: 7 | |
| 2021-07-14 18:05:23,862 DEBUG SenderThread:545287 [sender.py:send_request():193] send_request: defer | |
| 2021-07-14 18:05:23,862 INFO SenderThread:545287 [sender.py:send_request_defer():304] handle sender defer: 7 | |
| 2021-07-14 18:05:23,912 DEBUG HandlerThread:545287 [handler.py:handle_request():124] handle_request: poll_exit | |
| 2021-07-14 18:05:24,152 INFO SenderThread:545287 [sender.py:transition_state():308] send defer: 8 | |
| 2021-07-14 18:05:24,152 DEBUG SenderThread:545287 [sender.py:send_request():193] send_request: poll_exit | |
| 2021-07-14 18:05:24,153 DEBUG HandlerThread:545287 [handler.py:handle_request():124] handle_request: defer | |
| 2021-07-14 18:05:24,153 INFO HandlerThread:545287 [handler.py:handle_request_defer():141] handle defer: 8 | |
| 2021-07-14 18:05:24,154 DEBUG SenderThread:545287 [sender.py:send_request():193] send_request: defer | |
| 2021-07-14 18:05:24,154 INFO SenderThread:545287 [sender.py:send_request_defer():304] handle sender defer: 8 | |
| 2021-07-14 18:05:24,154 INFO SenderThread:545287 [sender.py:transition_state():308] send defer: 9 | |
| 2021-07-14 18:05:24,154 DEBUG HandlerThread:545287 [handler.py:handle_request():124] handle_request: defer | |
| 2021-07-14 18:05:24,155 INFO HandlerThread:545287 [handler.py:handle_request_defer():141] handle defer: 9 | |
| 2021-07-14 18:05:24,155 DEBUG SenderThread:545287 [sender.py:send():179] send: final | |
| 2021-07-14 18:05:24,155 DEBUG SenderThread:545287 [sender.py:send():179] send: footer | |
| 2021-07-14 18:05:24,155 DEBUG SenderThread:545287 [sender.py:send_request():193] send_request: defer | |
| 2021-07-14 18:05:24,155 INFO SenderThread:545287 [sender.py:send_request_defer():304] handle sender defer: 9 | |
| 2021-07-14 18:05:24,255 DEBUG HandlerThread:545287 [handler.py:handle_request():124] handle_request: poll_exit | |
| 2021-07-14 18:05:24,255 DEBUG SenderThread:545287 [sender.py:send_request():193] send_request: poll_exit | |
| 2021-07-14 18:05:24,255 INFO SenderThread:545287 [file_pusher.py:join():182] waiting for file pusher | |
| 2021-07-14 18:05:24,257 DEBUG HandlerThread:545287 [handler.py:handle_request():124] handle_request: get_summary | |
| 2021-07-14 18:05:24,257 DEBUG HandlerThread:545287 [handler.py:handle_request():124] handle_request: sampled_history | |
| 2021-07-14 18:05:24,258 DEBUG HandlerThread:545287 [handler.py:handle_request():124] handle_request: shutdown | |
| 2021-07-14 18:05:24,258 INFO HandlerThread:545287 [handler.py:finish():638] shutting down handler | |
| 2021-07-14 18:05:25,155 INFO WriterThread:545287 [datastore.py:close():288] close: /home/dat/pino-roberta-base/wandb/run-20210714_175518-t843i2it/run-t843i2it.wandb | |
| 2021-07-14 18:05:25,256 INFO SenderThread:545287 [sender.py:finish():945] shutting down sender | |
| 2021-07-14 18:05:25,256 INFO SenderThread:545287 [file_pusher.py:finish():177] shutting down file pusher | |
| 2021-07-14 18:05:25,256 INFO SenderThread:545287 [file_pusher.py:join():182] waiting for file pusher | |
| 2021-07-14 18:05:25,275 INFO MainThread:545287 [internal.py:handle_exit():78] Internal process exited | |