| 2021-07-14 20:31:18,076 INFO MainThread:569706 [internal.py:wandb_internal():88] W&B internal server running at pid: 569706, started at: 2021-07-14 20:31:18.075968 | |
| 2021-07-14 20:31:18,078 INFO WriterThread:569706 [datastore.py:open_for_write():80] open: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/run-3emij6kq.wandb | |
| 2021-07-14 20:31:18,078 DEBUG HandlerThread:569706 [handler.py:handle_request():124] handle_request: check_version | |
| 2021-07-14 20:31:18,079 DEBUG SenderThread:569706 [sender.py:send():179] send: header | |
| 2021-07-14 20:31:18,079 DEBUG SenderThread:569706 [sender.py:send_request():193] send_request: check_version | |
| 2021-07-14 20:31:18,118 DEBUG SenderThread:569706 [sender.py:send():179] send: run | |
| 2021-07-14 20:31:18,293 INFO SenderThread:569706 [dir_watcher.py:__init__():168] watching files in: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files | |
| 2021-07-14 20:31:18,293 INFO SenderThread:569706 [sender.py:_start_run_threads():716] run started: 3emij6kq with start time 1626294677 | |
| 2021-07-14 20:31:18,293 DEBUG SenderThread:569706 [sender.py:send():179] send: summary | |
| 2021-07-14 20:31:18,293 INFO SenderThread:569706 [sender.py:_save_file():841] saving file wandb-summary.json with policy end | |
| 2021-07-14 20:31:18,293 DEBUG HandlerThread:569706 [handler.py:handle_request():124] handle_request: run_start | |
| 2021-07-14 20:31:19,296 INFO Thread-8 :569706 [dir_watcher.py:_on_file_created():216] file/dir created: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/wandb-summary.json | |
| 2021-07-14 20:31:19,373 DEBUG HandlerThread:569706 [meta.py:__init__():39] meta init | |
| 2021-07-14 20:31:19,373 DEBUG HandlerThread:569706 [meta.py:__init__():53] meta init done | |
| 2021-07-14 20:31:19,373 DEBUG HandlerThread:569706 [meta.py:probe():210] probe | |
| 2021-07-14 20:31:19,374 DEBUG HandlerThread:569706 [meta.py:_setup_git():200] setup git | |
| 2021-07-14 20:31:19,403 DEBUG HandlerThread:569706 [meta.py:_setup_git():207] setup git done | |
| 2021-07-14 20:31:19,403 DEBUG HandlerThread:569706 [meta.py:_save_pip():57] save pip | |
| 2021-07-14 20:31:19,404 DEBUG HandlerThread:569706 [meta.py:_save_pip():71] save pip done | |
| 2021-07-14 20:31:19,404 DEBUG HandlerThread:569706 [meta.py:probe():252] probe done | |
| 2021-07-14 20:31:19,407 DEBUG SenderThread:569706 [sender.py:send():179] send: files | |
| 2021-07-14 20:31:19,407 INFO SenderThread:569706 [sender.py:_save_file():841] saving file wandb-metadata.json with policy now | |
| 2021-07-14 20:31:19,413 DEBUG HandlerThread:569706 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:31:19,413 DEBUG SenderThread:569706 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:31:19,541 DEBUG SenderThread:569706 [sender.py:send():179] send: config | |
| 2021-07-14 20:31:19,541 DEBUG SenderThread:569706 [sender.py:send():179] send: config | |
| 2021-07-14 20:31:19,542 DEBUG SenderThread:569706 [sender.py:send():179] send: config | |
| 2021-07-14 20:31:19,853 INFO Thread-11 :569706 [upload_job.py:push():137] Uploaded file /tmp/tmprctrqfj8wandb/151zr105-wandb-metadata.json | |
| 2021-07-14 20:31:20,294 INFO Thread-8 :569706 [dir_watcher.py:_on_file_created():216] file/dir created: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/requirements.txt | |
| 2021-07-14 20:31:20,295 INFO Thread-8 :569706 [dir_watcher.py:_on_file_created():216] file/dir created: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:31:20,295 INFO Thread-8 :569706 [dir_watcher.py:_on_file_created():216] file/dir created: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/wandb-metadata.json | |
| 2021-07-14 20:31:34,300 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:31:34,543 DEBUG HandlerThread:569706 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:31:34,543 DEBUG SenderThread:569706 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:31:36,300 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:31:40,302 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:31:47,455 DEBUG SenderThread:569706 [sender.py:send():179] send: stats | |
| 2021-07-14 20:31:49,306 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/config.yaml | |
| 2021-07-14 20:31:49,691 DEBUG HandlerThread:569706 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:31:49,692 DEBUG SenderThread:569706 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:32:04,839 DEBUG HandlerThread:569706 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:32:04,839 DEBUG SenderThread:569706 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:32:17,538 DEBUG SenderThread:569706 [sender.py:send():179] send: stats | |
| 2021-07-14 20:32:19,973 DEBUG HandlerThread:569706 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:32:19,973 DEBUG SenderThread:569706 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:32:35,105 DEBUG HandlerThread:569706 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:32:35,105 DEBUG SenderThread:569706 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:32:47,616 DEBUG SenderThread:569706 [sender.py:send():179] send: stats | |
| 2021-07-14 20:32:50,237 DEBUG HandlerThread:569706 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:32:50,237 DEBUG SenderThread:569706 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:33:05,367 DEBUG HandlerThread:569706 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:33:05,367 DEBUG SenderThread:569706 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:33:17,693 DEBUG SenderThread:569706 [sender.py:send():179] send: stats | |
| 2021-07-14 20:33:18,340 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:33:20,341 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:33:20,549 DEBUG HandlerThread:569706 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:33:20,549 DEBUG SenderThread:569706 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:33:22,342 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:33:24,343 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:33:26,344 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:33:28,345 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:33:30,346 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:33:32,346 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:33:34,347 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:33:35,690 DEBUG HandlerThread:569706 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:33:35,691 DEBUG SenderThread:569706 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:33:36,348 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:33:38,349 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:33:40,350 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:33:42,351 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:33:44,352 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:33:46,353 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:33:47,771 DEBUG SenderThread:569706 [sender.py:send():179] send: stats | |
| 2021-07-14 20:33:48,354 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:33:50,355 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:33:50,833 DEBUG HandlerThread:569706 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:33:50,833 DEBUG SenderThread:569706 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:33:52,356 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:33:54,356 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:33:56,357 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:33:58,358 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:34:00,359 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:34:02,360 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:34:04,361 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:34:05,963 DEBUG HandlerThread:569706 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:34:05,963 DEBUG SenderThread:569706 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:34:06,362 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:34:08,363 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:34:10,364 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:34:12,365 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:34:14,366 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:34:16,367 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:34:17,848 DEBUG SenderThread:569706 [sender.py:send():179] send: stats | |
| 2021-07-14 20:34:18,368 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:34:20,369 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:34:21,092 DEBUG HandlerThread:569706 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:34:21,093 DEBUG SenderThread:569706 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:34:22,369 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:34:24,370 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:34:26,371 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:34:28,372 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:34:30,373 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:34:32,374 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:34:34,375 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:34:36,254 DEBUG HandlerThread:569706 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:34:36,254 DEBUG SenderThread:569706 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:34:36,376 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:34:38,377 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:34:40,377 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:34:42,379 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:34:44,380 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:34:46,381 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:34:47,925 DEBUG SenderThread:569706 [sender.py:send():179] send: stats | |
| 2021-07-14 20:34:48,381 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:34:50,382 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:34:51,390 DEBUG HandlerThread:569706 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:34:51,390 DEBUG SenderThread:569706 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:34:52,383 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:34:54,384 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:34:56,385 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:34:58,386 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:35:00,386 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:35:02,387 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:35:04,388 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:35:06,389 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:35:06,526 DEBUG HandlerThread:569706 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:35:06,527 DEBUG SenderThread:569706 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:35:08,390 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:35:10,391 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:35:12,391 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:35:14,392 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:35:16,393 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:35:18,004 DEBUG SenderThread:569706 [sender.py:send():179] send: stats | |
| 2021-07-14 20:35:18,394 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:35:20,395 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:35:21,675 DEBUG HandlerThread:569706 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:35:21,675 DEBUG SenderThread:569706 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:35:22,396 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:35:24,397 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:35:26,398 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:35:28,398 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:35:30,399 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:35:32,400 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:35:34,401 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:35:36,402 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:35:36,809 DEBUG HandlerThread:569706 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:35:36,810 DEBUG SenderThread:569706 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:35:39,403 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:35:41,404 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:35:43,405 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:35:45,406 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:35:47,407 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:35:48,079 DEBUG SenderThread:569706 [sender.py:send():179] send: stats | |
| 2021-07-14 20:35:49,408 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:35:51,409 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:35:51,945 DEBUG HandlerThread:569706 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:35:51,945 DEBUG SenderThread:569706 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:35:53,410 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:36:07,092 DEBUG HandlerThread:569706 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:36:07,093 DEBUG SenderThread:569706 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:36:13,798 DEBUG SenderThread:569706 [sender.py:send():179] send: history | |
| 2021-07-14 20:36:13,798 DEBUG SenderThread:569706 [sender.py:send():179] send: summary | |
| 2021-07-14 20:36:13,800 INFO SenderThread:569706 [sender.py:_save_file():841] saving file wandb-summary.json with policy end | |
| 2021-07-14 20:36:14,418 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:36:14,418 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/wandb-summary.json | |
| 2021-07-14 20:36:15,419 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:36:17,419 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:36:18,159 DEBUG SenderThread:569706 [sender.py:send():179] send: stats | |
| 2021-07-14 20:36:19,420 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:36:21,421 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:36:22,223 DEBUG HandlerThread:569706 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:36:22,223 DEBUG SenderThread:569706 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:36:23,422 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:36:25,423 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:36:27,424 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:36:29,425 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:36:31,426 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:36:33,427 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:36:35,428 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:36:37,354 DEBUG HandlerThread:569706 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:36:37,354 DEBUG SenderThread:569706 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:36:37,429 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:36:39,430 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:36:41,431 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:36:43,431 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:36:45,432 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:36:47,433 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:36:48,232 DEBUG SenderThread:569706 [sender.py:send():179] send: stats | |
| 2021-07-14 20:36:49,434 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:36:51,435 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:36:52,488 DEBUG HandlerThread:569706 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:36:52,488 DEBUG SenderThread:569706 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:36:53,436 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:36:55,437 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:36:57,438 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:36:59,439 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:37:01,440 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:37:03,441 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:37:05,442 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:37:07,443 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:37:07,644 DEBUG HandlerThread:569706 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:37:07,644 DEBUG SenderThread:569706 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:37:09,444 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:37:11,445 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:37:13,446 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:37:15,447 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:37:17,447 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:37:18,309 DEBUG SenderThread:569706 [sender.py:send():179] send: stats | |
| 2021-07-14 20:37:19,448 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:37:21,449 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:37:22,825 DEBUG HandlerThread:569706 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:37:22,825 DEBUG SenderThread:569706 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:37:23,450 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:37:25,451 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:37:27,452 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:37:29,453 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:37:31,454 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:37:33,455 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:37:35,456 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:37:37,457 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:37:37,956 DEBUG HandlerThread:569706 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:37:37,956 DEBUG SenderThread:569706 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:37:39,458 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:37:41,459 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:37:43,460 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:37:45,461 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:37:47,462 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:37:48,387 DEBUG SenderThread:569706 [sender.py:send():179] send: stats | |
| 2021-07-14 20:37:49,463 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:37:51,463 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:37:53,090 DEBUG HandlerThread:569706 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:37:53,091 DEBUG SenderThread:569706 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:37:53,464 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:37:55,465 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:37:57,466 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:37:59,467 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:38:01,468 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:38:03,469 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:38:05,470 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:38:07,470 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:38:08,227 DEBUG HandlerThread:569706 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:38:08,228 DEBUG SenderThread:569706 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:38:09,471 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:38:11,472 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:38:13,473 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:38:15,474 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:38:17,474 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:38:18,464 DEBUG SenderThread:569706 [sender.py:send():179] send: stats | |
| 2021-07-14 20:38:19,475 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:38:21,476 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:38:23,361 DEBUG HandlerThread:569706 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:38:23,362 DEBUG SenderThread:569706 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:38:23,477 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:38:25,478 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:38:27,479 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:38:29,480 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:38:31,481 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:38:33,482 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:38:35,483 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:38:37,484 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:38:38,492 DEBUG HandlerThread:569706 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:38:38,492 DEBUG SenderThread:569706 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:38:39,485 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:38:41,486 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:38:43,487 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:38:45,488 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:38:47,488 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:38:48,541 DEBUG SenderThread:569706 [sender.py:send():179] send: stats | |
| 2021-07-14 20:38:53,658 DEBUG HandlerThread:569706 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:38:53,658 DEBUG SenderThread:569706 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:39:08,787 DEBUG HandlerThread:569706 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:39:08,788 DEBUG SenderThread:569706 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:39:09,733 DEBUG SenderThread:569706 [sender.py:send():179] send: history | |
| 2021-07-14 20:39:09,733 DEBUG SenderThread:569706 [sender.py:send():179] send: summary | |
| 2021-07-14 20:39:09,734 INFO SenderThread:569706 [sender.py:_save_file():841] saving file wandb-summary.json with policy end | |
| 2021-07-14 20:39:10,498 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/wandb-summary.json | |
| 2021-07-14 20:39:11,498 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:39:13,499 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:39:15,500 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:39:17,501 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:39:18,620 DEBUG SenderThread:569706 [sender.py:send():179] send: stats | |
| 2021-07-14 20:39:19,502 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:39:21,503 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:39:23,504 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:39:23,928 DEBUG HandlerThread:569706 [handler.py:handle_request():124] handle_request: stop_status | |
| 2021-07-14 20:39:23,928 DEBUG SenderThread:569706 [sender.py:send_request():193] send_request: stop_status | |
| 2021-07-14 20:39:25,505 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:39:27,119 WARNING MainThread:569706 [internal.py:wandb_internal():147] Internal process interrupt: 1 | |
| 2021-07-14 20:39:27,506 INFO Thread-8 :569706 [dir_watcher.py:_on_file_modified():229] file/dir modified: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log | |
| 2021-07-14 20:39:27,538 WARNING MainThread:569706 [internal.py:wandb_internal():147] Internal process interrupt: 2 | |
| 2021-07-14 20:39:27,539 ERROR MainThread:569706 [internal.py:wandb_internal():150] Internal process interrupted. | |
| 2021-07-14 20:39:28,203 INFO SenderThread:569706 [sender.py:finish():945] shutting down sender | |
| 2021-07-14 20:39:28,203 INFO WriterThread:569706 [datastore.py:close():288] close: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/run-3emij6kq.wandb | |
| 2021-07-14 20:39:28,204 INFO SenderThread:569706 [dir_watcher.py:finish():282] shutting down directory watcher | |
| 2021-07-14 20:39:28,204 INFO HandlerThread:569706 [handler.py:finish():638] shutting down handler | |
| 2021-07-14 20:39:28,507 INFO SenderThread:569706 [dir_watcher.py:finish():312] scan: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files | |
| 2021-07-14 20:39:28,507 INFO SenderThread:569706 [dir_watcher.py:finish():318] scan save: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/requirements.txt requirements.txt | |
| 2021-07-14 20:39:28,507 INFO SenderThread:569706 [dir_watcher.py:finish():318] scan save: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/output.log output.log | |
| 2021-07-14 20:39:28,507 INFO SenderThread:569706 [dir_watcher.py:finish():318] scan save: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/wandb-metadata.json wandb-metadata.json | |
| 2021-07-14 20:39:28,507 INFO SenderThread:569706 [dir_watcher.py:finish():318] scan save: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/config.yaml config.yaml | |
| 2021-07-14 20:39:28,508 INFO SenderThread:569706 [dir_watcher.py:finish():318] scan save: /home/dat/pino-roberta-base/wandb/run-20210714_203117-3emij6kq/files/wandb-summary.json wandb-summary.json | |
| 2021-07-14 20:39:28,508 INFO SenderThread:569706 [file_pusher.py:finish():177] shutting down file pusher | |
| 2021-07-14 20:39:28,508 INFO SenderThread:569706 [file_pusher.py:join():182] waiting for file pusher | |
| 2021-07-14 20:39:28,823 INFO MainThread:569706 [internal.py:handle_exit():78] Internal process exited | |