| 2022-02-01 10:30:05,624 INFO MainThread:61045 [internal.py:wandb_internal():87] W&B internal server running at pid: 61045, started at: 2022-02-01 10:30:05.624191 | |
| 2022-02-01 10:30:05,628 DEBUG HandlerThread:61045 [handler.py:handle_request():130] handle_request: check_version | |
| 2022-02-01 10:30:05,628 INFO WriterThread:61045 [datastore.py:open_for_write():77] open: /workspace/xls-r-300m-fr/wandb/run-20220201_103004-1yfj7vwy/run-1yfj7vwy.wandb | |
| 2022-02-01 10:30:05,632 DEBUG SenderThread:61045 [sender.py:send():234] send: header | |
| 2022-02-01 10:30:05,632 DEBUG SenderThread:61045 [sender.py:send_request():248] send_request: check_version | |
| 2022-02-01 10:30:05,706 DEBUG SenderThread:61045 [sender.py:send():234] send: run | |
| 2022-02-01 10:30:05,896 INFO SenderThread:61045 [dir_watcher.py:__init__():169] watching files in: /workspace/xls-r-300m-fr/wandb/run-20220201_103004-1yfj7vwy/files | |
| 2022-02-01 10:30:05,896 INFO SenderThread:61045 [sender.py:_start_run_threads():804] run started: 1yfj7vwy with start time 1643711404 | |
| 2022-02-01 10:30:05,896 DEBUG SenderThread:61045 [sender.py:send():234] send: summary | |
| 2022-02-01 10:30:05,897 INFO SenderThread:61045 [sender.py:_save_file():939] saving file wandb-summary.json with policy end | |
| 2022-02-01 10:30:05,898 DEBUG HandlerThread:61045 [handler.py:handle_request():130] handle_request: run_start | |
| 2022-02-01 10:30:05,908 DEBUG HandlerThread:61045 [meta.py:__init__():40] meta init | |
| 2022-02-01 10:30:05,909 DEBUG HandlerThread:61045 [meta.py:__init__():54] meta init done | |
| 2022-02-01 10:30:05,909 DEBUG HandlerThread:61045 [meta.py:probe():214] probe | |
| 2022-02-01 10:30:05,917 DEBUG HandlerThread:61045 [meta.py:_setup_git():204] setup git | |
| 2022-02-01 10:30:05,952 DEBUG HandlerThread:61045 [meta.py:_setup_git():211] setup git done | |
| 2022-02-01 10:30:05,953 DEBUG HandlerThread:61045 [meta.py:_save_pip():58] save pip | |
| 2022-02-01 10:30:05,954 DEBUG HandlerThread:61045 [meta.py:_save_pip():72] save pip done | |
| 2022-02-01 10:30:05,954 DEBUG HandlerThread:61045 [meta.py:_save_conda():79] save conda | |
| 2022-02-01 10:30:06,494 DEBUG HandlerThread:61045 [meta.py:_save_conda():89] save conda done | |
| 2022-02-01 10:30:06,495 DEBUG HandlerThread:61045 [meta.py:probe():252] probe done | |
| 2022-02-01 10:30:06,500 DEBUG SenderThread:61045 [sender.py:send():234] send: files | |
| 2022-02-01 10:30:06,501 INFO SenderThread:61045 [sender.py:_save_file():939] saving file wandb-metadata.json with policy now | |
| 2022-02-01 10:30:06,511 DEBUG HandlerThread:61045 [handler.py:handle_request():130] handle_request: stop_status | |
| 2022-02-01 10:30:06,512 DEBUG SenderThread:61045 [sender.py:send_request():248] send_request: stop_status | |
| 2022-02-01 10:30:06,673 DEBUG SenderThread:61045 [sender.py:send():234] send: config | |
| 2022-02-01 10:30:06,675 DEBUG SenderThread:61045 [sender.py:send():234] send: metric | |
| 2022-02-01 10:30:06,676 DEBUG SenderThread:61045 [sender.py:send():234] send: metric | |
| 2022-02-01 10:30:06,676 WARNING SenderThread:61045 [sender.py:send_metric():897] Seen metric with glob (shouldnt happen) | |
| 2022-02-01 10:30:06,676 DEBUG SenderThread:61045 [sender.py:send():234] send: metric | |
| 2022-02-01 10:30:06,676 DEBUG SenderThread:61045 [sender.py:send():234] send: metric | |
| 2022-02-01 10:30:06,676 DEBUG SenderThread:61045 [sender.py:send():234] send: metric | |
| 2022-02-01 10:30:06,677 DEBUG SenderThread:61045 [sender.py:send():234] send: metric | |
| 2022-02-01 10:30:06,677 DEBUG SenderThread:61045 [sender.py:send():234] send: metric | |
| 2022-02-01 10:30:06,677 DEBUG SenderThread:61045 [sender.py:send():234] send: history | |
| 2022-02-01 10:30:06,677 DEBUG SenderThread:61045 [sender.py:send():234] send: summary | |
| 2022-02-01 10:30:06,678 INFO SenderThread:61045 [sender.py:_save_file():939] saving file wandb-summary.json with policy end | |
| 2022-02-01 10:30:06,898 INFO Thread-8 :61045 [dir_watcher.py:_on_file_created():217] file/dir created: /workspace/xls-r-300m-fr/wandb/run-20220201_103004-1yfj7vwy/files/conda-environment.yaml | |
| 2022-02-01 10:30:06,898 INFO Thread-8 :61045 [dir_watcher.py:_on_file_created():217] file/dir created: /workspace/xls-r-300m-fr/wandb/run-20220201_103004-1yfj7vwy/files/output.log | |
| 2022-02-01 10:30:06,898 INFO Thread-8 :61045 [dir_watcher.py:_on_file_created():217] file/dir created: /workspace/xls-r-300m-fr/wandb/run-20220201_103004-1yfj7vwy/files/wandb-summary.json | |
| 2022-02-01 10:30:06,898 INFO Thread-8 :61045 [dir_watcher.py:_on_file_created():217] file/dir created: /workspace/xls-r-300m-fr/wandb/run-20220201_103004-1yfj7vwy/files/wandb-metadata.json | |
| 2022-02-01 10:30:06,899 INFO Thread-8 :61045 [dir_watcher.py:_on_file_created():217] file/dir created: /workspace/xls-r-300m-fr/wandb/run-20220201_103004-1yfj7vwy/files/requirements.txt | |
| 2022-02-01 10:30:07,139 INFO Thread-11 :61045 [upload_job.py:push():137] Uploaded file /tmp/tmpzeoerhazwandb/25qe2qjl-wandb-metadata.json | |
| 2022-02-01 10:30:08,898 INFO Thread-8 :61045 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220201_103004-1yfj7vwy/files/output.log | |
| 2022-02-01 10:30:10,899 INFO Thread-8 :61045 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220201_103004-1yfj7vwy/files/output.log | |
| 2022-02-01 10:30:21,675 DEBUG HandlerThread:61045 [handler.py:handle_request():130] handle_request: stop_status | |
| 2022-02-01 10:30:21,676 DEBUG SenderThread:61045 [sender.py:send_request():248] send_request: stop_status | |
| 2022-02-01 10:30:34,180 DEBUG SenderThread:61045 [sender.py:send():234] send: stats | |
| 2022-02-01 10:30:36,839 DEBUG HandlerThread:61045 [handler.py:handle_request():130] handle_request: stop_status | |
| 2022-02-01 10:30:36,839 DEBUG SenderThread:61045 [sender.py:send_request():248] send_request: stop_status | |
| 2022-02-01 10:30:36,921 INFO Thread-8 :61045 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220201_103004-1yfj7vwy/files/config.yaml | |
| 2022-02-01 10:30:52,000 DEBUG HandlerThread:61045 [handler.py:handle_request():130] handle_request: stop_status | |
| 2022-02-01 10:30:52,001 DEBUG SenderThread:61045 [sender.py:send_request():248] send_request: stop_status | |
| 2022-02-01 10:31:04,514 DEBUG SenderThread:61045 [sender.py:send():234] send: stats | |
| 2022-02-01 10:31:07,162 DEBUG HandlerThread:61045 [handler.py:handle_request():130] handle_request: stop_status | |
| 2022-02-01 10:31:07,163 DEBUG SenderThread:61045 [sender.py:send_request():248] send_request: stop_status | |
| 2022-02-01 10:31:22,319 DEBUG HandlerThread:61045 [handler.py:handle_request():130] handle_request: stop_status | |
| 2022-02-01 10:31:22,320 DEBUG SenderThread:61045 [sender.py:send_request():248] send_request: stop_status | |
| 2022-02-01 10:31:24,949 INFO Thread-8 :61045 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220201_103004-1yfj7vwy/files/output.log | |
| 2022-02-01 10:31:30,952 INFO Thread-8 :61045 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220201_103004-1yfj7vwy/files/output.log | |
| 2022-02-01 10:31:32,953 INFO Thread-8 :61045 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220201_103004-1yfj7vwy/files/output.log | |
| 2022-02-01 10:31:34,785 DEBUG SenderThread:61045 [sender.py:send():234] send: stats | |
| 2022-02-01 10:31:34,954 INFO Thread-8 :61045 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220201_103004-1yfj7vwy/files/output.log | |
| 2022-02-01 10:31:36,955 INFO Thread-8 :61045 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220201_103004-1yfj7vwy/files/output.log | |
| 2022-02-01 10:31:37,488 DEBUG HandlerThread:61045 [handler.py:handle_request():130] handle_request: stop_status | |
| 2022-02-01 10:31:37,489 DEBUG SenderThread:61045 [sender.py:send_request():248] send_request: stop_status | |
| 2022-02-01 10:31:38,957 INFO Thread-8 :61045 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220201_103004-1yfj7vwy/files/output.log | |
| 2022-02-01 10:31:52,642 DEBUG HandlerThread:61045 [handler.py:handle_request():130] handle_request: stop_status | |
| 2022-02-01 10:31:52,643 DEBUG SenderThread:61045 [sender.py:send_request():248] send_request: stop_status | |
| 2022-02-01 10:31:52,963 INFO Thread-8 :61045 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220201_103004-1yfj7vwy/files/output.log | |
| 2022-02-01 10:32:05,018 DEBUG SenderThread:61045 [sender.py:send():234] send: stats | |
| 2022-02-01 10:32:07,832 DEBUG HandlerThread:61045 [handler.py:handle_request():130] handle_request: stop_status | |
| 2022-02-01 10:32:07,833 DEBUG SenderThread:61045 [sender.py:send_request():248] send_request: stop_status | |
| 2022-02-01 10:32:23,003 DEBUG HandlerThread:61045 [handler.py:handle_request():130] handle_request: stop_status | |
| 2022-02-01 10:32:23,003 DEBUG SenderThread:61045 [sender.py:send_request():248] send_request: stop_status | |
| 2022-02-01 10:32:35,259 DEBUG SenderThread:61045 [sender.py:send():234] send: stats | |
| 2022-02-01 10:32:38,164 DEBUG HandlerThread:61045 [handler.py:handle_request():130] handle_request: stop_status | |
| 2022-02-01 10:32:38,165 DEBUG SenderThread:61045 [sender.py:send_request():248] send_request: stop_status | |
| 2022-02-01 10:32:42,989 INFO Thread-8 :61045 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220201_103004-1yfj7vwy/files/output.log | |
| 2022-02-01 10:32:44,990 INFO Thread-8 :61045 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220201_103004-1yfj7vwy/files/output.log | |