| 2022-05-07 17:41:49,607 INFO MainThread:14620 [internal.py:wandb_internal():92] W&B internal server running at pid: 14620, started at: 2022-05-07 17:41:49.606834 | |
| 2022-05-07 17:41:49,609 INFO WriterThread:14620 [datastore.py:open_for_write():77] open: /home/sanchitgandhi/flax-wav2vec2-2-bart-dummy/wandb/run-20220507_174148-2l3y3icw/run-2l3y3icw.wandb | |
| 2022-05-07 17:41:49,610 DEBUG SenderThread:14620 [sender.py:send():235] send: header | |
| 2022-05-07 17:41:49,610 DEBUG SenderThread:14620 [sender.py:send():235] send: run | |
| 2022-05-07 17:41:49,678 DEBUG HandlerThread:14620 [handler.py:handle_request():141] handle_request: check_version | |
| 2022-05-07 17:41:49,678 INFO SenderThread:14620 [dir_watcher.py:__init__():169] watching files in: /home/sanchitgandhi/flax-wav2vec2-2-bart-dummy/wandb/run-20220507_174148-2l3y3icw/files | |
| 2022-05-07 17:41:49,678 INFO SenderThread:14620 [sender.py:_start_run_threads():812] run started: 2l3y3icw with start time 1651945308 | |
| 2022-05-07 17:41:49,678 DEBUG SenderThread:14620 [sender.py:send():235] send: summary | |
| 2022-05-07 17:41:49,679 INFO SenderThread:14620 [sender.py:_save_file():947] saving file wandb-summary.json with policy end | |
| 2022-05-07 17:41:49,679 DEBUG SenderThread:14620 [sender.py:send_request():249] send_request: check_version | |
| 2022-05-07 17:41:49,739 DEBUG HandlerThread:14620 [handler.py:handle_request():141] handle_request: run_start | |
| 2022-05-07 17:41:50,689 INFO Thread-7 :14620 [dir_watcher.py:_on_file_created():217] file/dir created: /home/sanchitgandhi/flax-wav2vec2-2-bart-dummy/wandb/run-20220507_174148-2l3y3icw/files/wandb-summary.json | |
| 2022-05-07 17:41:51,879 DEBUG HandlerThread:14620 [meta.py:__init__():37] meta init | |
| 2022-05-07 17:41:51,879 DEBUG HandlerThread:14620 [meta.py:__init__():51] meta init done | |
| 2022-05-07 17:41:51,879 DEBUG HandlerThread:14620 [meta.py:probe():211] probe | |
| 2022-05-07 17:41:51,880 DEBUG HandlerThread:14620 [meta.py:_setup_git():201] setup git | |
| 2022-05-07 17:41:51,909 DEBUG HandlerThread:14620 [meta.py:_setup_git():208] setup git done | |
| 2022-05-07 17:41:51,909 DEBUG HandlerThread:14620 [meta.py:_save_pip():55] save pip | |
| 2022-05-07 17:41:51,910 DEBUG HandlerThread:14620 [meta.py:_save_pip():69] save pip done | |
| 2022-05-07 17:41:51,910 DEBUG HandlerThread:14620 [meta.py:probe():249] probe done | |
| 2022-05-07 17:41:51,912 DEBUG SenderThread:14620 [sender.py:send():235] send: files | |
| 2022-05-07 17:41:51,913 INFO SenderThread:14620 [sender.py:_save_file():947] saving file wandb-metadata.json with policy now | |
| 2022-05-07 17:41:51,918 DEBUG HandlerThread:14620 [handler.py:handle_request():141] handle_request: stop_status | |
| 2022-05-07 17:41:51,918 DEBUG SenderThread:14620 [sender.py:send_request():249] send_request: stop_status | |
| 2022-05-07 17:41:51,948 DEBUG SenderThread:14620 [sender.py:send():235] send: telemetry | |
| 2022-05-07 17:41:52,187 INFO Thread-11 :14620 [upload_job.py:push():137] Uploaded file /tmp/tmpandpth3dwandb/1ih5opi9-wandb-metadata.json | |
| 2022-05-07 17:41:52,688 INFO Thread-7 :14620 [dir_watcher.py:_on_file_created():217] file/dir created: /home/sanchitgandhi/flax-wav2vec2-2-bart-dummy/wandb/run-20220507_174148-2l3y3icw/files/requirements.txt | |
| 2022-05-07 17:41:52,689 INFO Thread-7 :14620 [dir_watcher.py:_on_file_created():217] file/dir created: /home/sanchitgandhi/flax-wav2vec2-2-bart-dummy/wandb/run-20220507_174148-2l3y3icw/files/output.log | |
| 2022-05-07 17:41:52,689 INFO Thread-7 :14620 [dir_watcher.py:_on_file_created():217] file/dir created: /home/sanchitgandhi/flax-wav2vec2-2-bart-dummy/wandb/run-20220507_174148-2l3y3icw/files/wandb-metadata.json | |
| 2022-05-07 17:41:53,838 DEBUG SenderThread:14620 [sender.py:send():235] send: telemetry | |
| 2022-05-07 17:41:53,838 DEBUG HandlerThread:14620 [handler.py:handle_request():141] handle_request: partial_history | |
| 2022-05-07 17:41:54,689 INFO Thread-7 :14620 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-dummy/wandb/run-20220507_174148-2l3y3icw/files/output.log | |
| 2022-05-07 17:41:55,690 INFO Thread-7 :14620 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-dummy/wandb/run-20220507_174148-2l3y3icw/files/output.log | |
| 2022-05-07 17:41:56,439 DEBUG HandlerThread:14620 [handler.py:handle_request():141] handle_request: poll_exit | |
| 2022-05-07 17:41:56,439 DEBUG SenderThread:14620 [sender.py:send():235] send: exit | |
| 2022-05-07 17:41:56,439 INFO SenderThread:14620 [sender.py:send_exit():371] handling exit code: 1 | |
| 2022-05-07 17:41:56,439 INFO SenderThread:14620 [sender.py:send_exit():373] handling runtime: 6 | |
| 2022-05-07 17:41:56,440 INFO SenderThread:14620 [sender.py:_save_file():947] saving file wandb-summary.json with policy end | |
| 2022-05-07 17:41:56,440 INFO SenderThread:14620 [sender.py:send_exit():379] send defer | |
| 2022-05-07 17:41:56,440 DEBUG SenderThread:14620 [sender.py:send_request():249] send_request: poll_exit | |
| 2022-05-07 17:41:56,441 DEBUG HandlerThread:14620 [handler.py:handle_request():141] handle_request: defer | |
| 2022-05-07 17:41:56,441 INFO HandlerThread:14620 [handler.py:handle_request_defer():164] handle defer: 0 | |
| 2022-05-07 17:41:56,441 DEBUG SenderThread:14620 [sender.py:send_request():249] send_request: defer | |
| 2022-05-07 17:41:56,441 INFO SenderThread:14620 [sender.py:send_request_defer():388] handle sender defer: 0 | |
| 2022-05-07 17:41:56,441 INFO SenderThread:14620 [sender.py:transition_state():392] send defer: 1 | |
| 2022-05-07 17:41:56,441 DEBUG HandlerThread:14620 [handler.py:handle_request():141] handle_request: defer | |
| 2022-05-07 17:41:56,441 INFO HandlerThread:14620 [handler.py:handle_request_defer():164] handle defer: 1 | |
| 2022-05-07 17:41:56,501 DEBUG SenderThread:14620 [sender.py:send_request():249] send_request: defer | |
| 2022-05-07 17:41:56,501 INFO SenderThread:14620 [sender.py:send_request_defer():388] handle sender defer: 1 | |
| 2022-05-07 17:41:56,501 INFO SenderThread:14620 [sender.py:transition_state():392] send defer: 2 | |
| 2022-05-07 17:41:56,501 DEBUG SenderThread:14620 [sender.py:send():235] send: stats | |
| 2022-05-07 17:41:56,502 DEBUG HandlerThread:14620 [handler.py:handle_request():141] handle_request: defer | |
| 2022-05-07 17:41:56,502 INFO HandlerThread:14620 [handler.py:handle_request_defer():164] handle defer: 2 | |
| 2022-05-07 17:41:56,502 DEBUG SenderThread:14620 [sender.py:send_request():249] send_request: defer | |
| 2022-05-07 17:41:56,502 INFO SenderThread:14620 [sender.py:send_request_defer():388] handle sender defer: 2 | |
| 2022-05-07 17:41:56,502 INFO SenderThread:14620 [sender.py:transition_state():392] send defer: 3 | |
| 2022-05-07 17:41:56,502 DEBUG HandlerThread:14620 [handler.py:handle_request():141] handle_request: defer | |
| 2022-05-07 17:41:56,502 INFO HandlerThread:14620 [handler.py:handle_request_defer():164] handle defer: 3 | |
| 2022-05-07 17:41:56,503 DEBUG SenderThread:14620 [sender.py:send_request():249] send_request: defer | |
| 2022-05-07 17:41:56,503 INFO SenderThread:14620 [sender.py:send_request_defer():388] handle sender defer: 3 | |
| 2022-05-07 17:41:56,503 INFO SenderThread:14620 [sender.py:transition_state():392] send defer: 4 | |
| 2022-05-07 17:41:56,503 DEBUG HandlerThread:14620 [handler.py:handle_request():141] handle_request: defer | |
| 2022-05-07 17:41:56,503 INFO HandlerThread:14620 [handler.py:handle_request_defer():164] handle defer: 4 | |
| 2022-05-07 17:41:56,503 DEBUG SenderThread:14620 [sender.py:send():235] send: summary | |
| 2022-05-07 17:41:56,503 INFO SenderThread:14620 [sender.py:_save_file():947] saving file wandb-summary.json with policy end | |
| 2022-05-07 17:41:56,504 DEBUG SenderThread:14620 [sender.py:send_request():249] send_request: defer | |
| 2022-05-07 17:41:56,504 INFO SenderThread:14620 [sender.py:send_request_defer():388] handle sender defer: 4 | |
| 2022-05-07 17:41:56,504 INFO SenderThread:14620 [sender.py:transition_state():392] send defer: 5 | |
| 2022-05-07 17:41:56,504 DEBUG HandlerThread:14620 [handler.py:handle_request():141] handle_request: defer | |
| 2022-05-07 17:41:56,504 INFO HandlerThread:14620 [handler.py:handle_request_defer():164] handle defer: 5 | |
| 2022-05-07 17:41:56,504 DEBUG SenderThread:14620 [sender.py:send_request():249] send_request: defer | |
| 2022-05-07 17:41:56,504 INFO SenderThread:14620 [sender.py:send_request_defer():388] handle sender defer: 5 | |
| 2022-05-07 17:41:56,542 DEBUG HandlerThread:14620 [handler.py:handle_request():141] handle_request: poll_exit | |
| 2022-05-07 17:41:56,563 INFO SenderThread:14620 [sender.py:transition_state():392] send defer: 6 | |
| 2022-05-07 17:41:56,563 DEBUG SenderThread:14620 [sender.py:send_request():249] send_request: poll_exit | |
| 2022-05-07 17:41:56,564 DEBUG HandlerThread:14620 [handler.py:handle_request():141] handle_request: defer | |
| 2022-05-07 17:41:56,564 INFO HandlerThread:14620 [handler.py:handle_request_defer():164] handle defer: 6 | |
| 2022-05-07 17:41:56,564 DEBUG SenderThread:14620 [sender.py:send_request():249] send_request: defer | |
| 2022-05-07 17:41:56,564 INFO SenderThread:14620 [sender.py:send_request_defer():388] handle sender defer: 6 | |
| 2022-05-07 17:41:56,564 INFO SenderThread:14620 [dir_watcher.py:finish():283] shutting down directory watcher | |
| 2022-05-07 17:41:56,665 DEBUG HandlerThread:14620 [handler.py:handle_request():141] handle_request: poll_exit | |
| 2022-05-07 17:41:56,690 INFO Thread-7 :14620 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-dummy/wandb/run-20220507_174148-2l3y3icw/files/wandb-summary.json | |
| 2022-05-07 17:41:56,691 INFO SenderThread:14620 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-dummy/wandb/run-20220507_174148-2l3y3icw/files/config.yaml | |
| 2022-05-07 17:41:56,691 INFO SenderThread:14620 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-wav2vec2-2-bart-dummy/wandb/run-20220507_174148-2l3y3icw/files/output.log | |
| 2022-05-07 17:41:56,691 INFO SenderThread:14620 [dir_watcher.py:finish():313] scan: /home/sanchitgandhi/flax-wav2vec2-2-bart-dummy/wandb/run-20220507_174148-2l3y3icw/files | |
| 2022-05-07 17:41:56,691 INFO SenderThread:14620 [dir_watcher.py:finish():327] scan save: /home/sanchitgandhi/flax-wav2vec2-2-bart-dummy/wandb/run-20220507_174148-2l3y3icw/files/requirements.txt requirements.txt | |
| 2022-05-07 17:41:56,691 INFO SenderThread:14620 [dir_watcher.py:finish():327] scan save: /home/sanchitgandhi/flax-wav2vec2-2-bart-dummy/wandb/run-20220507_174148-2l3y3icw/files/output.log output.log | |
| 2022-05-07 17:41:56,692 INFO SenderThread:14620 [dir_watcher.py:finish():327] scan save: /home/sanchitgandhi/flax-wav2vec2-2-bart-dummy/wandb/run-20220507_174148-2l3y3icw/files/wandb-metadata.json wandb-metadata.json | |
| 2022-05-07 17:41:56,692 INFO SenderThread:14620 [dir_watcher.py:finish():327] scan save: /home/sanchitgandhi/flax-wav2vec2-2-bart-dummy/wandb/run-20220507_174148-2l3y3icw/files/config.yaml config.yaml | |
| 2022-05-07 17:41:56,692 INFO SenderThread:14620 [dir_watcher.py:finish():327] scan save: /home/sanchitgandhi/flax-wav2vec2-2-bart-dummy/wandb/run-20220507_174148-2l3y3icw/files/wandb-summary.json wandb-summary.json | |
| 2022-05-07 17:41:56,692 INFO SenderThread:14620 [sender.py:transition_state():392] send defer: 7 | |
| 2022-05-07 17:41:56,693 DEBUG SenderThread:14620 [sender.py:send_request():249] send_request: poll_exit | |
| 2022-05-07 17:41:56,693 DEBUG HandlerThread:14620 [handler.py:handle_request():141] handle_request: defer | |
| 2022-05-07 17:41:56,693 INFO HandlerThread:14620 [handler.py:handle_request_defer():164] handle defer: 7 | |
| 2022-05-07 17:41:56,696 DEBUG SenderThread:14620 [sender.py:send_request():249] send_request: defer | |
| 2022-05-07 17:41:56,696 INFO SenderThread:14620 [sender.py:send_request_defer():388] handle sender defer: 7 | |
| 2022-05-07 17:41:56,696 INFO SenderThread:14620 [file_pusher.py:finish():145] shutting down file pusher | |
| 2022-05-07 17:41:56,794 DEBUG HandlerThread:14620 [handler.py:handle_request():141] handle_request: poll_exit | |
| 2022-05-07 17:41:56,794 DEBUG SenderThread:14620 [sender.py:send_request():249] send_request: poll_exit | |
| 2022-05-07 17:41:56,896 DEBUG HandlerThread:14620 [handler.py:handle_request():141] handle_request: poll_exit | |
| 2022-05-07 17:41:56,896 DEBUG SenderThread:14620 [sender.py:send_request():249] send_request: poll_exit | |
| 2022-05-07 17:41:56,940 INFO Thread-13 :14620 [upload_job.py:push():137] Uploaded file /home/sanchitgandhi/flax-wav2vec2-2-bart-dummy/wandb/run-20220507_174148-2l3y3icw/files/output.log | |
| 2022-05-07 17:41:56,947 INFO Thread-14 :14620 [upload_job.py:push():137] Uploaded file /home/sanchitgandhi/flax-wav2vec2-2-bart-dummy/wandb/run-20220507_174148-2l3y3icw/files/config.yaml | |
| 2022-05-07 17:41:56,969 INFO Thread-15 :14620 [upload_job.py:push():137] Uploaded file /home/sanchitgandhi/flax-wav2vec2-2-bart-dummy/wandb/run-20220507_174148-2l3y3icw/files/wandb-summary.json | |
| 2022-05-07 17:41:56,988 INFO Thread-12 :14620 [upload_job.py:push():137] Uploaded file /home/sanchitgandhi/flax-wav2vec2-2-bart-dummy/wandb/run-20220507_174148-2l3y3icw/files/requirements.txt | |
| 2022-05-07 17:41:56,998 DEBUG HandlerThread:14620 [handler.py:handle_request():141] handle_request: poll_exit | |
| 2022-05-07 17:41:56,998 DEBUG SenderThread:14620 [sender.py:send_request():249] send_request: poll_exit | |
| 2022-05-07 17:41:57,100 DEBUG HandlerThread:14620 [handler.py:handle_request():141] handle_request: poll_exit | |
| 2022-05-07 17:41:57,100 DEBUG SenderThread:14620 [sender.py:send_request():249] send_request: poll_exit | |
| 2022-05-07 17:41:57,189 INFO Thread-6 :14620 [sender.py:transition_state():392] send defer: 8 | |
| 2022-05-07 17:41:57,189 DEBUG HandlerThread:14620 [handler.py:handle_request():141] handle_request: defer | |
| 2022-05-07 17:41:57,189 INFO HandlerThread:14620 [handler.py:handle_request_defer():164] handle defer: 8 | |
| 2022-05-07 17:41:57,189 DEBUG SenderThread:14620 [sender.py:send_request():249] send_request: defer | |
| 2022-05-07 17:41:57,189 INFO SenderThread:14620 [sender.py:send_request_defer():388] handle sender defer: 8 | |
| 2022-05-07 17:41:57,202 DEBUG HandlerThread:14620 [handler.py:handle_request():141] handle_request: poll_exit | |
| 2022-05-07 17:41:59,454 INFO SenderThread:14620 [sender.py:transition_state():392] send defer: 9 | |
| 2022-05-07 17:41:59,455 DEBUG SenderThread:14620 [sender.py:send_request():249] send_request: poll_exit | |
| 2022-05-07 17:41:59,455 DEBUG HandlerThread:14620 [handler.py:handle_request():141] handle_request: defer | |
| 2022-05-07 17:41:59,455 INFO HandlerThread:14620 [handler.py:handle_request_defer():164] handle defer: 9 | |
| 2022-05-07 17:41:59,455 DEBUG SenderThread:14620 [sender.py:send_request():249] send_request: defer | |
| 2022-05-07 17:41:59,456 INFO SenderThread:14620 [sender.py:send_request_defer():388] handle sender defer: 9 | |
| 2022-05-07 17:41:59,456 INFO SenderThread:14620 [sender.py:transition_state():392] send defer: 10 | |
| 2022-05-07 17:41:59,456 DEBUG HandlerThread:14620 [handler.py:handle_request():141] handle_request: defer | |
| 2022-05-07 17:41:59,456 INFO HandlerThread:14620 [handler.py:handle_request_defer():164] handle defer: 10 | |
| 2022-05-07 17:41:59,456 DEBUG SenderThread:14620 [sender.py:send():235] send: final | |
| 2022-05-07 17:41:59,456 DEBUG SenderThread:14620 [sender.py:send():235] send: footer | |
| 2022-05-07 17:41:59,456 DEBUG SenderThread:14620 [sender.py:send_request():249] send_request: defer | |
| 2022-05-07 17:41:59,456 INFO SenderThread:14620 [sender.py:send_request_defer():388] handle sender defer: 10 | |
| 2022-05-07 17:41:59,556 DEBUG HandlerThread:14620 [handler.py:handle_request():141] handle_request: poll_exit | |
| 2022-05-07 17:41:59,557 DEBUG SenderThread:14620 [sender.py:send_request():249] send_request: poll_exit | |
| 2022-05-07 17:41:59,557 INFO SenderThread:14620 [file_pusher.py:join():150] waiting for file pusher | |
| 2022-05-07 17:42:00,139 DEBUG HandlerThread:14620 [handler.py:handle_request():141] handle_request: sampled_history | |
| 2022-05-07 17:42:00,140 DEBUG HandlerThread:14620 [handler.py:handle_request():141] handle_request: get_summary | |
| 2022-05-07 17:42:00,141 DEBUG HandlerThread:14620 [handler.py:handle_request():141] handle_request: shutdown | |
| 2022-05-07 17:42:00,141 INFO HandlerThread:14620 [handler.py:finish():778] shutting down handler | |
| 2022-05-07 17:42:00,457 INFO WriterThread:14620 [datastore.py:close():281] close: /home/sanchitgandhi/flax-wav2vec2-2-bart-dummy/wandb/run-20220507_174148-2l3y3icw/run-2l3y3icw.wandb | |
| 2022-05-07 17:42:01,038 INFO SenderThread:14620 [sender.py:finish():1078] shutting down sender | |
| 2022-05-07 17:42:01,038 INFO SenderThread:14620 [file_pusher.py:finish():145] shutting down file pusher | |
| 2022-05-07 17:42:01,038 INFO SenderThread:14620 [file_pusher.py:join():150] waiting for file pusher | |
| 2022-05-07 17:42:01,040 INFO MainThread:14620 [internal.py:handle_exit():82] Internal process exited | |