flax-wav2vec2-2-bart-large-cnn-gradient-accumulation
/
wandb
/run-20220323_184406-3j6a5wn7
/logs
/debug-internal.log
| 2022-03-23 18:44:07,501 INFO MainThread:32213 [internal.py:wandb_internal():92] W&B internal server running at pid: 32213, started at: 2022-03-23 18:44:07.500863 | |
| 2022-03-23 18:44:07,528 INFO WriterThread:32213 [datastore.py:open_for_write():77] open: /home/sanchit_huggingface_co/flax-wav2vec2-2-bart-large-cnn-gradient-accumulation/wandb/run-20220323_184406-3j6a5wn7/run-3j6a5wn7.wandb | |
| 2022-03-23 18:44:07,529 DEBUG SenderThread:32213 [sender.py:send():235] send: header | |
| 2022-03-23 18:44:07,529 DEBUG SenderThread:32213 [sender.py:send():235] send: run | |
| 2022-03-23 18:44:07,629 DEBUG HandlerThread:32213 [handler.py:handle_request():141] handle_request: check_version | |
| 2022-03-23 18:44:07,630 INFO SenderThread:32213 [dir_watcher.py:__init__():169] watching files in: /home/sanchit_huggingface_co/flax-wav2vec2-2-bart-large-cnn-gradient-accumulation/wandb/run-20220323_184406-3j6a5wn7/files | |
| 2022-03-23 18:44:07,630 INFO SenderThread:32213 [sender.py:_start_run_threads():812] run started: 3j6a5wn7 with start time 1648061046 | |
| 2022-03-23 18:44:07,630 DEBUG SenderThread:32213 [sender.py:send():235] send: summary | |
| 2022-03-23 18:44:07,630 INFO SenderThread:32213 [sender.py:_save_file():947] saving file wandb-summary.json with policy end | |
| 2022-03-23 18:44:07,630 DEBUG SenderThread:32213 [sender.py:send_request():249] send_request: check_version | |
| 2022-03-23 18:44:07,689 DEBUG HandlerThread:32213 [handler.py:handle_request():141] handle_request: run_start | |
| 2022-03-23 18:44:08,680 INFO Thread-7 :32213 [dir_watcher.py:_on_file_created():217] file/dir created: /home/sanchit_huggingface_co/flax-wav2vec2-2-bart-large-cnn-gradient-accumulation/wandb/run-20220323_184406-3j6a5wn7/files/wandb-summary.json | |
| 2022-03-23 18:44:09,993 DEBUG HandlerThread:32213 [meta.py:__init__():37] meta init | |
| 2022-03-23 18:44:09,994 DEBUG HandlerThread:32213 [meta.py:__init__():51] meta init done | |
| 2022-03-23 18:44:09,994 DEBUG HandlerThread:32213 [meta.py:probe():211] probe | |
| 2022-03-23 18:44:09,995 DEBUG HandlerThread:32213 [meta.py:_setup_git():201] setup git | |
| 2022-03-23 18:44:10,033 DEBUG HandlerThread:32213 [meta.py:_setup_git():208] setup git done | |
| 2022-03-23 18:44:10,033 DEBUG HandlerThread:32213 [meta.py:_save_pip():55] save pip | |
| 2022-03-23 18:44:10,033 DEBUG HandlerThread:32213 [meta.py:_save_pip():69] save pip done | |
| 2022-03-23 18:44:10,033 DEBUG HandlerThread:32213 [meta.py:probe():249] probe done | |
| 2022-03-23 18:44:10,037 DEBUG SenderThread:32213 [sender.py:send():235] send: files | |
| 2022-03-23 18:44:10,037 INFO SenderThread:32213 [sender.py:_save_file():947] saving file wandb-metadata.json with policy now | |
| 2022-03-23 18:44:10,043 DEBUG HandlerThread:32213 [handler.py:handle_request():141] handle_request: stop_status | |
| 2022-03-23 18:44:10,043 DEBUG SenderThread:32213 [sender.py:send_request():249] send_request: stop_status | |
| 2022-03-23 18:44:10,073 DEBUG SenderThread:32213 [sender.py:send():235] send: telemetry | |
| 2022-03-23 18:44:10,309 INFO Thread-11 :32213 [upload_job.py:push():137] Uploaded file /tmp/tmp70kpwpj6wandb/bh9ty10j-wandb-metadata.json | |
| 2022-03-23 18:44:10,684 INFO Thread-7 :32213 [dir_watcher.py:_on_file_created():217] file/dir created: /home/sanchit_huggingface_co/flax-wav2vec2-2-bart-large-cnn-gradient-accumulation/wandb/run-20220323_184406-3j6a5wn7/files/output.log | |
| 2022-03-23 18:44:10,684 INFO Thread-7 :32213 [dir_watcher.py:_on_file_created():217] file/dir created: /home/sanchit_huggingface_co/flax-wav2vec2-2-bart-large-cnn-gradient-accumulation/wandb/run-20220323_184406-3j6a5wn7/files/requirements.txt | |
| 2022-03-23 18:44:10,684 INFO Thread-7 :32213 [dir_watcher.py:_on_file_created():217] file/dir created: /home/sanchit_huggingface_co/flax-wav2vec2-2-bart-large-cnn-gradient-accumulation/wandb/run-20220323_184406-3j6a5wn7/files/wandb-metadata.json | |
| 2022-03-23 18:44:12,684 INFO Thread-7 :32213 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchit_huggingface_co/flax-wav2vec2-2-bart-large-cnn-gradient-accumulation/wandb/run-20220323_184406-3j6a5wn7/files/output.log | |
| 2022-03-23 18:44:25,076 DEBUG HandlerThread:32213 [handler.py:handle_request():141] handle_request: stop_status | |
| 2022-03-23 18:44:25,076 DEBUG SenderThread:32213 [sender.py:send_request():249] send_request: stop_status | |
| 2022-03-23 18:44:28,690 INFO Thread-7 :32213 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchit_huggingface_co/flax-wav2vec2-2-bart-large-cnn-gradient-accumulation/wandb/run-20220323_184406-3j6a5wn7/files/output.log | |
| 2022-03-23 18:44:32,692 INFO Thread-7 :32213 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchit_huggingface_co/flax-wav2vec2-2-bart-large-cnn-gradient-accumulation/wandb/run-20220323_184406-3j6a5wn7/files/output.log | |
| 2022-03-23 18:44:34,693 INFO Thread-7 :32213 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchit_huggingface_co/flax-wav2vec2-2-bart-large-cnn-gradient-accumulation/wandb/run-20220323_184406-3j6a5wn7/files/output.log | |
| 2022-03-23 18:44:36,694 INFO Thread-7 :32213 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchit_huggingface_co/flax-wav2vec2-2-bart-large-cnn-gradient-accumulation/wandb/run-20220323_184406-3j6a5wn7/files/output.log | |
| 2022-03-23 18:44:38,079 DEBUG SenderThread:32213 [sender.py:send():235] send: stats | |
| 2022-03-23 18:44:38,695 INFO Thread-7 :32213 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchit_huggingface_co/flax-wav2vec2-2-bart-large-cnn-gradient-accumulation/wandb/run-20220323_184406-3j6a5wn7/files/output.log | |
| 2022-03-23 18:44:38,695 INFO Thread-7 :32213 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchit_huggingface_co/flax-wav2vec2-2-bart-large-cnn-gradient-accumulation/wandb/run-20220323_184406-3j6a5wn7/files/config.yaml | |
| 2022-03-23 18:44:40,111 DEBUG HandlerThread:32213 [handler.py:handle_request():141] handle_request: stop_status | |
| 2022-03-23 18:44:40,111 DEBUG SenderThread:32213 [sender.py:send_request():249] send_request: stop_status | |
| 2022-03-23 18:44:44,697 INFO Thread-7 :32213 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchit_huggingface_co/flax-wav2vec2-2-bart-large-cnn-gradient-accumulation/wandb/run-20220323_184406-3j6a5wn7/files/output.log | |
| 2022-03-23 18:44:55,224 DEBUG HandlerThread:32213 [handler.py:handle_request():141] handle_request: stop_status | |
| 2022-03-23 18:44:55,225 DEBUG SenderThread:32213 [sender.py:send_request():249] send_request: stop_status | |
| 2022-03-23 18:45:08,157 DEBUG SenderThread:32213 [sender.py:send():235] send: stats | |
| 2022-03-23 18:45:10,347 DEBUG HandlerThread:32213 [handler.py:handle_request():141] handle_request: stop_status | |
| 2022-03-23 18:45:10,347 DEBUG SenderThread:32213 [sender.py:send_request():249] send_request: stop_status | |
| 2022-03-23 18:45:25,849 DEBUG HandlerThread:32213 [handler.py:handle_request():141] handle_request: stop_status | |
| 2022-03-23 18:45:25,850 DEBUG SenderThread:32213 [sender.py:send_request():249] send_request: stop_status | |
| 2022-03-23 18:45:38,235 DEBUG SenderThread:32213 [sender.py:send():235] send: stats | |
| 2022-03-23 18:45:41,934 DEBUG HandlerThread:32213 [handler.py:handle_request():141] handle_request: stop_status | |
| 2022-03-23 18:45:41,935 DEBUG SenderThread:32213 [sender.py:send_request():249] send_request: stop_status | |
| 2022-03-23 18:45:56,976 DEBUG HandlerThread:32213 [handler.py:handle_request():141] handle_request: stop_status | |
| 2022-03-23 18:45:56,976 DEBUG SenderThread:32213 [sender.py:send_request():249] send_request: stop_status | |
| 2022-03-23 18:46:08,312 DEBUG SenderThread:32213 [sender.py:send():235] send: stats | |
| 2022-03-23 18:46:12,020 DEBUG HandlerThread:32213 [handler.py:handle_request():141] handle_request: stop_status | |
| 2022-03-23 18:46:12,020 DEBUG SenderThread:32213 [sender.py:send_request():249] send_request: stop_status | |
| 2022-03-23 18:46:27,068 DEBUG HandlerThread:32213 [handler.py:handle_request():141] handle_request: stop_status | |
| 2022-03-23 18:46:27,068 DEBUG SenderThread:32213 [sender.py:send_request():249] send_request: stop_status | |
| 2022-03-23 18:46:38,389 DEBUG SenderThread:32213 [sender.py:send():235] send: stats | |
| 2022-03-23 18:46:42,109 DEBUG HandlerThread:32213 [handler.py:handle_request():141] handle_request: stop_status | |
| 2022-03-23 18:46:42,109 DEBUG SenderThread:32213 [sender.py:send_request():249] send_request: stop_status | |
| 2022-03-23 18:46:57,154 DEBUG HandlerThread:32213 [handler.py:handle_request():141] handle_request: stop_status | |
| 2022-03-23 18:46:57,154 DEBUG SenderThread:32213 [sender.py:send_request():249] send_request: stop_status | |
| 2022-03-23 18:47:08,468 DEBUG SenderThread:32213 [sender.py:send():235] send: stats | |
| 2022-03-23 18:47:12,194 DEBUG HandlerThread:32213 [handler.py:handle_request():141] handle_request: stop_status | |
| 2022-03-23 18:47:12,194 DEBUG SenderThread:32213 [sender.py:send_request():249] send_request: stop_status | |
| 2022-03-23 18:47:27,263 DEBUG HandlerThread:32213 [handler.py:handle_request():141] handle_request: stop_status | |
| 2022-03-23 18:47:27,264 DEBUG SenderThread:32213 [sender.py:send_request():249] send_request: stop_status | |
| 2022-03-23 18:47:38,545 DEBUG SenderThread:32213 [sender.py:send():235] send: stats | |
| 2022-03-23 18:47:42,314 DEBUG HandlerThread:32213 [handler.py:handle_request():141] handle_request: stop_status | |
| 2022-03-23 18:47:42,314 DEBUG SenderThread:32213 [sender.py:send_request():249] send_request: stop_status | |
| 2022-03-23 18:47:57,356 DEBUG HandlerThread:32213 [handler.py:handle_request():141] handle_request: stop_status | |
| 2022-03-23 18:47:57,357 DEBUG SenderThread:32213 [sender.py:send_request():249] send_request: stop_status | |
| 2022-03-23 18:48:08,624 DEBUG SenderThread:32213 [sender.py:send():235] send: stats | |
| 2022-03-23 18:48:12,408 DEBUG HandlerThread:32213 [handler.py:handle_request():141] handle_request: stop_status | |
| 2022-03-23 18:48:12,409 DEBUG SenderThread:32213 [sender.py:send_request():249] send_request: stop_status | |
| 2022-03-23 18:48:19,558 DEBUG SenderThread:32213 [sender.py:send():235] send: telemetry | |
| 2022-03-23 18:48:19,559 DEBUG HandlerThread:32213 [handler.py:handle_request():141] handle_request: partial_history | |
| 2022-03-23 18:48:21,586 DEBUG HandlerThread:32213 [handler.py:handle_request():141] handle_request: poll_exit | |
| 2022-03-23 18:48:21,586 DEBUG SenderThread:32213 [sender.py:send():235] send: exit | |
| 2022-03-23 18:48:21,587 INFO SenderThread:32213 [sender.py:send_exit():371] handling exit code: 1 | |
| 2022-03-23 18:48:21,587 INFO SenderThread:32213 [sender.py:send_exit():373] handling runtime: 253 | |
| 2022-03-23 18:48:21,587 INFO SenderThread:32213 [sender.py:_save_file():947] saving file wandb-summary.json with policy end | |
| 2022-03-23 18:48:21,587 INFO SenderThread:32213 [sender.py:send_exit():379] send defer | |
| 2022-03-23 18:48:21,588 DEBUG SenderThread:32213 [sender.py:send_request():249] send_request: poll_exit | |
| 2022-03-23 18:48:21,588 DEBUG HandlerThread:32213 [handler.py:handle_request():141] handle_request: defer | |
| 2022-03-23 18:48:21,588 INFO HandlerThread:32213 [handler.py:handle_request_defer():164] handle defer: 0 | |
| 2022-03-23 18:48:21,589 DEBUG SenderThread:32213 [sender.py:send_request():249] send_request: defer | |
| 2022-03-23 18:48:21,589 INFO SenderThread:32213 [sender.py:send_request_defer():388] handle sender defer: 0 | |
| 2022-03-23 18:48:21,589 INFO SenderThread:32213 [sender.py:transition_state():392] send defer: 1 | |
| 2022-03-23 18:48:21,589 DEBUG HandlerThread:32213 [handler.py:handle_request():141] handle_request: defer | |
| 2022-03-23 18:48:21,589 INFO HandlerThread:32213 [handler.py:handle_request_defer():164] handle defer: 1 | |
| 2022-03-23 18:48:21,659 DEBUG SenderThread:32213 [sender.py:send_request():249] send_request: defer | |
| 2022-03-23 18:48:21,659 INFO SenderThread:32213 [sender.py:send_request_defer():388] handle sender defer: 1 | |
| 2022-03-23 18:48:21,659 INFO SenderThread:32213 [sender.py:transition_state():392] send defer: 2 | |
| 2022-03-23 18:48:21,659 DEBUG SenderThread:32213 [sender.py:send():235] send: stats | |
| 2022-03-23 18:48:21,659 DEBUG HandlerThread:32213 [handler.py:handle_request():141] handle_request: defer | |
| 2022-03-23 18:48:21,659 INFO HandlerThread:32213 [handler.py:handle_request_defer():164] handle defer: 2 | |
| 2022-03-23 18:48:21,660 DEBUG SenderThread:32213 [sender.py:send_request():249] send_request: defer | |
| 2022-03-23 18:48:21,660 INFO SenderThread:32213 [sender.py:send_request_defer():388] handle sender defer: 2 | |
| 2022-03-23 18:48:21,660 INFO SenderThread:32213 [sender.py:transition_state():392] send defer: 3 | |
| 2022-03-23 18:48:21,660 DEBUG HandlerThread:32213 [handler.py:handle_request():141] handle_request: defer | |
| 2022-03-23 18:48:21,660 INFO HandlerThread:32213 [handler.py:handle_request_defer():164] handle defer: 3 | |
| 2022-03-23 18:48:21,660 DEBUG SenderThread:32213 [sender.py:send_request():249] send_request: defer | |
| 2022-03-23 18:48:21,660 INFO SenderThread:32213 [sender.py:send_request_defer():388] handle sender defer: 3 | |
| 2022-03-23 18:48:21,660 INFO SenderThread:32213 [sender.py:transition_state():392] send defer: 4 | |
| 2022-03-23 18:48:21,660 DEBUG HandlerThread:32213 [handler.py:handle_request():141] handle_request: defer | |
| 2022-03-23 18:48:21,660 INFO HandlerThread:32213 [handler.py:handle_request_defer():164] handle defer: 4 | |
| 2022-03-23 18:48:21,661 DEBUG SenderThread:32213 [sender.py:send():235] send: summary | |
| 2022-03-23 18:48:21,661 INFO SenderThread:32213 [sender.py:_save_file():947] saving file wandb-summary.json with policy end | |
| 2022-03-23 18:48:21,661 DEBUG SenderThread:32213 [sender.py:send_request():249] send_request: defer | |
| 2022-03-23 18:48:21,661 INFO SenderThread:32213 [sender.py:send_request_defer():388] handle sender defer: 4 | |
| 2022-03-23 18:48:21,661 INFO SenderThread:32213 [sender.py:transition_state():392] send defer: 5 | |
| 2022-03-23 18:48:21,662 DEBUG HandlerThread:32213 [handler.py:handle_request():141] handle_request: defer | |
| 2022-03-23 18:48:21,662 INFO HandlerThread:32213 [handler.py:handle_request_defer():164] handle defer: 5 | |
| 2022-03-23 18:48:21,662 DEBUG SenderThread:32213 [sender.py:send_request():249] send_request: defer | |
| 2022-03-23 18:48:21,662 INFO SenderThread:32213 [sender.py:send_request_defer():388] handle sender defer: 5 | |
| 2022-03-23 18:48:21,689 DEBUG HandlerThread:32213 [handler.py:handle_request():141] handle_request: poll_exit | |
| 2022-03-23 18:48:21,733 INFO SenderThread:32213 [sender.py:transition_state():392] send defer: 6 | |
| 2022-03-23 18:48:21,733 DEBUG SenderThread:32213 [sender.py:send_request():249] send_request: poll_exit | |
| 2022-03-23 18:48:21,733 DEBUG HandlerThread:32213 [handler.py:handle_request():141] handle_request: defer | |
| 2022-03-23 18:48:21,733 INFO HandlerThread:32213 [handler.py:handle_request_defer():164] handle defer: 6 | |
| 2022-03-23 18:48:21,734 DEBUG SenderThread:32213 [sender.py:send_request():249] send_request: defer | |
| 2022-03-23 18:48:21,734 INFO SenderThread:32213 [sender.py:send_request_defer():388] handle sender defer: 6 | |
| 2022-03-23 18:48:21,734 INFO SenderThread:32213 [dir_watcher.py:finish():283] shutting down directory watcher | |
| 2022-03-23 18:48:21,743 INFO SenderThread:32213 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchit_huggingface_co/flax-wav2vec2-2-bart-large-cnn-gradient-accumulation/wandb/run-20220323_184406-3j6a5wn7/files/output.log | |
| 2022-03-23 18:48:21,743 INFO SenderThread:32213 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchit_huggingface_co/flax-wav2vec2-2-bart-large-cnn-gradient-accumulation/wandb/run-20220323_184406-3j6a5wn7/files/wandb-summary.json | |
| 2022-03-23 18:48:21,743 INFO SenderThread:32213 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchit_huggingface_co/flax-wav2vec2-2-bart-large-cnn-gradient-accumulation/wandb/run-20220323_184406-3j6a5wn7/files/config.yaml | |
| 2022-03-23 18:48:21,744 INFO SenderThread:32213 [dir_watcher.py:finish():313] scan: /home/sanchit_huggingface_co/flax-wav2vec2-2-bart-large-cnn-gradient-accumulation/wandb/run-20220323_184406-3j6a5wn7/files | |
| 2022-03-23 18:48:21,744 INFO SenderThread:32213 [dir_watcher.py:finish():327] scan save: /home/sanchit_huggingface_co/flax-wav2vec2-2-bart-large-cnn-gradient-accumulation/wandb/run-20220323_184406-3j6a5wn7/files/requirements.txt requirements.txt | |
| 2022-03-23 18:48:21,744 INFO SenderThread:32213 [dir_watcher.py:finish():327] scan save: /home/sanchit_huggingface_co/flax-wav2vec2-2-bart-large-cnn-gradient-accumulation/wandb/run-20220323_184406-3j6a5wn7/files/output.log output.log | |
| 2022-03-23 18:48:21,744 INFO SenderThread:32213 [dir_watcher.py:finish():327] scan save: /home/sanchit_huggingface_co/flax-wav2vec2-2-bart-large-cnn-gradient-accumulation/wandb/run-20220323_184406-3j6a5wn7/files/wandb-metadata.json wandb-metadata.json | |
| 2022-03-23 18:48:21,744 INFO SenderThread:32213 [dir_watcher.py:finish():327] scan save: /home/sanchit_huggingface_co/flax-wav2vec2-2-bart-large-cnn-gradient-accumulation/wandb/run-20220323_184406-3j6a5wn7/files/config.yaml config.yaml | |
| 2022-03-23 18:48:21,744 INFO SenderThread:32213 [dir_watcher.py:finish():327] scan save: /home/sanchit_huggingface_co/flax-wav2vec2-2-bart-large-cnn-gradient-accumulation/wandb/run-20220323_184406-3j6a5wn7/files/wandb-summary.json wandb-summary.json | |
| 2022-03-23 18:48:21,745 INFO SenderThread:32213 [sender.py:transition_state():392] send defer: 7 | |
| 2022-03-23 18:48:21,750 DEBUG HandlerThread:32213 [handler.py:handle_request():141] handle_request: defer | |
| 2022-03-23 18:48:21,750 INFO HandlerThread:32213 [handler.py:handle_request_defer():164] handle defer: 7 | |
| 2022-03-23 18:48:21,750 DEBUG SenderThread:32213 [sender.py:send_request():249] send_request: defer | |
| 2022-03-23 18:48:21,750 INFO SenderThread:32213 [sender.py:send_request_defer():388] handle sender defer: 7 | |
| 2022-03-23 18:48:21,750 INFO SenderThread:32213 [file_pusher.py:finish():145] shutting down file pusher | |
| 2022-03-23 18:48:21,835 DEBUG HandlerThread:32213 [handler.py:handle_request():141] handle_request: poll_exit | |
| 2022-03-23 18:48:21,835 DEBUG SenderThread:32213 [sender.py:send_request():249] send_request: poll_exit | |
| 2022-03-23 18:48:21,936 DEBUG HandlerThread:32213 [handler.py:handle_request():141] handle_request: poll_exit | |
| 2022-03-23 18:48:21,937 DEBUG SenderThread:32213 [sender.py:send_request():249] send_request: poll_exit | |
| 2022-03-23 18:48:22,038 DEBUG HandlerThread:32213 [handler.py:handle_request():141] handle_request: poll_exit | |
| 2022-03-23 18:48:22,038 DEBUG SenderThread:32213 [sender.py:send_request():249] send_request: poll_exit | |
| 2022-03-23 18:48:22,054 INFO Thread-13 :32213 [upload_job.py:push():137] Uploaded file /home/sanchit_huggingface_co/flax-wav2vec2-2-bart-large-cnn-gradient-accumulation/wandb/run-20220323_184406-3j6a5wn7/files/output.log | |
| 2022-03-23 18:48:22,117 INFO Thread-14 :32213 [upload_job.py:push():137] Uploaded file /home/sanchit_huggingface_co/flax-wav2vec2-2-bart-large-cnn-gradient-accumulation/wandb/run-20220323_184406-3j6a5wn7/files/config.yaml | |
| 2022-03-23 18:48:22,140 DEBUG HandlerThread:32213 [handler.py:handle_request():141] handle_request: poll_exit | |
| 2022-03-23 18:48:22,140 DEBUG SenderThread:32213 [sender.py:send_request():249] send_request: poll_exit | |
| 2022-03-23 18:48:22,203 INFO Thread-12 :32213 [upload_job.py:push():137] Uploaded file /home/sanchit_huggingface_co/flax-wav2vec2-2-bart-large-cnn-gradient-accumulation/wandb/run-20220323_184406-3j6a5wn7/files/requirements.txt | |
| 2022-03-23 18:48:22,212 INFO Thread-15 :32213 [upload_job.py:push():137] Uploaded file /home/sanchit_huggingface_co/flax-wav2vec2-2-bart-large-cnn-gradient-accumulation/wandb/run-20220323_184406-3j6a5wn7/files/wandb-summary.json | |
| 2022-03-23 18:48:22,242 DEBUG HandlerThread:32213 [handler.py:handle_request():141] handle_request: poll_exit | |
| 2022-03-23 18:48:22,242 DEBUG SenderThread:32213 [sender.py:send_request():249] send_request: poll_exit | |
| 2022-03-23 18:48:22,343 DEBUG HandlerThread:32213 [handler.py:handle_request():141] handle_request: poll_exit | |
| 2022-03-23 18:48:22,344 DEBUG SenderThread:32213 [sender.py:send_request():249] send_request: poll_exit | |
| 2022-03-23 18:48:22,412 INFO Thread-6 :32213 [sender.py:transition_state():392] send defer: 8 | |
| 2022-03-23 18:48:22,413 DEBUG HandlerThread:32213 [handler.py:handle_request():141] handle_request: defer | |
| 2022-03-23 18:48:22,413 INFO HandlerThread:32213 [handler.py:handle_request_defer():164] handle defer: 8 | |
| 2022-03-23 18:48:22,413 DEBUG SenderThread:32213 [sender.py:send_request():249] send_request: defer | |
| 2022-03-23 18:48:22,413 INFO SenderThread:32213 [sender.py:send_request_defer():388] handle sender defer: 8 | |
| 2022-03-23 18:48:22,445 DEBUG HandlerThread:32213 [handler.py:handle_request():141] handle_request: poll_exit | |
| 2022-03-23 18:48:22,557 INFO SenderThread:32213 [sender.py:transition_state():392] send defer: 9 | |
| 2022-03-23 18:48:22,557 DEBUG SenderThread:32213 [sender.py:send_request():249] send_request: poll_exit | |
| 2022-03-23 18:48:22,558 DEBUG HandlerThread:32213 [handler.py:handle_request():141] handle_request: defer | |
| 2022-03-23 18:48:22,558 INFO HandlerThread:32213 [handler.py:handle_request_defer():164] handle defer: 9 | |
| 2022-03-23 18:48:22,558 DEBUG SenderThread:32213 [sender.py:send_request():249] send_request: defer | |
| 2022-03-23 18:48:22,558 INFO SenderThread:32213 [sender.py:send_request_defer():388] handle sender defer: 9 | |
| 2022-03-23 18:48:22,558 INFO SenderThread:32213 [sender.py:transition_state():392] send defer: 10 | |
| 2022-03-23 18:48:22,559 DEBUG SenderThread:32213 [sender.py:send():235] send: final | |
| 2022-03-23 18:48:22,559 DEBUG SenderThread:32213 [sender.py:send():235] send: footer | |
| 2022-03-23 18:48:22,559 DEBUG HandlerThread:32213 [handler.py:handle_request():141] handle_request: defer | |
| 2022-03-23 18:48:22,559 INFO HandlerThread:32213 [handler.py:handle_request_defer():164] handle defer: 10 | |
| 2022-03-23 18:48:22,559 DEBUG SenderThread:32213 [sender.py:send_request():249] send_request: defer | |
| 2022-03-23 18:48:22,559 INFO SenderThread:32213 [sender.py:send_request_defer():388] handle sender defer: 10 | |
| 2022-03-23 18:48:22,659 DEBUG HandlerThread:32213 [handler.py:handle_request():141] handle_request: poll_exit | |
| 2022-03-23 18:48:22,659 DEBUG SenderThread:32213 [sender.py:send_request():249] send_request: poll_exit | |
| 2022-03-23 18:48:22,659 INFO SenderThread:32213 [file_pusher.py:join():150] waiting for file pusher | |
| 2022-03-23 18:48:23,093 DEBUG HandlerThread:32213 [handler.py:handle_request():141] handle_request: sampled_history | |
| 2022-03-23 18:48:23,094 DEBUG HandlerThread:32213 [handler.py:handle_request():141] handle_request: get_summary | |
| 2022-03-23 18:48:23,095 DEBUG HandlerThread:32213 [handler.py:handle_request():141] handle_request: shutdown | |
| 2022-03-23 18:48:23,095 INFO HandlerThread:32213 [handler.py:finish():778] shutting down handler | |
| 2022-03-23 18:48:23,559 INFO WriterThread:32213 [datastore.py:close():281] close: /home/sanchit_huggingface_co/flax-wav2vec2-2-bart-large-cnn-gradient-accumulation/wandb/run-20220323_184406-3j6a5wn7/run-3j6a5wn7.wandb | |
| 2022-03-23 18:48:23,992 INFO SenderThread:32213 [sender.py:finish():1078] shutting down sender | |
| 2022-03-23 18:48:23,992 INFO SenderThread:32213 [file_pusher.py:finish():145] shutting down file pusher | |
| 2022-03-23 18:48:23,992 INFO SenderThread:32213 [file_pusher.py:join():150] waiting for file pusher | |
| 2022-03-23 18:48:23,995 INFO MainThread:32213 [internal.py:handle_exit():82] Internal process exited | |