flax-wav2vec2-2-bart-large-cnn-gradient-accumulation
/
wandb
/run-20220323_170702-w50ub6ac
/logs
/debug-internal.log
| 2022-03-23 17:07:03,443 INFO MainThread:14495 [internal.py:wandb_internal():92] W&B internal server running at pid: 14495, started at: 2022-03-23 17:07:03.443406 | |
| 2022-03-23 17:07:03,469 INFO WriterThread:14495 [datastore.py:open_for_write():77] open: /home/sanchit_huggingface_co/flax-wav2vec2-2-bart-large-cnn-gradient-accumulation/wandb/run-20220323_170702-w50ub6ac/run-w50ub6ac.wandb | |
| 2022-03-23 17:07:03,471 DEBUG SenderThread:14495 [sender.py:send():235] send: header | |
| 2022-03-23 17:07:03,471 DEBUG SenderThread:14495 [sender.py:send():235] send: run | |
| 2022-03-23 17:07:03,629 DEBUG HandlerThread:14495 [handler.py:handle_request():141] handle_request: check_version | |
| 2022-03-23 17:07:03,630 INFO SenderThread:14495 [dir_watcher.py:__init__():169] watching files in: /home/sanchit_huggingface_co/flax-wav2vec2-2-bart-large-cnn-gradient-accumulation/wandb/run-20220323_170702-w50ub6ac/files | |
| 2022-03-23 17:07:03,630 INFO SenderThread:14495 [sender.py:_start_run_threads():812] run started: w50ub6ac with start time 1648055222 | |
| 2022-03-23 17:07:03,630 DEBUG SenderThread:14495 [sender.py:send():235] send: summary | |
| 2022-03-23 17:07:03,630 INFO SenderThread:14495 [sender.py:_save_file():947] saving file wandb-summary.json with policy end | |
| 2022-03-23 17:07:03,630 DEBUG SenderThread:14495 [sender.py:send_request():249] send_request: check_version | |
| 2022-03-23 17:07:03,706 DEBUG HandlerThread:14495 [handler.py:handle_request():141] handle_request: run_start | |
| 2022-03-23 17:07:04,633 INFO Thread-7 :14495 [dir_watcher.py:_on_file_created():217] file/dir created: /home/sanchit_huggingface_co/flax-wav2vec2-2-bart-large-cnn-gradient-accumulation/wandb/run-20220323_170702-w50ub6ac/files/wandb-summary.json | |
| 2022-03-23 17:07:06,080 DEBUG HandlerThread:14495 [meta.py:__init__():37] meta init | |
| 2022-03-23 17:07:06,080 DEBUG HandlerThread:14495 [meta.py:__init__():51] meta init done | |
| 2022-03-23 17:07:06,080 DEBUG HandlerThread:14495 [meta.py:probe():211] probe | |
| 2022-03-23 17:07:06,081 DEBUG HandlerThread:14495 [meta.py:_setup_git():201] setup git | |
| 2022-03-23 17:07:06,114 DEBUG HandlerThread:14495 [meta.py:_setup_git():208] setup git done | |
| 2022-03-23 17:07:06,114 DEBUG HandlerThread:14495 [meta.py:_save_pip():55] save pip | |
| 2022-03-23 17:07:06,115 DEBUG HandlerThread:14495 [meta.py:_save_pip():69] save pip done | |
| 2022-03-23 17:07:06,115 DEBUG HandlerThread:14495 [meta.py:probe():249] probe done | |
| 2022-03-23 17:07:06,118 DEBUG SenderThread:14495 [sender.py:send():235] send: files | |
| 2022-03-23 17:07:06,118 INFO SenderThread:14495 [sender.py:_save_file():947] saving file wandb-metadata.json with policy now | |
| 2022-03-23 17:07:06,125 DEBUG HandlerThread:14495 [handler.py:handle_request():141] handle_request: stop_status | |
| 2022-03-23 17:07:06,125 DEBUG SenderThread:14495 [sender.py:send_request():249] send_request: stop_status | |
| 2022-03-23 17:07:06,199 DEBUG SenderThread:14495 [sender.py:send():235] send: telemetry | |
| 2022-03-23 17:07:06,451 INFO Thread-11 :14495 [upload_job.py:push():137] Uploaded file /tmp/tmpvkf_ouoiwandb/2u99pz4k-wandb-metadata.json | |
| 2022-03-23 17:07:06,634 INFO Thread-7 :14495 [dir_watcher.py:_on_file_created():217] file/dir created: /home/sanchit_huggingface_co/flax-wav2vec2-2-bart-large-cnn-gradient-accumulation/wandb/run-20220323_170702-w50ub6ac/files/requirements.txt | |
| 2022-03-23 17:07:06,635 INFO Thread-7 :14495 [dir_watcher.py:_on_file_created():217] file/dir created: /home/sanchit_huggingface_co/flax-wav2vec2-2-bart-large-cnn-gradient-accumulation/wandb/run-20220323_170702-w50ub6ac/files/wandb-metadata.json | |
| 2022-03-23 17:07:06,635 INFO Thread-7 :14495 [dir_watcher.py:_on_file_created():217] file/dir created: /home/sanchit_huggingface_co/flax-wav2vec2-2-bart-large-cnn-gradient-accumulation/wandb/run-20220323_170702-w50ub6ac/files/output.log | |
| 2022-03-23 17:07:08,635 INFO Thread-7 :14495 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchit_huggingface_co/flax-wav2vec2-2-bart-large-cnn-gradient-accumulation/wandb/run-20220323_170702-w50ub6ac/files/output.log | |
| 2022-03-23 17:07:21,201 DEBUG HandlerThread:14495 [handler.py:handle_request():141] handle_request: stop_status | |
| 2022-03-23 17:07:21,201 DEBUG SenderThread:14495 [sender.py:send_request():249] send_request: stop_status | |
| 2022-03-23 17:07:24,642 INFO Thread-7 :14495 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchit_huggingface_co/flax-wav2vec2-2-bart-large-cnn-gradient-accumulation/wandb/run-20220323_170702-w50ub6ac/files/output.log | |
| 2022-03-23 17:07:28,643 INFO Thread-7 :14495 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchit_huggingface_co/flax-wav2vec2-2-bart-large-cnn-gradient-accumulation/wandb/run-20220323_170702-w50ub6ac/files/output.log | |
| 2022-03-23 17:07:30,644 INFO Thread-7 :14495 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchit_huggingface_co/flax-wav2vec2-2-bart-large-cnn-gradient-accumulation/wandb/run-20220323_170702-w50ub6ac/files/output.log | |
| 2022-03-23 17:07:34,165 DEBUG SenderThread:14495 [sender.py:send():235] send: stats | |
| 2022-03-23 17:07:34,646 INFO Thread-7 :14495 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchit_huggingface_co/flax-wav2vec2-2-bart-large-cnn-gradient-accumulation/wandb/run-20220323_170702-w50ub6ac/files/config.yaml | |
| 2022-03-23 17:07:34,646 INFO Thread-7 :14495 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchit_huggingface_co/flax-wav2vec2-2-bart-large-cnn-gradient-accumulation/wandb/run-20220323_170702-w50ub6ac/files/output.log | |
| 2022-03-23 17:07:36,413 DEBUG HandlerThread:14495 [handler.py:handle_request():141] handle_request: stop_status | |
| 2022-03-23 17:07:36,414 DEBUG SenderThread:14495 [sender.py:send_request():249] send_request: stop_status | |
| 2022-03-23 17:07:38,648 INFO Thread-7 :14495 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchit_huggingface_co/flax-wav2vec2-2-bart-large-cnn-gradient-accumulation/wandb/run-20220323_170702-w50ub6ac/files/output.log | |
| 2022-03-23 17:07:51,584 DEBUG HandlerThread:14495 [handler.py:handle_request():141] handle_request: stop_status | |
| 2022-03-23 17:07:51,584 DEBUG SenderThread:14495 [sender.py:send_request():249] send_request: stop_status | |
| 2022-03-23 17:07:53,609 DEBUG HandlerThread:14495 [handler.py:handle_request():141] handle_request: partial_history | |
| 2022-03-23 17:07:53,610 DEBUG SenderThread:14495 [sender.py:send():235] send: telemetry | |
| 2022-03-23 17:07:54,654 INFO Thread-7 :14495 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchit_huggingface_co/flax-wav2vec2-2-bart-large-cnn-gradient-accumulation/wandb/run-20220323_170702-w50ub6ac/files/output.log | |
| 2022-03-23 17:07:55,871 DEBUG HandlerThread:14495 [handler.py:handle_request():141] handle_request: poll_exit | |
| 2022-03-23 17:07:55,872 DEBUG SenderThread:14495 [sender.py:send():235] send: exit | |
| 2022-03-23 17:07:55,872 INFO SenderThread:14495 [sender.py:send_exit():371] handling exit code: 1 | |
| 2022-03-23 17:07:55,872 INFO SenderThread:14495 [sender.py:send_exit():373] handling runtime: 52 | |
| 2022-03-23 17:07:55,873 INFO SenderThread:14495 [sender.py:_save_file():947] saving file wandb-summary.json with policy end | |
| 2022-03-23 17:07:55,873 INFO SenderThread:14495 [sender.py:send_exit():379] send defer | |
| 2022-03-23 17:07:55,873 DEBUG SenderThread:14495 [sender.py:send_request():249] send_request: poll_exit | |
| 2022-03-23 17:07:55,873 DEBUG HandlerThread:14495 [handler.py:handle_request():141] handle_request: defer | |
| 2022-03-23 17:07:55,874 INFO HandlerThread:14495 [handler.py:handle_request_defer():164] handle defer: 0 | |
| 2022-03-23 17:07:55,874 DEBUG SenderThread:14495 [sender.py:send_request():249] send_request: defer | |
| 2022-03-23 17:07:55,874 INFO SenderThread:14495 [sender.py:send_request_defer():388] handle sender defer: 0 | |
| 2022-03-23 17:07:55,874 INFO SenderThread:14495 [sender.py:transition_state():392] send defer: 1 | |
| 2022-03-23 17:07:55,874 DEBUG HandlerThread:14495 [handler.py:handle_request():141] handle_request: defer | |
| 2022-03-23 17:07:55,874 INFO HandlerThread:14495 [handler.py:handle_request_defer():164] handle defer: 1 | |
| 2022-03-23 17:07:55,918 DEBUG SenderThread:14495 [sender.py:send_request():249] send_request: defer | |
| 2022-03-23 17:07:55,918 INFO SenderThread:14495 [sender.py:send_request_defer():388] handle sender defer: 1 | |
| 2022-03-23 17:07:55,918 INFO SenderThread:14495 [sender.py:transition_state():392] send defer: 2 | |
| 2022-03-23 17:07:55,918 DEBUG SenderThread:14495 [sender.py:send():235] send: stats | |
| 2022-03-23 17:07:55,918 DEBUG HandlerThread:14495 [handler.py:handle_request():141] handle_request: defer | |
| 2022-03-23 17:07:55,918 INFO HandlerThread:14495 [handler.py:handle_request_defer():164] handle defer: 2 | |
| 2022-03-23 17:07:55,919 DEBUG SenderThread:14495 [sender.py:send_request():249] send_request: defer | |
| 2022-03-23 17:07:55,919 INFO SenderThread:14495 [sender.py:send_request_defer():388] handle sender defer: 2 | |
| 2022-03-23 17:07:55,919 INFO SenderThread:14495 [sender.py:transition_state():392] send defer: 3 | |
| 2022-03-23 17:07:55,919 DEBUG HandlerThread:14495 [handler.py:handle_request():141] handle_request: defer | |
| 2022-03-23 17:07:55,919 INFO HandlerThread:14495 [handler.py:handle_request_defer():164] handle defer: 3 | |
| 2022-03-23 17:07:55,919 DEBUG SenderThread:14495 [sender.py:send_request():249] send_request: defer | |
| 2022-03-23 17:07:55,919 INFO SenderThread:14495 [sender.py:send_request_defer():388] handle sender defer: 3 | |
| 2022-03-23 17:07:55,919 INFO SenderThread:14495 [sender.py:transition_state():392] send defer: 4 | |
| 2022-03-23 17:07:55,919 DEBUG HandlerThread:14495 [handler.py:handle_request():141] handle_request: defer | |
| 2022-03-23 17:07:55,920 INFO HandlerThread:14495 [handler.py:handle_request_defer():164] handle defer: 4 | |
| 2022-03-23 17:07:55,920 DEBUG SenderThread:14495 [sender.py:send():235] send: summary | |
| 2022-03-23 17:07:55,920 INFO SenderThread:14495 [sender.py:_save_file():947] saving file wandb-summary.json with policy end | |
| 2022-03-23 17:07:55,920 DEBUG SenderThread:14495 [sender.py:send_request():249] send_request: defer | |
| 2022-03-23 17:07:55,920 INFO SenderThread:14495 [sender.py:send_request_defer():388] handle sender defer: 4 | |
| 2022-03-23 17:07:55,920 INFO SenderThread:14495 [sender.py:transition_state():392] send defer: 5 | |
| 2022-03-23 17:07:55,921 DEBUG HandlerThread:14495 [handler.py:handle_request():141] handle_request: defer | |
| 2022-03-23 17:07:55,921 INFO HandlerThread:14495 [handler.py:handle_request_defer():164] handle defer: 5 | |
| 2022-03-23 17:07:55,921 DEBUG SenderThread:14495 [sender.py:send_request():249] send_request: defer | |
| 2022-03-23 17:07:55,921 INFO SenderThread:14495 [sender.py:send_request_defer():388] handle sender defer: 5 | |
| 2022-03-23 17:07:55,975 DEBUG HandlerThread:14495 [handler.py:handle_request():141] handle_request: poll_exit | |
| 2022-03-23 17:07:55,996 INFO SenderThread:14495 [sender.py:transition_state():392] send defer: 6 | |
| 2022-03-23 17:07:55,996 DEBUG SenderThread:14495 [sender.py:send_request():249] send_request: poll_exit | |
| 2022-03-23 17:07:55,996 DEBUG HandlerThread:14495 [handler.py:handle_request():141] handle_request: defer | |
| 2022-03-23 17:07:55,996 INFO HandlerThread:14495 [handler.py:handle_request_defer():164] handle defer: 6 | |
| 2022-03-23 17:07:55,997 DEBUG SenderThread:14495 [sender.py:send_request():249] send_request: defer | |
| 2022-03-23 17:07:55,997 INFO SenderThread:14495 [sender.py:send_request_defer():388] handle sender defer: 6 | |
| 2022-03-23 17:07:55,997 INFO SenderThread:14495 [dir_watcher.py:finish():283] shutting down directory watcher | |
| 2022-03-23 17:07:56,097 DEBUG HandlerThread:14495 [handler.py:handle_request():141] handle_request: poll_exit | |
| 2022-03-23 17:07:56,655 INFO Thread-7 :14495 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchit_huggingface_co/flax-wav2vec2-2-bart-large-cnn-gradient-accumulation/wandb/run-20220323_170702-w50ub6ac/files/output.log | |
| 2022-03-23 17:07:56,656 INFO SenderThread:14495 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchit_huggingface_co/flax-wav2vec2-2-bart-large-cnn-gradient-accumulation/wandb/run-20220323_170702-w50ub6ac/files/config.yaml | |
| 2022-03-23 17:07:56,656 INFO SenderThread:14495 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchit_huggingface_co/flax-wav2vec2-2-bart-large-cnn-gradient-accumulation/wandb/run-20220323_170702-w50ub6ac/files/wandb-summary.json | |
| 2022-03-23 17:07:56,656 INFO SenderThread:14495 [dir_watcher.py:finish():313] scan: /home/sanchit_huggingface_co/flax-wav2vec2-2-bart-large-cnn-gradient-accumulation/wandb/run-20220323_170702-w50ub6ac/files | |
| 2022-03-23 17:07:56,656 INFO SenderThread:14495 [dir_watcher.py:finish():327] scan save: /home/sanchit_huggingface_co/flax-wav2vec2-2-bart-large-cnn-gradient-accumulation/wandb/run-20220323_170702-w50ub6ac/files/requirements.txt requirements.txt | |
| 2022-03-23 17:07:56,656 INFO SenderThread:14495 [dir_watcher.py:finish():327] scan save: /home/sanchit_huggingface_co/flax-wav2vec2-2-bart-large-cnn-gradient-accumulation/wandb/run-20220323_170702-w50ub6ac/files/output.log output.log | |
| 2022-03-23 17:07:56,656 INFO SenderThread:14495 [dir_watcher.py:finish():327] scan save: /home/sanchit_huggingface_co/flax-wav2vec2-2-bart-large-cnn-gradient-accumulation/wandb/run-20220323_170702-w50ub6ac/files/wandb-metadata.json wandb-metadata.json | |
| 2022-03-23 17:07:56,656 INFO SenderThread:14495 [dir_watcher.py:finish():327] scan save: /home/sanchit_huggingface_co/flax-wav2vec2-2-bart-large-cnn-gradient-accumulation/wandb/run-20220323_170702-w50ub6ac/files/config.yaml config.yaml | |
| 2022-03-23 17:07:56,657 INFO SenderThread:14495 [dir_watcher.py:finish():327] scan save: /home/sanchit_huggingface_co/flax-wav2vec2-2-bart-large-cnn-gradient-accumulation/wandb/run-20220323_170702-w50ub6ac/files/wandb-summary.json wandb-summary.json | |
| 2022-03-23 17:07:56,657 INFO SenderThread:14495 [sender.py:transition_state():392] send defer: 7 | |
| 2022-03-23 17:07:56,657 DEBUG SenderThread:14495 [sender.py:send_request():249] send_request: poll_exit | |
| 2022-03-23 17:07:56,658 DEBUG HandlerThread:14495 [handler.py:handle_request():141] handle_request: defer | |
| 2022-03-23 17:07:56,658 INFO HandlerThread:14495 [handler.py:handle_request_defer():164] handle defer: 7 | |
| 2022-03-23 17:07:56,658 DEBUG SenderThread:14495 [sender.py:send_request():249] send_request: defer | |
| 2022-03-23 17:07:56,658 INFO SenderThread:14495 [sender.py:send_request_defer():388] handle sender defer: 7 | |
| 2022-03-23 17:07:56,658 INFO SenderThread:14495 [file_pusher.py:finish():145] shutting down file pusher | |
| 2022-03-23 17:07:56,759 DEBUG HandlerThread:14495 [handler.py:handle_request():141] handle_request: poll_exit | |
| 2022-03-23 17:07:56,759 DEBUG SenderThread:14495 [sender.py:send_request():249] send_request: poll_exit | |
| 2022-03-23 17:07:56,861 DEBUG HandlerThread:14495 [handler.py:handle_request():141] handle_request: poll_exit | |
| 2022-03-23 17:07:56,861 DEBUG SenderThread:14495 [sender.py:send_request():249] send_request: poll_exit | |
| 2022-03-23 17:07:56,916 INFO Thread-13 :14495 [upload_job.py:push():137] Uploaded file /home/sanchit_huggingface_co/flax-wav2vec2-2-bart-large-cnn-gradient-accumulation/wandb/run-20220323_170702-w50ub6ac/files/output.log | |
| 2022-03-23 17:07:56,924 INFO Thread-14 :14495 [upload_job.py:push():137] Uploaded file /home/sanchit_huggingface_co/flax-wav2vec2-2-bart-large-cnn-gradient-accumulation/wandb/run-20220323_170702-w50ub6ac/files/config.yaml | |
| 2022-03-23 17:07:56,944 INFO Thread-15 :14495 [upload_job.py:push():137] Uploaded file /home/sanchit_huggingface_co/flax-wav2vec2-2-bart-large-cnn-gradient-accumulation/wandb/run-20220323_170702-w50ub6ac/files/wandb-summary.json | |
| 2022-03-23 17:07:56,948 INFO Thread-12 :14495 [upload_job.py:push():137] Uploaded file /home/sanchit_huggingface_co/flax-wav2vec2-2-bart-large-cnn-gradient-accumulation/wandb/run-20220323_170702-w50ub6ac/files/requirements.txt | |
| 2022-03-23 17:07:56,963 DEBUG HandlerThread:14495 [handler.py:handle_request():141] handle_request: poll_exit | |
| 2022-03-23 17:07:56,963 DEBUG SenderThread:14495 [sender.py:send_request():249] send_request: poll_exit | |
| 2022-03-23 17:07:57,064 DEBUG HandlerThread:14495 [handler.py:handle_request():141] handle_request: poll_exit | |
| 2022-03-23 17:07:57,065 DEBUG SenderThread:14495 [sender.py:send_request():249] send_request: poll_exit | |
| 2022-03-23 17:07:57,148 INFO Thread-6 :14495 [sender.py:transition_state():392] send defer: 8 | |
| 2022-03-23 17:07:57,149 DEBUG HandlerThread:14495 [handler.py:handle_request():141] handle_request: defer | |
| 2022-03-23 17:07:57,149 INFO HandlerThread:14495 [handler.py:handle_request_defer():164] handle defer: 8 | |
| 2022-03-23 17:07:57,149 DEBUG SenderThread:14495 [sender.py:send_request():249] send_request: defer | |
| 2022-03-23 17:07:57,149 INFO SenderThread:14495 [sender.py:send_request_defer():388] handle sender defer: 8 | |
| 2022-03-23 17:07:57,166 DEBUG HandlerThread:14495 [handler.py:handle_request():141] handle_request: poll_exit | |
| 2022-03-23 17:07:57,212 INFO SenderThread:14495 [sender.py:transition_state():392] send defer: 9 | |
| 2022-03-23 17:07:57,212 DEBUG SenderThread:14495 [sender.py:send_request():249] send_request: poll_exit | |
| 2022-03-23 17:07:57,212 DEBUG HandlerThread:14495 [handler.py:handle_request():141] handle_request: defer | |
| 2022-03-23 17:07:57,212 INFO HandlerThread:14495 [handler.py:handle_request_defer():164] handle defer: 9 | |
| 2022-03-23 17:07:57,213 DEBUG SenderThread:14495 [sender.py:send_request():249] send_request: defer | |
| 2022-03-23 17:07:57,213 INFO SenderThread:14495 [sender.py:send_request_defer():388] handle sender defer: 9 | |
| 2022-03-23 17:07:57,213 INFO SenderThread:14495 [sender.py:transition_state():392] send defer: 10 | |
| 2022-03-23 17:07:57,213 DEBUG SenderThread:14495 [sender.py:send():235] send: final | |
| 2022-03-23 17:07:57,213 DEBUG SenderThread:14495 [sender.py:send():235] send: footer | |
| 2022-03-23 17:07:57,213 DEBUG HandlerThread:14495 [handler.py:handle_request():141] handle_request: defer | |
| 2022-03-23 17:07:57,214 INFO HandlerThread:14495 [handler.py:handle_request_defer():164] handle defer: 10 | |
| 2022-03-23 17:07:57,214 DEBUG SenderThread:14495 [sender.py:send_request():249] send_request: defer | |
| 2022-03-23 17:07:57,214 INFO SenderThread:14495 [sender.py:send_request_defer():388] handle sender defer: 10 | |
| 2022-03-23 17:07:57,313 DEBUG HandlerThread:14495 [handler.py:handle_request():141] handle_request: poll_exit | |
| 2022-03-23 17:07:57,314 DEBUG SenderThread:14495 [sender.py:send_request():249] send_request: poll_exit | |
| 2022-03-23 17:07:57,314 INFO SenderThread:14495 [file_pusher.py:join():150] waiting for file pusher | |
| 2022-03-23 17:07:57,507 DEBUG HandlerThread:14495 [handler.py:handle_request():141] handle_request: sampled_history | |
| 2022-03-23 17:07:57,508 DEBUG HandlerThread:14495 [handler.py:handle_request():141] handle_request: get_summary | |
| 2022-03-23 17:07:57,509 DEBUG HandlerThread:14495 [handler.py:handle_request():141] handle_request: shutdown | |
| 2022-03-23 17:07:57,509 INFO HandlerThread:14495 [handler.py:finish():778] shutting down handler | |
| 2022-03-23 17:07:58,214 INFO WriterThread:14495 [datastore.py:close():281] close: /home/sanchit_huggingface_co/flax-wav2vec2-2-bart-large-cnn-gradient-accumulation/wandb/run-20220323_170702-w50ub6ac/run-w50ub6ac.wandb | |
| 2022-03-23 17:07:58,406 INFO SenderThread:14495 [sender.py:finish():1078] shutting down sender | |
| 2022-03-23 17:07:58,406 INFO SenderThread:14495 [file_pusher.py:finish():145] shutting down file pusher | |
| 2022-03-23 17:07:58,406 INFO SenderThread:14495 [file_pusher.py:join():150] waiting for file pusher | |
| 2022-03-23 17:07:58,409 INFO MainThread:14495 [internal.py:handle_exit():82] Internal process exited | |