File size: 10,628 Bytes
6c4a6af
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
2022-03-02 09:41:37,076 INFO    MainThread:255753 [internal.py:wandb_internal():89] W&B internal server running at pid: 255753, started at: 2022-03-02 09:41:37.076228
2022-03-02 09:41:37,078 DEBUG   HandlerThread:255753 [handler.py:handle_request():131] handle_request: check_version
2022-03-02 09:41:37,078 INFO    WriterThread:255753 [datastore.py:open_for_write():77] open: /home/sanchit_huggingface_co/wav2vec2-2-rnd-grid-search/wandb/run-20220302_094136-agx803un/run-agx803un.wandb
2022-03-02 09:41:37,080 DEBUG   SenderThread:255753 [sender.py:send():235] send: header
2022-03-02 09:41:37,080 DEBUG   SenderThread:255753 [sender.py:send_request():249] send_request: check_version
2022-03-02 09:41:37,147 DEBUG   SenderThread:255753 [sender.py:send():235] send: run
2022-03-02 09:41:37,334 INFO    SenderThread:255753 [dir_watcher.py:__init__():169] watching files in: /home/sanchit_huggingface_co/wav2vec2-2-rnd-grid-search/wandb/run-20220302_094136-agx803un/files
2022-03-02 09:41:37,334 INFO    SenderThread:255753 [sender.py:_start_run_threads():809] run started: agx803un with start time 1646214096
2022-03-02 09:41:37,334 DEBUG   SenderThread:255753 [sender.py:send():235] send: summary
2022-03-02 09:41:37,334 INFO    SenderThread:255753 [sender.py:_save_file():944] saving file wandb-summary.json with policy end
2022-03-02 09:41:37,336 DEBUG   HandlerThread:255753 [handler.py:handle_request():131] handle_request: run_start
2022-03-02 09:41:37,341 DEBUG   HandlerThread:255753 [meta.py:__init__():36] meta init
2022-03-02 09:41:37,341 DEBUG   HandlerThread:255753 [meta.py:__init__():50] meta init done
2022-03-02 09:41:37,341 DEBUG   HandlerThread:255753 [meta.py:probe():210] probe
2022-03-02 09:41:37,347 DEBUG   HandlerThread:255753 [meta.py:_setup_git():200] setup git
2022-03-02 09:41:37,362 DEBUG   HandlerThread:255753 [meta.py:_setup_git():207] setup git done
2022-03-02 09:41:37,363 DEBUG   HandlerThread:255753 [meta.py:_save_pip():54] save pip
2022-03-02 09:41:37,363 DEBUG   HandlerThread:255753 [meta.py:_save_pip():68] save pip done
2022-03-02 09:41:37,364 DEBUG   HandlerThread:255753 [meta.py:probe():248] probe done
2022-03-02 09:41:37,448 DEBUG   SenderThread:255753 [sender.py:send():235] send: files
2022-03-02 09:41:37,449 INFO    SenderThread:255753 [sender.py:_save_file():944] saving file wandb-metadata.json with policy now
2022-03-02 09:41:37,455 DEBUG   HandlerThread:255753 [handler.py:handle_request():131] handle_request: stop_status
2022-03-02 09:41:37,456 DEBUG   SenderThread:255753 [sender.py:send_request():249] send_request: stop_status
2022-03-02 09:41:37,487 DEBUG   SenderThread:255753 [sender.py:send():235] send: config
2022-03-02 09:41:37,488 DEBUG   SenderThread:255753 [sender.py:send():235] send: metric
2022-03-02 09:41:37,488 DEBUG   SenderThread:255753 [sender.py:send():235] send: metric
2022-03-02 09:41:37,488 WARNING SenderThread:255753 [sender.py:send_metric():902] Seen metric with glob (shouldnt happen)
2022-03-02 09:41:37,718 INFO    Thread-11 :255753 [upload_job.py:push():137] Uploaded file /tmp/tmp06yxits1wandb/z660dan0-wandb-metadata.json
2022-03-02 09:41:38,336 INFO    Thread-8  :255753 [dir_watcher.py:_on_file_created():217] file/dir created: /home/sanchit_huggingface_co/wav2vec2-2-rnd-grid-search/wandb/run-20220302_094136-agx803un/files/wandb-summary.json
2022-03-02 09:41:38,336 INFO    Thread-8  :255753 [dir_watcher.py:_on_file_created():217] file/dir created: /home/sanchit_huggingface_co/wav2vec2-2-rnd-grid-search/wandb/run-20220302_094136-agx803un/files/requirements.txt
2022-03-02 09:41:38,336 INFO    Thread-8  :255753 [dir_watcher.py:_on_file_created():217] file/dir created: /home/sanchit_huggingface_co/wav2vec2-2-rnd-grid-search/wandb/run-20220302_094136-agx803un/files/output.log
2022-03-02 09:41:38,336 INFO    Thread-8  :255753 [dir_watcher.py:_on_file_created():217] file/dir created: /home/sanchit_huggingface_co/wav2vec2-2-rnd-grid-search/wandb/run-20220302_094136-agx803un/files/wandb-metadata.json
2022-03-02 09:41:40,335 INFO    Thread-8  :255753 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchit_huggingface_co/wav2vec2-2-rnd-grid-search/wandb/run-20220302_094136-agx803un/files/output.log
2022-03-02 09:41:40,438 DEBUG   SenderThread:255753 [sender.py:send():235] send: metric
2022-03-02 09:41:40,438 DEBUG   SenderThread:255753 [sender.py:send():235] send: metric
2022-03-02 09:41:40,438 DEBUG   SenderThread:255753 [sender.py:send():235] send: metric
2022-03-02 09:41:40,438 DEBUG   SenderThread:255753 [sender.py:send():235] send: history
2022-03-02 09:41:40,439 DEBUG   SenderThread:255753 [sender.py:send():235] send: summary
2022-03-02 09:41:40,439 INFO    SenderThread:255753 [sender.py:_save_file():944] saving file wandb-summary.json with policy end
2022-03-02 09:41:41,335 INFO    Thread-8  :255753 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchit_huggingface_co/wav2vec2-2-rnd-grid-search/wandb/run-20220302_094136-agx803un/files/wandb-summary.json
2022-03-02 09:41:42,336 INFO    Thread-8  :255753 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchit_huggingface_co/wav2vec2-2-rnd-grid-search/wandb/run-20220302_094136-agx803un/files/output.log
2022-03-02 09:41:42,687 DEBUG   SenderThread:255753 [sender.py:send():235] send: history
2022-03-02 09:41:42,688 DEBUG   SenderThread:255753 [sender.py:send():235] send: summary
2022-03-02 09:41:42,688 INFO    SenderThread:255753 [sender.py:_save_file():944] saving file wandb-summary.json with policy end
2022-03-02 09:41:43,336 INFO    Thread-8  :255753 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchit_huggingface_co/wav2vec2-2-rnd-grid-search/wandb/run-20220302_094136-agx803un/files/wandb-summary.json
2022-03-02 09:41:44,336 INFO    Thread-8  :255753 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchit_huggingface_co/wav2vec2-2-rnd-grid-search/wandb/run-20220302_094136-agx803un/files/output.log
2022-03-02 09:41:44,985 DEBUG   SenderThread:255753 [sender.py:send():235] send: history
2022-03-02 09:41:44,985 DEBUG   SenderThread:255753 [sender.py:send():235] send: summary
2022-03-02 09:41:44,986 INFO    SenderThread:255753 [sender.py:_save_file():944] saving file wandb-summary.json with policy end
2022-03-02 09:41:45,337 INFO    Thread-8  :255753 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchit_huggingface_co/wav2vec2-2-rnd-grid-search/wandb/run-20220302_094136-agx803un/files/wandb-summary.json
2022-03-02 09:41:46,337 INFO    Thread-8  :255753 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchit_huggingface_co/wav2vec2-2-rnd-grid-search/wandb/run-20220302_094136-agx803un/files/output.log
2022-03-02 09:41:47,219 DEBUG   SenderThread:255753 [sender.py:send():235] send: history
2022-03-02 09:41:47,219 DEBUG   SenderThread:255753 [sender.py:send():235] send: summary
2022-03-02 09:41:47,219 INFO    SenderThread:255753 [sender.py:_save_file():944] saving file wandb-summary.json with policy end
2022-03-02 09:41:47,338 INFO    Thread-8  :255753 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchit_huggingface_co/wav2vec2-2-rnd-grid-search/wandb/run-20220302_094136-agx803un/files/wandb-summary.json
2022-03-02 09:41:48,338 INFO    Thread-8  :255753 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchit_huggingface_co/wav2vec2-2-rnd-grid-search/wandb/run-20220302_094136-agx803un/files/output.log
2022-03-02 09:41:49,478 DEBUG   SenderThread:255753 [sender.py:send():235] send: history
2022-03-02 09:41:49,479 DEBUG   SenderThread:255753 [sender.py:send():235] send: summary
2022-03-02 09:41:49,479 INFO    SenderThread:255753 [sender.py:_save_file():944] saving file wandb-summary.json with policy end
2022-03-02 09:41:50,339 INFO    Thread-8  :255753 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchit_huggingface_co/wav2vec2-2-rnd-grid-search/wandb/run-20220302_094136-agx803un/files/wandb-summary.json
2022-03-02 09:41:50,339 INFO    Thread-8  :255753 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchit_huggingface_co/wav2vec2-2-rnd-grid-search/wandb/run-20220302_094136-agx803un/files/output.log
2022-03-02 09:41:51,747 DEBUG   SenderThread:255753 [sender.py:send():235] send: history
2022-03-02 09:41:51,748 DEBUG   SenderThread:255753 [sender.py:send():235] send: summary
2022-03-02 09:41:51,748 INFO    SenderThread:255753 [sender.py:_save_file():944] saving file wandb-summary.json with policy end
2022-03-02 09:41:52,339 INFO    Thread-8  :255753 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchit_huggingface_co/wav2vec2-2-rnd-grid-search/wandb/run-20220302_094136-agx803un/files/wandb-summary.json
2022-03-02 09:41:52,340 INFO    Thread-8  :255753 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchit_huggingface_co/wav2vec2-2-rnd-grid-search/wandb/run-20220302_094136-agx803un/files/output.log
2022-03-02 09:41:52,599 DEBUG   HandlerThread:255753 [handler.py:handle_request():131] handle_request: stop_status
2022-03-02 09:41:52,600 DEBUG   SenderThread:255753 [sender.py:send_request():249] send_request: stop_status
2022-03-02 09:41:53,950 DEBUG   SenderThread:255753 [sender.py:send():235] send: history
2022-03-02 09:41:53,950 DEBUG   SenderThread:255753 [sender.py:send():235] send: summary
2022-03-02 09:41:53,951 INFO    SenderThread:255753 [sender.py:_save_file():944] saving file wandb-summary.json with policy end
2022-03-02 09:41:54,340 INFO    Thread-8  :255753 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchit_huggingface_co/wav2vec2-2-rnd-grid-search/wandb/run-20220302_094136-agx803un/files/wandb-summary.json
2022-03-02 09:41:54,340 INFO    Thread-8  :255753 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchit_huggingface_co/wav2vec2-2-rnd-grid-search/wandb/run-20220302_094136-agx803un/files/output.log
2022-03-02 09:41:55,989 WARNING MainThread:255753 [internal.py:wandb_internal():152] Internal process interrupt: 1
2022-03-02 09:41:56,204 WARNING MainThread:255753 [internal.py:wandb_internal():152] Internal process interrupt: 2
2022-03-02 09:41:56,204 ERROR   MainThread:255753 [internal.py:wandb_internal():155] Internal process interrupted.
2022-03-02 09:41:56,341 INFO    Thread-8  :255753 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchit_huggingface_co/wav2vec2-2-rnd-grid-search/wandb/run-20220302_094136-agx803un/files/output.log
2022-03-02 09:41:56,530 INFO    MainThread:255753 [internal.py:handle_exit():79] Internal process exited