File size: 19,677 Bytes
22f8323
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
a869d63
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
2022-05-29 12:04:59,324 INFO    MainThread:1721989 [internal.py:wandb_internal():92] W&B internal server running at pid: 1721989, started at: 2022-05-29 12:04:59.324307
2022-05-29 12:04:59,326 INFO    WriterThread:1721989 [datastore.py:open_for_write():77] open: /home/sanchitgandhi/flax-dummy/wandb/run-20220529_120458-qf2iwkac/run-qf2iwkac.wandb
2022-05-29 12:04:59,327 DEBUG   SenderThread:1721989 [sender.py:send():235] send: header
2022-05-29 12:04:59,327 DEBUG   SenderThread:1721989 [sender.py:send():235] send: run
2022-05-29 12:04:59,384 DEBUG   HandlerThread:1721989 [handler.py:handle_request():141] handle_request: check_version
2022-05-29 12:04:59,385 INFO    SenderThread:1721989 [dir_watcher.py:__init__():169] watching files in: /home/sanchitgandhi/flax-dummy/wandb/run-20220529_120458-qf2iwkac/files
2022-05-29 12:04:59,385 INFO    SenderThread:1721989 [sender.py:_start_run_threads():812] run started: qf2iwkac with start time 1653825898
2022-05-29 12:04:59,385 DEBUG   SenderThread:1721989 [sender.py:send():235] send: summary
2022-05-29 12:04:59,385 INFO    SenderThread:1721989 [sender.py:_save_file():947] saving file wandb-summary.json with policy end
2022-05-29 12:04:59,385 DEBUG   SenderThread:1721989 [sender.py:send_request():249] send_request: check_version
2022-05-29 12:04:59,447 DEBUG   HandlerThread:1721989 [handler.py:handle_request():141] handle_request: run_start
2022-05-29 12:05:00,436 INFO    Thread-7  :1721989 [dir_watcher.py:_on_file_created():217] file/dir created: /home/sanchitgandhi/flax-dummy/wandb/run-20220529_120458-qf2iwkac/files/wandb-summary.json
2022-05-29 12:05:01,566 DEBUG   HandlerThread:1721989 [meta.py:__init__():37] meta init
2022-05-29 12:05:01,566 DEBUG   HandlerThread:1721989 [meta.py:__init__():51] meta init done
2022-05-29 12:05:01,566 DEBUG   HandlerThread:1721989 [meta.py:probe():211] probe
2022-05-29 12:05:01,568 DEBUG   HandlerThread:1721989 [meta.py:_setup_git():201] setup git
2022-05-29 12:05:01,597 DEBUG   HandlerThread:1721989 [meta.py:_setup_git():208] setup git done
2022-05-29 12:05:01,597 DEBUG   HandlerThread:1721989 [meta.py:_save_pip():55] save pip
2022-05-29 12:05:01,598 DEBUG   HandlerThread:1721989 [meta.py:_save_pip():69] save pip done
2022-05-29 12:05:01,598 DEBUG   HandlerThread:1721989 [meta.py:probe():249] probe done
2022-05-29 12:05:01,601 DEBUG   SenderThread:1721989 [sender.py:send():235] send: files
2022-05-29 12:05:01,601 INFO    SenderThread:1721989 [sender.py:_save_file():947] saving file wandb-metadata.json with policy now
2022-05-29 12:05:01,608 DEBUG   HandlerThread:1721989 [handler.py:handle_request():141] handle_request: stop_status
2022-05-29 12:05:01,608 DEBUG   SenderThread:1721989 [sender.py:send_request():249] send_request: stop_status
2022-05-29 12:05:01,639 DEBUG   SenderThread:1721989 [sender.py:send():235] send: telemetry
2022-05-29 12:05:01,849 INFO    Thread-11 :1721989 [upload_job.py:push():137] Uploaded file /tmp/tmpl7u6xxw8wandb/12iwbaqf-wandb-metadata.json
2022-05-29 12:05:02,475 INFO    Thread-7  :1721989 [dir_watcher.py:_on_file_created():217] file/dir created: /home/sanchitgandhi/flax-dummy/wandb/run-20220529_120458-qf2iwkac/files/wandb-metadata.json
2022-05-29 12:05:02,475 INFO    Thread-7  :1721989 [dir_watcher.py:_on_file_created():217] file/dir created: /home/sanchitgandhi/flax-dummy/wandb/run-20220529_120458-qf2iwkac/files/requirements.txt
2022-05-29 12:05:02,475 INFO    Thread-7  :1721989 [dir_watcher.py:_on_file_created():217] file/dir created: /home/sanchitgandhi/flax-dummy/wandb/run-20220529_120458-qf2iwkac/files/output.log
2022-05-29 12:05:04,476 INFO    Thread-7  :1721989 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-dummy/wandb/run-20220529_120458-qf2iwkac/files/output.log
2022-05-29 12:05:06,476 INFO    Thread-7  :1721989 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-dummy/wandb/run-20220529_120458-qf2iwkac/files/output.log
2022-05-29 12:05:16,479 INFO    Thread-7  :1721989 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-dummy/wandb/run-20220529_120458-qf2iwkac/files/output.log
2022-05-29 12:05:16,641 DEBUG   HandlerThread:1721989 [handler.py:handle_request():141] handle_request: stop_status
2022-05-29 12:05:16,641 DEBUG   SenderThread:1721989 [sender.py:send_request():249] send_request: stop_status
2022-05-29 12:05:18,480 INFO    Thread-7  :1721989 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-dummy/wandb/run-20220529_120458-qf2iwkac/files/output.log
2022-05-29 12:05:20,481 INFO    Thread-7  :1721989 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-dummy/wandb/run-20220529_120458-qf2iwkac/files/output.log
2022-05-29 12:05:29,648 DEBUG   SenderThread:1721989 [sender.py:send():235] send: stats
2022-05-29 12:05:30,485 INFO    Thread-7  :1721989 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-dummy/wandb/run-20220529_120458-qf2iwkac/files/config.yaml
2022-05-29 12:05:31,669 DEBUG   HandlerThread:1721989 [handler.py:handle_request():141] handle_request: stop_status
2022-05-29 12:05:31,669 DEBUG   SenderThread:1721989 [sender.py:send_request():249] send_request: stop_status
2022-05-29 12:05:42,516 DEBUG   HandlerThread:1721989 [handler.py:handle_request():141] handle_request: partial_history
2022-05-29 12:05:42,573 DEBUG   HandlerThread:1721989 [handler.py:handle_request():141] handle_request: partial_history
2022-05-29 12:05:44,489 INFO    Thread-7  :1721989 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-dummy/wandb/run-20220529_120458-qf2iwkac/files/output.log
2022-05-29 12:05:46,714 DEBUG   HandlerThread:1721989 [handler.py:handle_request():141] handle_request: stop_status
2022-05-29 12:05:46,715 DEBUG   SenderThread:1721989 [sender.py:send_request():249] send_request: stop_status
2022-05-29 12:05:59,724 DEBUG   SenderThread:1721989 [sender.py:send():235] send: stats
2022-05-29 12:06:01,767 DEBUG   HandlerThread:1721989 [handler.py:handle_request():141] handle_request: stop_status
2022-05-29 12:06:01,767 DEBUG   SenderThread:1721989 [sender.py:send_request():249] send_request: stop_status
2022-05-29 12:06:03,840 DEBUG   HandlerThread:1721989 [handler.py:handle_request():141] handle_request: partial_history
2022-05-29 12:06:03,845 DEBUG   SenderThread:1721989 [sender.py:send():235] send: history
2022-05-29 12:06:03,846 DEBUG   SenderThread:1721989 [sender.py:send():235] send: summary
2022-05-29 12:06:03,847 INFO    SenderThread:1721989 [sender.py:_save_file():947] saving file wandb-summary.json with policy end
2022-05-29 12:06:03,899 DEBUG   HandlerThread:1721989 [handler.py:handle_request():141] handle_request: partial_history
2022-05-29 12:06:04,497 INFO    Thread-7  :1721989 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-dummy/wandb/run-20220529_120458-qf2iwkac/files/wandb-summary.json
2022-05-29 12:06:06,497 INFO    Thread-7  :1721989 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-dummy/wandb/run-20220529_120458-qf2iwkac/files/output.log
2022-05-29 12:06:16,816 DEBUG   HandlerThread:1721989 [handler.py:handle_request():141] handle_request: stop_status
2022-05-29 12:06:16,816 DEBUG   SenderThread:1721989 [sender.py:send_request():249] send_request: stop_status
2022-05-29 12:06:26,186 DEBUG   HandlerThread:1721989 [handler.py:handle_request():141] handle_request: partial_history
2022-05-29 12:06:26,188 DEBUG   SenderThread:1721989 [sender.py:send():235] send: history
2022-05-29 12:06:26,189 DEBUG   SenderThread:1721989 [sender.py:send():235] send: summary
2022-05-29 12:06:26,191 INFO    SenderThread:1721989 [sender.py:_save_file():947] saving file wandb-summary.json with policy end
2022-05-29 12:06:26,256 DEBUG   HandlerThread:1721989 [handler.py:handle_request():141] handle_request: partial_history
2022-05-29 12:06:26,505 INFO    Thread-7  :1721989 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-dummy/wandb/run-20220529_120458-qf2iwkac/files/wandb-summary.json
2022-05-29 12:06:26,834 DEBUG   HandlerThread:1721989 [handler.py:handle_request():141] handle_request: partial_history
2022-05-29 12:06:26,836 DEBUG   SenderThread:1721989 [sender.py:send():235] send: history
2022-05-29 12:06:26,837 DEBUG   SenderThread:1721989 [sender.py:send():235] send: summary
2022-05-29 12:06:26,838 INFO    SenderThread:1721989 [sender.py:_save_file():947] saving file wandb-summary.json with policy end
2022-05-29 12:06:26,884 DEBUG   HandlerThread:1721989 [handler.py:handle_request():141] handle_request: partial_history
2022-05-29 12:06:27,505 INFO    Thread-7  :1721989 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-dummy/wandb/run-20220529_120458-qf2iwkac/files/wandb-summary.json
2022-05-29 12:06:27,513 DEBUG   HandlerThread:1721989 [handler.py:handle_request():141] handle_request: partial_history
2022-05-29 12:06:27,515 DEBUG   SenderThread:1721989 [sender.py:send():235] send: history
2022-05-29 12:06:27,515 DEBUG   SenderThread:1721989 [sender.py:send():235] send: summary
2022-05-29 12:06:27,516 INFO    SenderThread:1721989 [sender.py:_save_file():947] saving file wandb-summary.json with policy end
2022-05-29 12:06:27,570 DEBUG   HandlerThread:1721989 [handler.py:handle_request():141] handle_request: partial_history
2022-05-29 12:06:28,506 INFO    Thread-7  :1721989 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-dummy/wandb/run-20220529_120458-qf2iwkac/files/wandb-summary.json
2022-05-29 12:06:28,506 INFO    Thread-7  :1721989 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-dummy/wandb/run-20220529_120458-qf2iwkac/files/output.log
2022-05-29 12:06:29,794 DEBUG   SenderThread:1721989 [sender.py:send():235] send: stats
2022-05-29 12:06:31,871 DEBUG   HandlerThread:1721989 [handler.py:handle_request():141] handle_request: stop_status
2022-05-29 12:06:31,871 DEBUG   SenderThread:1721989 [sender.py:send_request():249] send_request: stop_status
2022-05-29 12:06:46,902 DEBUG   HandlerThread:1721989 [handler.py:handle_request():141] handle_request: stop_status
2022-05-29 12:06:46,903 DEBUG   SenderThread:1721989 [sender.py:send_request():249] send_request: stop_status
2022-05-29 12:06:54,515 INFO    Thread-7  :1721989 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-dummy/wandb/run-20220529_120458-qf2iwkac/files/output.log
2022-05-29 12:06:59,870 DEBUG   SenderThread:1721989 [sender.py:send():235] send: stats
2022-05-29 12:07:01,952 DEBUG   HandlerThread:1721989 [handler.py:handle_request():141] handle_request: stop_status
2022-05-29 12:07:01,952 DEBUG   SenderThread:1721989 [sender.py:send_request():249] send_request: stop_status
2022-05-29 12:07:12,521 INFO    Thread-7  :1721989 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-dummy/wandb/run-20220529_120458-qf2iwkac/files/output.log
2022-05-29 12:07:17,016 DEBUG   HandlerThread:1721989 [handler.py:handle_request():141] handle_request: stop_status
2022-05-29 12:07:17,016 DEBUG   SenderThread:1721989 [sender.py:send_request():249] send_request: stop_status
2022-05-29 12:07:29,946 DEBUG   SenderThread:1721989 [sender.py:send():235] send: stats
2022-05-29 12:07:32,044 DEBUG   HandlerThread:1721989 [handler.py:handle_request():141] handle_request: stop_status
2022-05-29 12:07:32,044 DEBUG   SenderThread:1721989 [sender.py:send_request():249] send_request: stop_status
2022-05-29 12:07:35,982 DEBUG   HandlerThread:1721989 [handler.py:handle_request():141] handle_request: partial_history
2022-05-29 12:07:36,081 DEBUG   HandlerThread:1721989 [handler.py:handle_request():141] handle_request: log_artifact
2022-05-29 12:07:36,081 DEBUG   SenderThread:1721989 [sender.py:send_request():249] send_request: log_artifact
2022-05-29 12:07:36,280 DEBUG   HandlerThread:1721989 [handler.py:handle_request():141] handle_request: partial_history
2022-05-29 12:07:36,354 DEBUG   HandlerThread:1721989 [handler.py:handle_request():141] handle_request: log_artifact
2022-05-29 12:07:36,453 DEBUG   HandlerThread:1721989 [handler.py:handle_request():141] handle_request: partial_history
2022-05-29 12:07:36,530 INFO    Thread-7  :1721989 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-dummy/wandb/run-20220529_120458-qf2iwkac/files/output.log
2022-05-29 12:07:36,530 INFO    Thread-7  :1721989 [dir_watcher.py:_on_file_created():217] file/dir created: /home/sanchitgandhi/flax-dummy/wandb/run-20220529_120458-qf2iwkac/files/media/table/eval/step_0k_5_92ae240f2472ae79ad31.table.json
2022-05-29 12:07:36,530 INFO    Thread-7  :1721989 [dir_watcher.py:_on_file_created():217] file/dir created: /home/sanchitgandhi/flax-dummy/wandb/run-20220529_120458-qf2iwkac/files/media/table/eval/step_0k_incorrect_5_b30e6671cbaaf5c82b6d.table.json
2022-05-29 12:07:36,530 INFO    Thread-7  :1721989 [dir_watcher.py:_on_file_created():217] file/dir created: /home/sanchitgandhi/flax-dummy/wandb/run-20220529_120458-qf2iwkac/files/media
2022-05-29 12:07:36,530 INFO    Thread-7  :1721989 [dir_watcher.py:_on_file_created():217] file/dir created: /home/sanchitgandhi/flax-dummy/wandb/run-20220529_120458-qf2iwkac/files/media/table
2022-05-29 12:07:36,530 INFO    Thread-7  :1721989 [dir_watcher.py:_on_file_created():217] file/dir created: /home/sanchitgandhi/flax-dummy/wandb/run-20220529_120458-qf2iwkac/files/media/table/eval
2022-05-29 12:07:36,589 INFO    Thread-13 :1721989 [upload_job.py:push():95] Uploaded file /home/sanchitgandhi/.cache/wandb/artifacts/obj/md5/00/66783abbd98b29e02682346c82293a
2022-05-29 12:07:36,995 INFO    SenderThread:1721989 [sender.py:send_request_log_artifact():976] logged artifact run-qf2iwkac-evalstep_0k - {'id': 'QXJ0aWZhY3Q6MTM1NjQyNTgy', 'digest': '1fc9411295ac28c0df16aa1f68611033', 'state': 'PENDING', 'aliases': [], 'artifactSequence': {'id': 'QXJ0aWZhY3RDb2xsZWN0aW9uOjIxNTAwMTY3', 'latestArtifact': None}, 'version': 'latest'}
2022-05-29 12:07:36,995 DEBUG   SenderThread:1721989 [sender.py:send():235] send: files
2022-05-29 12:07:36,995 INFO    SenderThread:1721989 [sender.py:_save_file():947] saving file media/table/eval/step_0k_5_92ae240f2472ae79ad31.table.json with policy now
2022-05-29 12:07:36,996 DEBUG   SenderThread:1721989 [sender.py:send_request():249] send_request: log_artifact
2022-05-29 12:07:37,260 INFO    Thread-14 :1721989 [upload_job.py:push():137] Uploaded file /tmp/tmpl7u6xxw8wandb/1pl7w3az-media/table/eval/step_0k_5_92ae240f2472ae79ad31.table.json
2022-05-29 12:07:37,399 INFO    Thread-16 :1721989 [upload_job.py:push():95] Uploaded file /home/sanchitgandhi/.cache/wandb/artifacts/obj/md5/61/78965daa44518dd50bc0e765d67f24
2022-05-29 12:07:37,903 INFO    SenderThread:1721989 [sender.py:send_request_log_artifact():976] logged artifact run-qf2iwkac-evalstep_0k_incorrect - {'id': 'QXJ0aWZhY3Q6MTM1NjQyNTkx', 'digest': 'cd720fb526739060a418c512f0e14c5f', 'state': 'PENDING', 'aliases': [], 'artifactSequence': {'id': 'QXJ0aWZhY3RDb2xsZWN0aW9uOjIxNTAwMTY4', 'latestArtifact': None}, 'version': 'latest'}
2022-05-29 12:07:37,903 DEBUG   SenderThread:1721989 [sender.py:send():235] send: files
2022-05-29 12:07:37,903 INFO    SenderThread:1721989 [sender.py:_save_file():947] saving file media/table/eval/step_0k_incorrect_5_b30e6671cbaaf5c82b6d.table.json with policy now
2022-05-29 12:07:38,134 INFO    Thread-17 :1721989 [upload_job.py:push():137] Uploaded file /tmp/tmpl7u6xxw8wandb/26g70uwz-media/table/eval/step_0k_incorrect_5_b30e6671cbaaf5c82b6d.table.json
2022-05-29 12:07:38,530 INFO    Thread-7  :1721989 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-dummy/wandb/run-20220529_120458-qf2iwkac/files/output.log
2022-05-29 12:07:40,346 DEBUG   HandlerThread:1721989 [handler.py:handle_request():141] handle_request: partial_history
2022-05-29 12:07:40,349 DEBUG   SenderThread:1721989 [sender.py:send():235] send: history
2022-05-29 12:07:40,349 DEBUG   SenderThread:1721989 [sender.py:send():235] send: summary
2022-05-29 12:07:40,350 INFO    SenderThread:1721989 [sender.py:_save_file():947] saving file wandb-summary.json with policy end
2022-05-29 12:07:40,392 DEBUG   HandlerThread:1721989 [handler.py:handle_request():141] handle_request: partial_history
2022-05-29 12:07:40,531 INFO    Thread-7  :1721989 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-dummy/wandb/run-20220529_120458-qf2iwkac/files/wandb-summary.json
2022-05-29 12:07:40,531 INFO    Thread-7  :1721989 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-dummy/wandb/run-20220529_120458-qf2iwkac/files/output.log
2022-05-29 12:07:40,896 DEBUG   HandlerThread:1721989 [handler.py:handle_request():141] handle_request: partial_history
2022-05-29 12:07:40,899 DEBUG   SenderThread:1721989 [sender.py:send():235] send: history
2022-05-29 12:07:40,899 DEBUG   SenderThread:1721989 [sender.py:send():235] send: summary
2022-05-29 12:07:40,900 INFO    SenderThread:1721989 [sender.py:_save_file():947] saving file wandb-summary.json with policy end
2022-05-29 12:07:40,946 DEBUG   HandlerThread:1721989 [handler.py:handle_request():141] handle_request: partial_history
2022-05-29 12:07:41,532 INFO    Thread-7  :1721989 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-dummy/wandb/run-20220529_120458-qf2iwkac/files/wandb-summary.json
2022-05-29 12:07:41,543 DEBUG   HandlerThread:1721989 [handler.py:handle_request():141] handle_request: partial_history
2022-05-29 12:07:41,545 DEBUG   SenderThread:1721989 [sender.py:send():235] send: history
2022-05-29 12:07:41,546 DEBUG   SenderThread:1721989 [sender.py:send():235] send: summary
2022-05-29 12:07:41,547 INFO    SenderThread:1721989 [sender.py:_save_file():947] saving file wandb-summary.json with policy end
2022-05-29 12:07:41,579 DEBUG   HandlerThread:1721989 [handler.py:handle_request():141] handle_request: partial_history
2022-05-29 12:07:42,158 DEBUG   HandlerThread:1721989 [handler.py:handle_request():141] handle_request: partial_history
2022-05-29 12:07:42,160 DEBUG   SenderThread:1721989 [sender.py:send():235] send: history
2022-05-29 12:07:42,161 DEBUG   SenderThread:1721989 [sender.py:send():235] send: summary
2022-05-29 12:07:42,162 INFO    SenderThread:1721989 [sender.py:_save_file():947] saving file wandb-summary.json with policy end
2022-05-29 12:07:42,200 DEBUG   HandlerThread:1721989 [handler.py:handle_request():141] handle_request: partial_history
2022-05-29 12:07:42,532 INFO    Thread-7  :1721989 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-dummy/wandb/run-20220529_120458-qf2iwkac/files/wandb-summary.json
2022-05-29 12:07:42,532 INFO    Thread-7  :1721989 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/sanchitgandhi/flax-dummy/wandb/run-20220529_120458-qf2iwkac/files/output.log
2022-05-29 12:07:42,826 DEBUG   HandlerThread:1721989 [handler.py:handle_request():141] handle_request: partial_history
2022-05-29 12:07:42,828 DEBUG   SenderThread:1721989 [sender.py:send():235] send: history
2022-05-29 12:07:42,828 DEBUG   SenderThread:1721989 [sender.py:send():235] send: summary
2022-05-29 12:07:42,829 INFO    SenderThread:1721989 [sender.py:_save_file():947] saving file wandb-summary.json with policy end
2022-05-29 12:07:42,857 DEBUG   HandlerThread:1721989 [handler.py:handle_request():141] handle_request: partial_history