2024-06-03 11:19:47,908 INFO StreamThr :112 [internal.py:wandb_internal():85] W&B internal server running at pid: 112, started at: 2024-06-03 11:19:47.907214 2024-06-03 11:19:47,912 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status 2024-06-03 11:19:48,334 INFO WriterThread:112 [datastore.py:open_for_write():87] open: /kaggle/working/wandb/run-20240603_111947-zd4tutif/run-zd4tutif.wandb 2024-06-03 11:19:48,335 DEBUG SenderThread:112 [sender.py:send():378] send: header 2024-06-03 11:19:48,338 DEBUG SenderThread:112 [sender.py:send():378] send: run 2024-06-03 11:19:48,534 INFO SenderThread:112 [dir_watcher.py:__init__():211] watching files in: /kaggle/working/wandb/run-20240603_111947-zd4tutif/files 2024-06-03 11:19:48,534 INFO SenderThread:112 [sender.py:_start_run_threads():1123] run started: zd4tutif with start time 1717413587.9083 2024-06-03 11:19:48,545 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: check_version 2024-06-03 11:19:48,545 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: check_version 2024-06-03 11:19:48,618 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: run_start 2024-06-03 11:19:48,630 DEBUG HandlerThread:112 [system_info.py:__init__():26] System info init 2024-06-03 11:19:48,630 DEBUG HandlerThread:112 [system_info.py:__init__():41] System info init done 2024-06-03 11:19:48,630 INFO HandlerThread:112 [system_monitor.py:start():194] Starting system monitor 2024-06-03 11:19:48,630 INFO SystemMonitor:112 [system_monitor.py:_start():158] Starting system asset monitoring threads 2024-06-03 11:19:48,631 INFO HandlerThread:112 [system_monitor.py:probe():214] Collecting system info 2024-06-03 11:19:48,631 INFO SystemMonitor:112 [interfaces.py:start():188] Started cpu monitoring 2024-06-03 11:19:48,632 INFO SystemMonitor:112 [interfaces.py:start():188] Started disk monitoring 2024-06-03 11:19:48,634 INFO SystemMonitor:112 [interfaces.py:start():188] Started gpu monitoring 2024-06-03 11:19:48,635 INFO SystemMonitor:112 [interfaces.py:start():188] Started memory monitoring 2024-06-03 11:19:48,635 INFO SystemMonitor:112 [interfaces.py:start():188] Started network monitoring 2024-06-03 11:19:48,640 DEBUG HandlerThread:112 [system_info.py:probe():150] Probing system 2024-06-03 11:19:48,644 DEBUG HandlerThread:112 [gitlib.py:_init_repo():56] git repository is invalid 2024-06-03 11:19:48,645 DEBUG HandlerThread:112 [system_info.py:probe():198] Probing system done 2024-06-03 11:19:48,645 DEBUG HandlerThread:112 [system_monitor.py:probe():223] {'os': 'Linux-5.15.133+-x86_64-with-glibc2.31', 'python': '3.10.13', 'heartbeatAt': '2024-06-03T11:19:48.640898', 'startedAt': '2024-06-03T11:19:47.900500', 'docker': None, 'cuda': None, 'args': (), 'state': 'running', 'program': 'kaggle.ipynb', 'codePathLocal': None, 'root': '/kaggle/working', 'host': '743112a2decd', 'username': 'root', 'executable': '/opt/conda/bin/python3.10', 'cpu_count': 2, 'cpu_count_logical': 4, 'cpu_freq': {'current': 2000.152, 'min': 0.0, 'max': 0.0}, 'cpu_freq_per_core': [{'current': 2000.152, 'min': 0.0, 'max': 0.0}, {'current': 2000.152, 'min': 0.0, 'max': 0.0}, {'current': 2000.152, 'min': 0.0, 'max': 0.0}, {'current': 2000.152, 'min': 0.0, 'max': 0.0}], 'disk': {'/': {'total': 8062.387607574463, 'used': 5644.639331817627}}, 'gpu': 'Tesla P100-PCIE-16GB', 'gpu_count': 1, 'gpu_devices': [{'name': 'Tesla P100-PCIE-16GB', 'memory_total': 17179869184}], 'memory': {'total': 31.357563018798828}} 2024-06-03 11:19:48,645 INFO HandlerThread:112 [system_monitor.py:probe():224] Finished collecting system info 2024-06-03 11:19:48,645 INFO HandlerThread:112 [system_monitor.py:probe():227] Publishing system info 2024-06-03 11:19:48,645 DEBUG HandlerThread:112 [system_info.py:_save_conda():207] Saving list of conda packages installed into the current environment 2024-06-03 11:19:49,536 INFO Thread-12 :112 [dir_watcher.py:_on_file_created():271] file/dir created: /kaggle/working/wandb/run-20240603_111947-zd4tutif/files/conda-environment.yaml 2024-06-03 11:20:03,660 ERROR HandlerThread:112 [system_info.py:_save_conda():221] Error saving conda packages: Command '['conda', 'env', 'export']' timed out after 15 seconds Traceback (most recent call last): File "/opt/conda/lib/python3.10/site-packages/wandb/sdk/internal/system/system_info.py", line 214, in _save_conda subprocess.call( File "/opt/conda/lib/python3.10/subprocess.py", line 347, in call return p.wait(timeout=timeout) File "/opt/conda/lib/python3.10/subprocess.py", line 1209, in wait return self._wait(timeout=timeout) File "/opt/conda/lib/python3.10/subprocess.py", line 1951, in _wait raise TimeoutExpired(self.args, timeout) subprocess.TimeoutExpired: Command '['conda', 'env', 'export']' timed out after 15 seconds 2024-06-03 11:20:03,663 DEBUG HandlerThread:112 [system_info.py:_save_conda():222] Saving conda packages done 2024-06-03 11:20:03,663 INFO HandlerThread:112 [system_monitor.py:probe():229] Finished publishing system info 2024-06-03 11:20:03,671 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:20:03,671 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: keepalive 2024-06-03 11:20:03,671 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:20:03,671 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: keepalive 2024-06-03 11:20:03,671 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:20:03,671 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: keepalive 2024-06-03 11:20:03,672 DEBUG SenderThread:112 [sender.py:send():378] send: files 2024-06-03 11:20:03,672 INFO SenderThread:112 [sender.py:_save_file():1389] saving file wandb-metadata.json with policy now 2024-06-03 11:20:03,993 INFO wandb-upload_0:112 [upload_job.py:push():130] Uploaded file /tmp/tmp5ozwvysywandb/1kpsrpwb-wandb-metadata.json 2024-06-03 11:20:04,539 INFO Thread-12 :112 [dir_watcher.py:_on_file_created():271] file/dir created: /kaggle/working/wandb/run-20240603_111947-zd4tutif/files/wandb-metadata.json 2024-06-03 11:20:04,876 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: python_packages 2024-06-03 11:20:04,877 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: python_packages 2024-06-03 11:20:04,879 DEBUG SenderThread:112 [sender.py:send():378] send: telemetry 2024-06-03 11:20:04,880 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:20:04,880 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:20:05,037 DEBUG SenderThread:112 [sender.py:send():378] send: config 2024-06-03 11:20:05,038 DEBUG SenderThread:112 [sender.py:send():378] send: metric 2024-06-03 11:20:05,039 DEBUG SenderThread:112 [sender.py:send():378] send: telemetry 2024-06-03 11:20:05,039 DEBUG SenderThread:112 [sender.py:send():378] send: telemetry 2024-06-03 11:20:05,039 DEBUG SenderThread:112 [sender.py:send():378] send: metric 2024-06-03 11:20:05,039 WARNING SenderThread:112 [sender.py:send_metric():1340] Seen metric with glob (shouldn't happen) 2024-06-03 11:20:05,039 DEBUG SenderThread:112 [sender.py:send():378] send: telemetry 2024-06-03 11:20:05,540 INFO Thread-12 :112 [dir_watcher.py:_on_file_created():271] file/dir created: /kaggle/working/wandb/run-20240603_111947-zd4tutif/files/requirements.txt 2024-06-03 11:20:09,040 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:20:14,041 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:20:19,054 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:20:19,545 INFO Thread-12 :112 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240603_111947-zd4tutif/files/config.yaml 2024-06-03 11:20:19,880 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:20:19,881 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:20:24,983 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:20:29,984 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:20:34,914 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:20:34,915 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:20:35,066 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:20:40,067 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:20:45,068 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:20:48,635 DEBUG SystemMonitor:112 [system_monitor.py:_start():172] Starting system metrics aggregation loop 2024-06-03 11:20:48,637 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:20:49,914 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:20:49,915 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:20:51,064 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:20:56,065 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:21:01,067 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:21:04,999 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:21:05,000 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:21:06,109 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:21:11,110 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:21:16,112 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:21:18,637 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:21:19,999 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:21:19,999 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:21:22,085 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:21:27,086 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:21:32,087 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:21:35,083 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:21:35,084 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:21:37,206 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:21:42,207 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:21:47,208 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:21:48,638 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:21:50,088 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:21:50,089 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:21:52,229 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:21:57,230 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:22:02,231 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:22:05,170 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:22:05,171 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:22:07,347 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:22:12,349 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:22:17,349 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:22:18,639 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:22:20,170 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:22:20,171 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:22:23,316 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:22:28,317 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:22:33,318 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:22:35,266 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:22:35,267 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:22:38,335 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:22:43,336 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:22:48,337 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:22:48,640 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:22:50,266 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:22:50,267 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:22:53,401 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:22:58,402 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:22:58,943 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: partial_history 2024-06-03 11:22:58,945 DEBUG SenderThread:112 [sender.py:send():378] send: metric 2024-06-03 11:22:58,945 DEBUG SenderThread:112 [sender.py:send():378] send: metric 2024-06-03 11:22:58,945 DEBUG SenderThread:112 [sender.py:send():378] send: metric 2024-06-03 11:22:58,945 DEBUG SenderThread:112 [sender.py:send():378] send: metric 2024-06-03 11:22:58,945 DEBUG SenderThread:112 [sender.py:send():378] send: history 2024-06-03 11:22:58,946 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: summary_record 2024-06-03 11:22:58,948 INFO SenderThread:112 [sender.py:_save_file():1389] saving file wandb-summary.json with policy end 2024-06-03 11:22:59,605 INFO Thread-12 :112 [dir_watcher.py:_on_file_created():271] file/dir created: /kaggle/working/wandb/run-20240603_111947-zd4tutif/files/wandb-summary.json 2024-06-03 11:23:03,949 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:23:05,338 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:23:05,339 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:23:09,486 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:23:14,487 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:23:18,640 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:23:19,641 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:23:20,337 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:23:20,338 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:23:25,518 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:23:26,656 INFO Thread-12 :112 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240603_111947-zd4tutif/files/config.yaml 2024-06-03 11:23:30,657 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:23:35,413 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:23:35,413 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:23:36,488 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:23:41,489 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:23:46,490 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:23:48,641 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:23:50,412 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:23:50,413 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:23:51,536 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:23:56,537 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:24:01,538 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:24:05,519 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:24:05,520 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:24:06,594 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:24:11,595 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:24:16,596 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:24:18,642 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:24:20,520 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:24:20,520 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:24:21,679 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:24:26,680 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:24:31,680 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:24:35,577 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:24:35,578 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:24:36,689 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:24:41,689 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:24:46,690 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:24:48,643 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:24:50,578 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:24:50,578 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:24:51,776 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:24:56,777 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:25:01,778 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:25:05,578 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:25:05,579 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:25:07,665 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:25:12,666 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:25:17,667 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:25:18,644 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:25:20,584 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:25:20,584 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:25:22,746 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:25:27,747 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:25:32,748 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:25:35,583 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:25:35,584 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:25:38,652 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:25:43,653 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:25:48,644 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:25:49,645 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:25:50,657 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:25:50,658 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:25:51,881 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: partial_history 2024-06-03 11:25:51,883 DEBUG SenderThread:112 [sender.py:send():378] send: history 2024-06-03 11:25:51,883 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: summary_record 2024-06-03 11:25:51,883 INFO SenderThread:112 [sender.py:_save_file():1389] saving file wandb-summary.json with policy end 2024-06-03 11:25:52,712 INFO Thread-12 :112 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240603_111947-zd4tutif/files/wandb-summary.json 2024-06-03 11:25:54,884 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:25:59,885 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:26:04,886 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:26:05,658 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:26:05,658 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:26:10,727 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:26:15,728 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:26:18,646 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:26:20,745 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:26:20,746 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:26:20,905 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:26:25,906 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:26:30,908 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:26:35,745 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:26:35,746 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:26:35,912 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:26:40,913 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:26:45,914 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:26:48,647 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:26:50,746 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:26:50,746 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:26:51,889 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:26:56,890 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:27:01,891 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:27:05,748 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:27:05,749 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:27:06,892 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:27:11,893 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:27:16,894 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:27:18,647 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:27:20,748 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:27:20,748 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:27:22,846 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:27:27,848 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:27:32,849 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:27:35,849 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:27:35,850 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:27:37,933 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:27:42,934 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:27:47,935 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:27:48,649 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:27:50,848 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:27:50,849 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:27:53,028 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:27:58,029 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:28:03,030 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:28:05,957 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:28:05,958 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:28:08,109 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:28:13,110 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:28:18,111 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:28:18,649 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:28:20,956 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:28:20,957 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:28:24,090 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:28:29,091 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:28:34,092 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:28:36,041 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:28:36,042 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:28:39,127 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:28:44,128 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:28:44,865 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: partial_history 2024-06-03 11:28:44,866 DEBUG SenderThread:112 [sender.py:send():378] send: history 2024-06-03 11:28:44,866 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: summary_record 2024-06-03 11:28:44,866 INFO SenderThread:112 [sender.py:_save_file():1389] saving file wandb-summary.json with policy end 2024-06-03 11:28:45,778 INFO Thread-12 :112 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240603_111947-zd4tutif/files/wandb-summary.json 2024-06-03 11:28:48,650 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:28:49,651 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:28:51,041 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:28:51,042 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:28:55,211 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:29:00,212 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:29:05,213 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:29:06,126 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:29:06,127 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:29:11,200 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:29:16,201 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:29:18,651 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:29:21,126 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:29:21,126 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:29:21,307 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:29:26,308 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:29:31,309 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:29:36,128 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:29:36,129 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:29:37,221 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:29:42,222 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:29:47,223 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:29:48,652 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:29:51,147 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:29:51,147 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:29:52,266 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:29:57,267 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:30:02,268 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:30:06,147 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:30:06,148 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:30:08,245 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:30:13,246 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:30:18,247 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:30:18,653 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:30:21,238 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:30:21,239 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:30:23,381 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:30:28,382 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:30:33,383 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:30:36,238 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:30:36,239 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:30:38,386 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:30:43,387 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:30:48,388 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:30:48,654 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:30:51,318 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:30:51,318 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:30:53,399 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:30:58,399 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:31:03,401 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:31:06,318 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:31:06,319 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:31:08,454 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:31:13,455 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:31:18,456 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:31:18,655 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:31:21,319 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:31:21,319 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:31:23,465 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:31:28,466 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:31:33,467 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:31:36,318 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:31:36,319 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:31:37,879 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: partial_history 2024-06-03 11:31:37,880 DEBUG SenderThread:112 [sender.py:send():378] send: history 2024-06-03 11:31:37,880 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: summary_record 2024-06-03 11:31:37,881 INFO SenderThread:112 [sender.py:_save_file():1389] saving file wandb-summary.json with policy end 2024-06-03 11:31:38,843 INFO Thread-12 :112 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240603_111947-zd4tutif/files/wandb-summary.json 2024-06-03 11:31:38,881 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:31:43,883 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:31:48,656 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:31:49,656 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:31:51,318 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:31:51,319 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:31:55,458 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:32:00,459 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:32:05,460 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:32:06,404 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:32:06,405 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:32:10,491 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:32:15,492 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:32:18,656 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:32:20,657 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:32:21,404 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:32:21,405 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:32:26,579 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:32:31,580 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:32:36,503 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:32:36,504 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:32:36,676 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:32:41,677 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:32:46,679 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:32:48,657 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:32:51,503 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:32:51,504 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:32:52,583 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:32:57,584 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:33:02,585 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:33:06,504 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:33:06,504 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:33:07,637 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:33:12,638 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:33:17,639 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:33:18,658 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:33:21,517 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:33:21,518 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:33:22,680 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:33:27,681 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:33:32,682 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:33:36,517 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:33:36,517 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:33:38,657 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:33:43,658 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:33:46,342 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: partial_history 2024-06-03 11:33:46,344 DEBUG SenderThread:112 [sender.py:send():378] send: metric 2024-06-03 11:33:46,344 DEBUG SenderThread:112 [sender.py:send():378] send: metric 2024-06-03 11:33:46,345 DEBUG SenderThread:112 [sender.py:send():378] send: metric 2024-06-03 11:33:46,345 DEBUG SenderThread:112 [sender.py:send():378] send: metric 2024-06-03 11:33:46,345 DEBUG SenderThread:112 [sender.py:send():378] send: history 2024-06-03 11:33:46,345 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: summary_record 2024-06-03 11:33:46,346 INFO SenderThread:112 [sender.py:_save_file():1389] saving file wandb-summary.json with policy end 2024-06-03 11:33:46,893 INFO Thread-12 :112 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240603_111947-zd4tutif/files/wandb-summary.json 2024-06-03 11:33:48,659 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:33:48,660 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:33:51,645 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:33:51,645 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:33:53,814 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:33:58,815 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:34:03,816 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:34:06,613 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:34:06,614 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:34:09,727 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:34:14,736 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:34:14,908 INFO Thread-12 :112 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240603_111947-zd4tutif/files/config.yaml 2024-06-03 11:34:18,660 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:34:20,661 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:34:21,613 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:34:21,614 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:34:25,763 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:34:30,764 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:34:35,765 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:34:36,613 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:34:36,613 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:34:40,774 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:34:45,775 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:34:48,661 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:34:50,394 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: partial_history 2024-06-03 11:34:50,395 DEBUG SenderThread:112 [sender.py:send():378] send: history 2024-06-03 11:34:50,395 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: summary_record 2024-06-03 11:34:50,395 INFO SenderThread:112 [sender.py:_save_file():1389] saving file wandb-summary.json with policy end 2024-06-03 11:34:50,922 INFO Thread-12 :112 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240603_111947-zd4tutif/files/wandb-summary.json 2024-06-03 11:34:51,396 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:34:51,615 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:34:51,616 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:34:56,740 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:35:01,741 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:35:06,614 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:35:06,615 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:35:07,695 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:35:12,696 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:35:17,697 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:35:18,662 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:35:21,614 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:35:21,615 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:35:22,762 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:35:27,763 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:35:32,764 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:35:36,614 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:35:36,615 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:35:37,771 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:35:42,772 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:35:47,773 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:35:48,663 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:35:51,614 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:35:51,615 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:35:52,782 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:35:57,783 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:36:02,784 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:36:06,614 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:36:06,615 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:36:08,740 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:36:13,741 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:36:18,664 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:36:19,665 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:36:21,616 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:36:21,617 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:36:24,769 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:36:29,770 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:36:34,771 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:36:36,681 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:36:36,681 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:36:39,797 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:36:44,798 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:36:48,665 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:36:50,666 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:36:51,679 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:36:51,680 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:36:55,759 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:37:00,761 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:37:05,762 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:37:06,680 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:37:06,681 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:37:10,763 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:37:15,764 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:37:18,666 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:37:21,667 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:37:21,680 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:37:21,680 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:37:26,441 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: partial_history 2024-06-03 11:37:26,442 DEBUG SenderThread:112 [sender.py:send():378] send: history 2024-06-03 11:37:26,443 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: summary_record 2024-06-03 11:37:26,443 INFO SenderThread:112 [sender.py:_save_file():1389] saving file wandb-summary.json with policy end 2024-06-03 11:37:26,982 INFO Thread-12 :112 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240603_111947-zd4tutif/files/wandb-summary.json 2024-06-03 11:37:27,444 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:37:32,445 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:37:36,682 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:37:36,683 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:37:37,848 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:37:42,849 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:37:47,850 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:37:48,667 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:37:51,682 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:37:51,682 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:37:53,762 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:37:58,764 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:38:03,765 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:38:06,685 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:38:06,685 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:38:08,848 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:38:13,849 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:38:18,667 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:38:19,668 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:38:21,682 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:38:21,683 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:38:24,843 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:38:29,844 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:38:34,845 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:38:36,683 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:38:36,684 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:38:40,772 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:38:45,773 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:38:48,674 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:38:51,675 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:38:51,682 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:38:51,683 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:38:56,822 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:39:01,823 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:39:06,765 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:39:06,766 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:39:06,925 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:39:11,926 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:39:16,927 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:39:18,669 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:39:21,765 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:39:21,766 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:39:22,849 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:39:27,850 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:39:32,851 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:39:36,764 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:39:36,765 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:39:37,886 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:39:42,887 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:39:47,887 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:39:48,670 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:39:51,765 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:39:51,766 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:39:52,936 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:39:57,937 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:40:02,537 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: partial_history 2024-06-03 11:40:02,538 DEBUG SenderThread:112 [sender.py:send():378] send: history 2024-06-03 11:40:02,538 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: summary_record 2024-06-03 11:40:02,539 INFO SenderThread:112 [sender.py:_save_file():1389] saving file wandb-summary.json with policy end 2024-06-03 11:40:03,045 INFO Thread-12 :112 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240603_111947-zd4tutif/files/wandb-summary.json 2024-06-03 11:40:03,539 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:40:06,765 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:40:06,765 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:40:08,865 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:40:13,866 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:40:18,671 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:40:19,672 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:40:21,765 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:40:21,766 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:40:24,866 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:40:29,867 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:40:34,868 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:40:36,766 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:40:36,766 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:40:39,923 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:40:44,924 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:40:48,672 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:40:50,673 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:40:51,765 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:40:51,766 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:40:55,905 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:41:00,906 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:41:05,907 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:41:06,766 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:41:06,766 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:41:11,848 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:41:16,849 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:41:18,673 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:41:21,766 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:41:21,767 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:41:21,928 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:41:26,929 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:41:31,930 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:41:36,814 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:41:36,815 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:41:37,904 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:41:42,906 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:41:47,907 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:41:48,674 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:41:51,814 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:41:51,815 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:41:52,988 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:41:57,989 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:42:02,990 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:42:06,814 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:42:06,815 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:42:08,963 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:42:13,964 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:42:18,675 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:42:19,675 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:42:21,814 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:42:21,815 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:42:24,937 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:42:29,938 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:42:34,939 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:42:36,818 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:42:36,819 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:42:38,420 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: partial_history 2024-06-03 11:42:38,421 DEBUG SenderThread:112 [sender.py:send():378] send: history 2024-06-03 11:42:38,421 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: summary_record 2024-06-03 11:42:38,421 INFO SenderThread:112 [sender.py:_save_file():1389] saving file wandb-summary.json with policy end 2024-06-03 11:42:39,109 INFO Thread-12 :112 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240603_111947-zd4tutif/files/wandb-summary.json 2024-06-03 11:42:40,422 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:42:45,423 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:42:48,675 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:42:50,676 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:42:51,820 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:42:51,820 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:42:55,908 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:43:00,909 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:43:05,910 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:43:06,890 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:43:06,891 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:43:11,033 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:43:16,034 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:43:18,676 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:43:21,677 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:43:21,889 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:43:21,890 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:43:27,067 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:43:32,069 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:43:36,889 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:43:36,890 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:43:37,958 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:43:42,960 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:43:47,961 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:43:48,678 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:43:51,889 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:43:51,890 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:43:53,041 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:43:58,042 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:44:03,043 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:44:06,890 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:44:06,890 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:44:08,978 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:44:13,979 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:44:18,678 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:44:19,678 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:44:21,897 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:44:21,898 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:44:25,022 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:44:30,023 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:44:35,024 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:44:36,890 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:44:36,891 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:44:40,038 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:44:45,039 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:44:48,679 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:44:50,680 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:44:51,890 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:44:51,891 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:44:56,067 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:45:01,069 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:45:06,069 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:45:06,928 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:45:06,929 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:45:12,054 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:45:14,277 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: partial_history 2024-06-03 11:45:14,279 DEBUG SenderThread:112 [sender.py:send():378] send: history 2024-06-03 11:45:14,279 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: summary_record 2024-06-03 11:45:14,280 INFO SenderThread:112 [sender.py:_save_file():1389] saving file wandb-summary.json with policy end 2024-06-03 11:45:15,173 INFO Thread-12 :112 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240603_111947-zd4tutif/files/wandb-summary.json 2024-06-03 11:45:17,280 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:45:18,679 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:45:21,928 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:45:21,929 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:45:23,049 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:45:28,050 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:45:33,051 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:45:36,928 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:45:36,929 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:45:38,065 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:45:43,067 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:45:48,068 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:45:48,680 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:45:51,932 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:45:51,932 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:45:54,001 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:45:59,002 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:46:04,003 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:46:06,932 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:46:06,932 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:46:09,101 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:46:14,102 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:46:15,856 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: partial_history 2024-06-03 11:46:15,857 DEBUG SenderThread:112 [sender.py:send():378] send: history 2024-06-03 11:46:15,857 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: summary_record 2024-06-03 11:46:15,857 INFO SenderThread:112 [sender.py:_save_file():1389] saving file wandb-summary.json with policy end 2024-06-03 11:46:16,197 INFO Thread-12 :112 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240603_111947-zd4tutif/files/wandb-summary.json 2024-06-03 11:46:18,681 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:46:19,682 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:46:23,361 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:46:23,361 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:46:25,428 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:46:30,429 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:46:35,430 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:46:37,208 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:46:37,209 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:46:41,296 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:46:46,298 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:46:48,682 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:46:51,683 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:46:52,213 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:46:52,214 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:46:57,353 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:47:02,355 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:47:07,210 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:47:07,210 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:47:07,382 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:47:12,383 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:47:17,384 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:47:18,683 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:47:22,212 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:47:22,213 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:47:23,375 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:47:28,377 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:47:33,378 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:47:37,213 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:47:37,213 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:47:39,368 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:47:44,369 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:47:48,683 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:47:49,684 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:47:52,214 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:47:52,214 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:47:55,301 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:48:00,302 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:48:05,303 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:48:07,213 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:48:07,213 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:48:10,312 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:48:15,312 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:48:15,963 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: partial_history 2024-06-03 11:48:15,965 DEBUG SenderThread:112 [sender.py:send():378] send: history 2024-06-03 11:48:15,965 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: summary_record 2024-06-03 11:48:15,965 INFO SenderThread:112 [sender.py:_save_file():1389] saving file wandb-summary.json with policy end 2024-06-03 11:48:16,245 INFO Thread-12 :112 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240603_111947-zd4tutif/files/wandb-summary.json 2024-06-03 11:48:18,684 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:48:20,685 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:48:22,221 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:48:22,221 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:48:26,374 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:48:31,376 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:48:36,376 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:48:37,213 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:48:37,214 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:48:42,344 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:48:47,345 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:48:48,685 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:48:52,286 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:48:52,286 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:48:52,373 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:48:57,375 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:49:02,376 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:49:07,286 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:49:07,287 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:49:07,451 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:49:12,453 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:49:17,454 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:49:18,686 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:49:22,286 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:49:22,287 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:49:23,376 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:49:28,377 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:49:33,378 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:49:37,286 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:49:37,287 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:49:38,420 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:49:43,421 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:49:48,422 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:49:48,687 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:49:52,287 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:49:52,287 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:49:54,400 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:49:59,401 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:50:04,401 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:50:07,288 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:50:07,289 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:50:10,392 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:50:15,393 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:50:18,687 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:50:20,688 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:50:22,350 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:50:22,350 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:50:26,510 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:50:31,511 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:50:36,512 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:50:37,367 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:50:37,367 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:50:42,467 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:50:47,468 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:50:48,688 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:50:51,918 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: partial_history 2024-06-03 11:50:51,920 DEBUG SenderThread:112 [sender.py:send():378] send: history 2024-06-03 11:50:51,920 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: summary_record 2024-06-03 11:50:51,920 INFO SenderThread:112 [sender.py:_save_file():1389] saving file wandb-summary.json with policy end 2024-06-03 11:50:52,310 INFO Thread-12 :112 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240603_111947-zd4tutif/files/wandb-summary.json 2024-06-03 11:50:52,367 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:50:52,367 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:50:53,456 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:50:58,457 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:51:03,458 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:51:07,367 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:51:07,367 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:51:08,476 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:51:13,477 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:51:18,478 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:51:18,689 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:51:22,367 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:51:22,367 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:51:24,444 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:51:29,446 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:51:34,447 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:51:37,366 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:51:37,367 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:51:39,520 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:51:44,521 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:51:48,690 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:51:49,691 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:51:52,367 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:51:52,368 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:51:55,490 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:52:00,491 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:52:05,492 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:52:07,367 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:52:07,368 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:52:11,437 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:52:16,438 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:52:18,691 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:52:21,692 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:52:22,436 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:52:22,437 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:52:27,538 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:52:32,539 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:52:37,436 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:52:37,437 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:52:37,617 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:52:42,618 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:52:47,619 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:52:48,692 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:52:52,436 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:52:52,437 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:52:53,580 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:52:58,580 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:53:03,582 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:53:07,436 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:53:07,437 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:53:08,604 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:53:13,605 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:53:18,606 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:53:18,692 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:53:22,438 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:53:22,439 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:53:24,582 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:53:27,733 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: partial_history 2024-06-03 11:53:27,734 DEBUG SenderThread:112 [sender.py:send():378] send: history 2024-06-03 11:53:27,734 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: summary_record 2024-06-03 11:53:27,734 INFO SenderThread:112 [sender.py:_save_file():1389] saving file wandb-summary.json with policy end 2024-06-03 11:53:28,370 INFO Thread-12 :112 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240603_111947-zd4tutif/files/wandb-summary.json 2024-06-03 11:53:29,735 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:53:34,736 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:53:37,493 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:53:37,494 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:53:40,605 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:53:45,606 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:53:48,694 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:53:50,694 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:53:52,492 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:53:52,493 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:53:56,648 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:54:01,649 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:54:06,650 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:54:07,493 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:54:07,494 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:54:12,602 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:54:17,603 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:54:18,695 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:54:22,493 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:54:22,494 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:54:22,609 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:54:27,610 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:54:32,611 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:54:37,494 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:54:37,494 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:54:37,644 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:54:42,645 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:54:47,646 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:54:48,696 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:54:52,493 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:54:52,494 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:54:53,571 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:54:58,572 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:55:03,573 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:55:07,498 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:55:07,499 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:55:08,651 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:55:13,652 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:55:18,653 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:55:18,696 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:55:22,496 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:55:22,497 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:55:24,640 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:55:29,641 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:55:34,642 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:55:37,560 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:55:37,560 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:55:39,695 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:55:44,696 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:55:48,697 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:55:49,698 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:55:52,559 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:55:52,560 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:55:55,674 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:56:00,676 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:56:03,636 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: partial_history 2024-06-03 11:56:03,637 DEBUG SenderThread:112 [sender.py:send():378] send: history 2024-06-03 11:56:03,638 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: summary_record 2024-06-03 11:56:03,638 INFO SenderThread:112 [sender.py:_save_file():1389] saving file wandb-summary.json with policy end 2024-06-03 11:56:04,431 INFO Thread-12 :112 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240603_111947-zd4tutif/files/wandb-summary.json 2024-06-03 11:56:06,639 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:56:07,560 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:56:07,560 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:56:11,726 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:56:16,727 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:56:18,698 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:56:22,559 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:56:22,560 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:56:22,561 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:56:27,733 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:56:32,734 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:56:37,560 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:56:37,560 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:56:38,731 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:56:43,732 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:56:48,699 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:56:49,700 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:56:52,560 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:56:52,560 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:56:54,710 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:56:59,711 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:57:04,712 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:57:07,560 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:57:07,561 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:57:09,741 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:57:14,743 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:57:18,700 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:57:20,701 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:57:22,561 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:57:22,562 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:57:26,633 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:57:31,635 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:57:36,636 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:57:37,641 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:57:37,642 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:57:41,716 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:57:46,717 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:57:48,701 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:57:52,641 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:57:52,642 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:57:52,642 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:57:57,716 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:58:02,717 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:58:07,642 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:58:07,642 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:58:07,822 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:58:12,823 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:58:17,824 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:58:18,701 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:58:22,642 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:58:22,642 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:58:23,711 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:58:28,712 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:58:33,713 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:58:37,642 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:58:37,642 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:58:38,724 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:58:43,725 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:58:45,360 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: partial_history 2024-06-03 11:58:45,361 DEBUG SenderThread:112 [sender.py:send():378] send: history 2024-06-03 11:58:45,362 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: summary_record 2024-06-03 11:58:45,362 INFO SenderThread:112 [sender.py:_save_file():1389] saving file wandb-summary.json with policy end 2024-06-03 11:58:45,494 INFO Thread-12 :112 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240603_111947-zd4tutif/files/wandb-summary.json 2024-06-03 11:58:48,702 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:58:49,703 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:58:52,904 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:58:52,904 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:58:53,883 DEBUG SenderThread:112 [sender.py:send():378] send: telemetry 2024-06-03 11:58:53,883 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: partial_history 2024-06-03 11:58:53,885 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: summary_record 2024-06-03 11:58:53,887 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: pause 2024-06-03 11:58:53,887 INFO HandlerThread:112 [handler.py:handle_request_pause():724] stopping system metrics thread 2024-06-03 11:58:53,887 INFO HandlerThread:112 [system_monitor.py:finish():203] Stopping system monitor 2024-06-03 11:58:53,888 DEBUG SystemMonitor:112 [system_monitor.py:_start():179] Finished system metrics aggregation loop 2024-06-03 11:58:53,888 DEBUG SystemMonitor:112 [system_monitor.py:_start():183] Publishing last batch of metrics 2024-06-03 11:58:53,889 INFO SenderThread:112 [sender.py:_save_file():1389] saving file wandb-summary.json with policy end 2024-06-03 11:58:53,890 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: summary_record 2024-06-03 11:58:53,893 INFO HandlerThread:112 [interfaces.py:finish():200] Joined cpu monitor 2024-06-03 11:58:53,893 INFO HandlerThread:112 [interfaces.py:finish():200] Joined disk monitor 2024-06-03 11:58:53,894 INFO SenderThread:112 [sender.py:_save_file():1389] saving file wandb-summary.json with policy end 2024-06-03 11:58:53,894 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: summary_record 2024-06-03 11:58:53,895 INFO SenderThread:112 [sender.py:_save_file():1389] saving file wandb-summary.json with policy end 2024-06-03 11:58:53,896 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: summary_record 2024-06-03 11:58:53,899 INFO SenderThread:112 [sender.py:_save_file():1389] saving file wandb-summary.json with policy end 2024-06-03 11:58:53,899 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: summary_record 2024-06-03 11:58:53,901 INFO HandlerThread:112 [interfaces.py:finish():200] Joined gpu monitor 2024-06-03 11:58:53,902 INFO HandlerThread:112 [interfaces.py:finish():200] Joined memory monitor 2024-06-03 11:58:53,902 INFO HandlerThread:112 [interfaces.py:finish():200] Joined network monitor 2024-06-03 11:58:53,902 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: resume 2024-06-03 11:58:53,902 INFO HandlerThread:112 [handler.py:handle_request_resume():715] starting system metrics thread 2024-06-03 11:58:53,902 INFO HandlerThread:112 [system_monitor.py:start():194] Starting system monitor 2024-06-03 11:58:53,903 INFO SenderThread:112 [sender.py:_save_file():1389] saving file wandb-summary.json with policy end 2024-06-03 11:58:53,903 DEBUG SenderThread:112 [sender.py:send():378] send: history 2024-06-03 11:58:53,903 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: summary_record 2024-06-03 11:58:53,904 INFO SystemMonitor:112 [system_monitor.py:_start():158] Starting system asset monitoring threads 2024-06-03 11:58:53,904 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: pause 2024-06-03 11:58:53,904 INFO SystemMonitor:112 [interfaces.py:start():188] Started cpu monitoring 2024-06-03 11:58:53,904 INFO HandlerThread:112 [handler.py:handle_request_pause():724] stopping system metrics thread 2024-06-03 11:58:53,905 INFO SenderThread:112 [sender.py:_save_file():1389] saving file wandb-summary.json with policy end 2024-06-03 11:58:53,905 INFO SystemMonitor:112 [interfaces.py:start():188] Started disk monitoring 2024-06-03 11:58:53,905 INFO HandlerThread:112 [system_monitor.py:finish():203] Stopping system monitor 2024-06-03 11:58:53,906 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:58:53,906 INFO SystemMonitor:112 [interfaces.py:start():188] Started gpu monitoring 2024-06-03 11:58:53,907 INFO HandlerThread:112 [interfaces.py:finish():200] Joined cpu monitor 2024-06-03 11:58:53,907 DEBUG SystemMonitor:112 [system_monitor.py:_start():172] Starting system metrics aggregation loop 2024-06-03 11:58:53,908 DEBUG SystemMonitor:112 [system_monitor.py:_start():179] Finished system metrics aggregation loop 2024-06-03 11:58:53,908 DEBUG SystemMonitor:112 [system_monitor.py:_start():183] Publishing last batch of metrics 2024-06-03 11:58:53,909 INFO HandlerThread:112 [interfaces.py:finish():200] Joined disk monitor 2024-06-03 11:58:53,916 INFO HandlerThread:112 [interfaces.py:finish():200] Joined gpu monitor 2024-06-03 11:58:53,916 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: resume 2024-06-03 11:58:53,916 INFO HandlerThread:112 [handler.py:handle_request_resume():715] starting system metrics thread 2024-06-03 11:58:53,916 INFO HandlerThread:112 [system_monitor.py:start():194] Starting system monitor 2024-06-03 11:58:53,916 INFO SystemMonitor:112 [system_monitor.py:_start():158] Starting system asset monitoring threads 2024-06-03 11:58:53,917 INFO SystemMonitor:112 [interfaces.py:start():188] Started cpu monitoring 2024-06-03 11:58:53,918 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:58:53,919 INFO SystemMonitor:112 [interfaces.py:start():188] Started disk monitoring 2024-06-03 11:58:53,919 INFO SystemMonitor:112 [interfaces.py:start():188] Started gpu monitoring 2024-06-03 11:58:53,920 INFO SystemMonitor:112 [interfaces.py:start():188] Started memory monitoring 2024-06-03 11:58:53,921 INFO SystemMonitor:112 [interfaces.py:start():188] Started network monitoring 2024-06-03 11:58:54,497 INFO Thread-12 :112 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240603_111947-zd4tutif/files/wandb-summary.json 2024-06-03 11:58:54,497 INFO Thread-12 :112 [dir_watcher.py:_on_file_created():271] file/dir created: /kaggle/working/wandb/run-20240603_111947-zd4tutif/files/output.log 2024-06-03 11:58:54,919 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:58:56,498 INFO Thread-12 :112 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240603_111947-zd4tutif/files/output.log 2024-06-03 11:58:59,919 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:59:04,920 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:59:05,977 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: pause 2024-06-03 11:59:05,977 INFO HandlerThread:112 [handler.py:handle_request_pause():724] stopping system metrics thread 2024-06-03 11:59:05,977 INFO HandlerThread:112 [system_monitor.py:finish():203] Stopping system monitor 2024-06-03 11:59:05,978 DEBUG SystemMonitor:112 [system_monitor.py:_start():172] Starting system metrics aggregation loop 2024-06-03 11:59:05,978 INFO HandlerThread:112 [interfaces.py:finish():200] Joined cpu monitor 2024-06-03 11:59:05,979 INFO HandlerThread:112 [interfaces.py:finish():200] Joined disk monitor 2024-06-03 11:59:05,978 DEBUG SystemMonitor:112 [system_monitor.py:_start():179] Finished system metrics aggregation loop 2024-06-03 11:59:05,980 DEBUG SystemMonitor:112 [system_monitor.py:_start():183] Publishing last batch of metrics 2024-06-03 11:59:05,988 INFO HandlerThread:112 [interfaces.py:finish():200] Joined gpu monitor 2024-06-03 11:59:05,988 INFO HandlerThread:112 [interfaces.py:finish():200] Joined memory monitor 2024-06-03 11:59:05,988 INFO HandlerThread:112 [interfaces.py:finish():200] Joined network monitor 2024-06-03 11:59:05,988 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: resume 2024-06-03 11:59:05,989 INFO HandlerThread:112 [handler.py:handle_request_resume():715] starting system metrics thread 2024-06-03 11:59:05,989 INFO HandlerThread:112 [system_monitor.py:start():194] Starting system monitor 2024-06-03 11:59:05,989 INFO SystemMonitor:112 [system_monitor.py:_start():158] Starting system asset monitoring threads 2024-06-03 11:59:05,989 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:59:05,990 INFO SystemMonitor:112 [interfaces.py:start():188] Started cpu monitoring 2024-06-03 11:59:05,990 INFO SystemMonitor:112 [interfaces.py:start():188] Started disk monitoring 2024-06-03 11:59:05,991 INFO SystemMonitor:112 [interfaces.py:start():188] Started gpu monitoring 2024-06-03 11:59:05,991 INFO SystemMonitor:112 [interfaces.py:start():188] Started memory monitoring 2024-06-03 11:59:05,992 INFO SystemMonitor:112 [interfaces.py:start():188] Started network monitoring 2024-06-03 11:59:06,531 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: pause 2024-06-03 11:59:06,531 INFO HandlerThread:112 [handler.py:handle_request_pause():724] stopping system metrics thread 2024-06-03 11:59:06,531 INFO HandlerThread:112 [system_monitor.py:finish():203] Stopping system monitor 2024-06-03 11:59:06,531 INFO HandlerThread:112 [interfaces.py:finish():200] Joined cpu monitor 2024-06-03 11:59:06,532 DEBUG SystemMonitor:112 [system_monitor.py:_start():172] Starting system metrics aggregation loop 2024-06-03 11:59:06,532 INFO HandlerThread:112 [interfaces.py:finish():200] Joined disk monitor 2024-06-03 11:59:06,533 DEBUG SystemMonitor:112 [system_monitor.py:_start():179] Finished system metrics aggregation loop 2024-06-03 11:59:06,533 DEBUG SystemMonitor:112 [system_monitor.py:_start():183] Publishing last batch of metrics 2024-06-03 11:59:06,540 INFO HandlerThread:112 [interfaces.py:finish():200] Joined gpu monitor 2024-06-03 11:59:06,540 INFO HandlerThread:112 [interfaces.py:finish():200] Joined memory monitor 2024-06-03 11:59:06,541 INFO HandlerThread:112 [interfaces.py:finish():200] Joined network monitor 2024-06-03 11:59:06,541 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: resume 2024-06-03 11:59:06,541 INFO HandlerThread:112 [handler.py:handle_request_resume():715] starting system metrics thread 2024-06-03 11:59:06,541 INFO HandlerThread:112 [system_monitor.py:start():194] Starting system monitor 2024-06-03 11:59:06,541 INFO SystemMonitor:112 [system_monitor.py:_start():158] Starting system asset monitoring threads 2024-06-03 11:59:06,542 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:59:06,543 INFO SystemMonitor:112 [interfaces.py:start():188] Started cpu monitoring 2024-06-03 11:59:06,543 INFO SystemMonitor:112 [interfaces.py:start():188] Started disk monitoring 2024-06-03 11:59:06,544 INFO SystemMonitor:112 [interfaces.py:start():188] Started gpu monitoring 2024-06-03 11:59:06,544 INFO SystemMonitor:112 [interfaces.py:start():188] Started memory monitoring 2024-06-03 11:59:06,545 INFO SystemMonitor:112 [interfaces.py:start():188] Started network monitoring 2024-06-03 11:59:06,936 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: pause 2024-06-03 11:59:06,936 INFO HandlerThread:112 [handler.py:handle_request_pause():724] stopping system metrics thread 2024-06-03 11:59:06,936 INFO HandlerThread:112 [system_monitor.py:finish():203] Stopping system monitor 2024-06-03 11:59:06,936 DEBUG SystemMonitor:112 [system_monitor.py:_start():172] Starting system metrics aggregation loop 2024-06-03 11:59:06,936 DEBUG SystemMonitor:112 [system_monitor.py:_start():179] Finished system metrics aggregation loop 2024-06-03 11:59:06,937 DEBUG SystemMonitor:112 [system_monitor.py:_start():183] Publishing last batch of metrics 2024-06-03 11:59:06,937 INFO HandlerThread:112 [interfaces.py:finish():200] Joined cpu monitor 2024-06-03 11:59:06,938 INFO HandlerThread:112 [interfaces.py:finish():200] Joined disk monitor 2024-06-03 11:59:06,945 INFO HandlerThread:112 [interfaces.py:finish():200] Joined gpu monitor 2024-06-03 11:59:06,945 INFO HandlerThread:112 [interfaces.py:finish():200] Joined memory monitor 2024-06-03 11:59:06,945 INFO HandlerThread:112 [interfaces.py:finish():200] Joined network monitor 2024-06-03 11:59:06,946 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: resume 2024-06-03 11:59:06,946 INFO HandlerThread:112 [handler.py:handle_request_resume():715] starting system metrics thread 2024-06-03 11:59:06,946 INFO HandlerThread:112 [system_monitor.py:start():194] Starting system monitor 2024-06-03 11:59:06,946 INFO SystemMonitor:112 [system_monitor.py:_start():158] Starting system asset monitoring threads 2024-06-03 11:59:06,947 INFO SystemMonitor:112 [interfaces.py:start():188] Started cpu monitoring 2024-06-03 11:59:06,948 INFO SystemMonitor:112 [interfaces.py:start():188] Started disk monitoring 2024-06-03 11:59:06,949 INFO SystemMonitor:112 [interfaces.py:start():188] Started gpu monitoring 2024-06-03 11:59:06,949 INFO SystemMonitor:112 [interfaces.py:start():188] Started memory monitoring 2024-06-03 11:59:06,950 INFO SystemMonitor:112 [interfaces.py:start():188] Started network monitoring 2024-06-03 11:59:06,951 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:59:07,130 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: pause 2024-06-03 11:59:07,130 INFO HandlerThread:112 [handler.py:handle_request_pause():724] stopping system metrics thread 2024-06-03 11:59:07,130 INFO HandlerThread:112 [system_monitor.py:finish():203] Stopping system monitor 2024-06-03 11:59:07,131 INFO HandlerThread:112 [interfaces.py:finish():200] Joined cpu monitor 2024-06-03 11:59:07,131 INFO HandlerThread:112 [interfaces.py:finish():200] Joined disk monitor 2024-06-03 11:59:07,131 DEBUG SystemMonitor:112 [system_monitor.py:_start():172] Starting system metrics aggregation loop 2024-06-03 11:59:07,131 DEBUG SystemMonitor:112 [system_monitor.py:_start():179] Finished system metrics aggregation loop 2024-06-03 11:59:07,132 DEBUG SystemMonitor:112 [system_monitor.py:_start():183] Publishing last batch of metrics 2024-06-03 11:59:07,139 INFO HandlerThread:112 [interfaces.py:finish():200] Joined gpu monitor 2024-06-03 11:59:07,140 INFO HandlerThread:112 [interfaces.py:finish():200] Joined memory monitor 2024-06-03 11:59:07,140 INFO HandlerThread:112 [interfaces.py:finish():200] Joined network monitor 2024-06-03 11:59:07,140 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: resume 2024-06-03 11:59:07,140 INFO HandlerThread:112 [handler.py:handle_request_resume():715] starting system metrics thread 2024-06-03 11:59:07,140 INFO HandlerThread:112 [system_monitor.py:start():194] Starting system monitor 2024-06-03 11:59:07,141 INFO SystemMonitor:112 [system_monitor.py:_start():158] Starting system asset monitoring threads 2024-06-03 11:59:07,141 INFO SystemMonitor:112 [interfaces.py:start():188] Started cpu monitoring 2024-06-03 11:59:07,142 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:59:07,143 INFO SystemMonitor:112 [interfaces.py:start():188] Started disk monitoring 2024-06-03 11:59:07,144 INFO SystemMonitor:112 [interfaces.py:start():188] Started gpu monitoring 2024-06-03 11:59:07,144 INFO SystemMonitor:112 [interfaces.py:start():188] Started memory monitoring 2024-06-03 11:59:07,145 INFO SystemMonitor:112 [interfaces.py:start():188] Started network monitoring 2024-06-03 11:59:07,402 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: pause 2024-06-03 11:59:07,402 INFO HandlerThread:112 [handler.py:handle_request_pause():724] stopping system metrics thread 2024-06-03 11:59:07,402 INFO HandlerThread:112 [system_monitor.py:finish():203] Stopping system monitor 2024-06-03 11:59:07,403 INFO HandlerThread:112 [interfaces.py:finish():200] Joined cpu monitor 2024-06-03 11:59:07,403 DEBUG SystemMonitor:112 [system_monitor.py:_start():172] Starting system metrics aggregation loop 2024-06-03 11:59:07,404 DEBUG SystemMonitor:112 [system_monitor.py:_start():179] Finished system metrics aggregation loop 2024-06-03 11:59:07,404 DEBUG SystemMonitor:112 [system_monitor.py:_start():183] Publishing last batch of metrics 2024-06-03 11:59:07,405 INFO HandlerThread:112 [interfaces.py:finish():200] Joined disk monitor 2024-06-03 11:59:07,414 INFO HandlerThread:112 [interfaces.py:finish():200] Joined gpu monitor 2024-06-03 11:59:07,414 INFO HandlerThread:112 [interfaces.py:finish():200] Joined memory monitor 2024-06-03 11:59:07,415 INFO HandlerThread:112 [interfaces.py:finish():200] Joined network monitor 2024-06-03 11:59:07,415 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: resume 2024-06-03 11:59:07,415 INFO HandlerThread:112 [handler.py:handle_request_resume():715] starting system metrics thread 2024-06-03 11:59:07,415 INFO HandlerThread:112 [system_monitor.py:start():194] Starting system monitor 2024-06-03 11:59:07,415 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:59:07,416 INFO SystemMonitor:112 [system_monitor.py:_start():158] Starting system asset monitoring threads 2024-06-03 11:59:07,417 INFO SystemMonitor:112 [interfaces.py:start():188] Started cpu monitoring 2024-06-03 11:59:07,418 INFO SystemMonitor:112 [interfaces.py:start():188] Started disk monitoring 2024-06-03 11:59:07,419 INFO SystemMonitor:112 [interfaces.py:start():188] Started gpu monitoring 2024-06-03 11:59:07,420 INFO SystemMonitor:112 [interfaces.py:start():188] Started memory monitoring 2024-06-03 11:59:07,421 INFO SystemMonitor:112 [interfaces.py:start():188] Started network monitoring 2024-06-03 11:59:07,694 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: pause 2024-06-03 11:59:07,694 INFO HandlerThread:112 [handler.py:handle_request_pause():724] stopping system metrics thread 2024-06-03 11:59:07,694 INFO HandlerThread:112 [system_monitor.py:finish():203] Stopping system monitor 2024-06-03 11:59:07,694 DEBUG SystemMonitor:112 [system_monitor.py:_start():172] Starting system metrics aggregation loop 2024-06-03 11:59:07,695 INFO HandlerThread:112 [interfaces.py:finish():200] Joined cpu monitor 2024-06-03 11:59:07,695 INFO HandlerThread:112 [interfaces.py:finish():200] Joined disk monitor 2024-06-03 11:59:07,695 DEBUG SystemMonitor:112 [system_monitor.py:_start():179] Finished system metrics aggregation loop 2024-06-03 11:59:07,695 DEBUG SystemMonitor:112 [system_monitor.py:_start():183] Publishing last batch of metrics 2024-06-03 11:59:07,704 INFO HandlerThread:112 [interfaces.py:finish():200] Joined gpu monitor 2024-06-03 11:59:07,704 INFO HandlerThread:112 [interfaces.py:finish():200] Joined memory monitor 2024-06-03 11:59:07,704 INFO HandlerThread:112 [interfaces.py:finish():200] Joined network monitor 2024-06-03 11:59:07,705 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:59:07,705 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: resume 2024-06-03 11:59:07,706 INFO HandlerThread:112 [handler.py:handle_request_resume():715] starting system metrics thread 2024-06-03 11:59:07,706 INFO HandlerThread:112 [system_monitor.py:start():194] Starting system monitor 2024-06-03 11:59:07,706 INFO SystemMonitor:112 [system_monitor.py:_start():158] Starting system asset monitoring threads 2024-06-03 11:59:07,707 INFO SystemMonitor:112 [interfaces.py:start():188] Started cpu monitoring 2024-06-03 11:59:07,707 INFO SystemMonitor:112 [interfaces.py:start():188] Started disk monitoring 2024-06-03 11:59:07,710 INFO SystemMonitor:112 [interfaces.py:start():188] Started gpu monitoring 2024-06-03 11:59:07,716 INFO SystemMonitor:112 [interfaces.py:start():188] Started memory monitoring 2024-06-03 11:59:07,718 INFO SystemMonitor:112 [interfaces.py:start():188] Started network monitoring 2024-06-03 11:59:07,853 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:59:07,853 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:59:08,135 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: pause 2024-06-03 11:59:08,135 INFO HandlerThread:112 [handler.py:handle_request_pause():724] stopping system metrics thread 2024-06-03 11:59:08,135 INFO HandlerThread:112 [system_monitor.py:finish():203] Stopping system monitor 2024-06-03 11:59:08,136 DEBUG SystemMonitor:112 [system_monitor.py:_start():172] Starting system metrics aggregation loop 2024-06-03 11:59:08,136 DEBUG SystemMonitor:112 [system_monitor.py:_start():179] Finished system metrics aggregation loop 2024-06-03 11:59:08,136 DEBUG SystemMonitor:112 [system_monitor.py:_start():183] Publishing last batch of metrics 2024-06-03 11:59:08,137 INFO HandlerThread:112 [interfaces.py:finish():200] Joined cpu monitor 2024-06-03 11:59:08,137 INFO HandlerThread:112 [interfaces.py:finish():200] Joined disk monitor 2024-06-03 11:59:08,145 INFO HandlerThread:112 [interfaces.py:finish():200] Joined gpu monitor 2024-06-03 11:59:08,145 INFO HandlerThread:112 [interfaces.py:finish():200] Joined memory monitor 2024-06-03 11:59:08,145 INFO HandlerThread:112 [interfaces.py:finish():200] Joined network monitor 2024-06-03 11:59:08,146 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:59:08,176 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: resume 2024-06-03 11:59:08,176 INFO HandlerThread:112 [handler.py:handle_request_resume():715] starting system metrics thread 2024-06-03 11:59:08,176 INFO HandlerThread:112 [system_monitor.py:start():194] Starting system monitor 2024-06-03 11:59:08,176 INFO SystemMonitor:112 [system_monitor.py:_start():158] Starting system asset monitoring threads 2024-06-03 11:59:08,177 INFO SystemMonitor:112 [interfaces.py:start():188] Started cpu monitoring 2024-06-03 11:59:08,177 INFO SystemMonitor:112 [interfaces.py:start():188] Started disk monitoring 2024-06-03 11:59:08,179 INFO SystemMonitor:112 [interfaces.py:start():188] Started gpu monitoring 2024-06-03 11:59:08,181 INFO SystemMonitor:112 [interfaces.py:start():188] Started memory monitoring 2024-06-03 11:59:08,184 INFO SystemMonitor:112 [interfaces.py:start():188] Started network monitoring 2024-06-03 11:59:08,503 INFO Thread-12 :112 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240603_111947-zd4tutif/files/output.log 2024-06-03 11:59:10,147 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:59:10,504 INFO Thread-12 :112 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240603_111947-zd4tutif/files/output.log 2024-06-03 11:59:11,076 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: pause 2024-06-03 11:59:11,076 INFO HandlerThread:112 [handler.py:handle_request_pause():724] stopping system metrics thread 2024-06-03 11:59:11,076 INFO HandlerThread:112 [system_monitor.py:finish():203] Stopping system monitor 2024-06-03 11:59:11,076 DEBUG SystemMonitor:112 [system_monitor.py:_start():172] Starting system metrics aggregation loop 2024-06-03 11:59:11,076 DEBUG SystemMonitor:112 [system_monitor.py:_start():179] Finished system metrics aggregation loop 2024-06-03 11:59:11,077 INFO HandlerThread:112 [interfaces.py:finish():200] Joined cpu monitor 2024-06-03 11:59:11,077 DEBUG SystemMonitor:112 [system_monitor.py:_start():183] Publishing last batch of metrics 2024-06-03 11:59:11,077 INFO HandlerThread:112 [interfaces.py:finish():200] Joined disk monitor 2024-06-03 11:59:11,085 INFO HandlerThread:112 [interfaces.py:finish():200] Joined gpu monitor 2024-06-03 11:59:11,085 INFO HandlerThread:112 [interfaces.py:finish():200] Joined memory monitor 2024-06-03 11:59:11,085 INFO HandlerThread:112 [interfaces.py:finish():200] Joined network monitor 2024-06-03 11:59:11,086 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: resume 2024-06-03 11:59:11,086 INFO HandlerThread:112 [handler.py:handle_request_resume():715] starting system metrics thread 2024-06-03 11:59:11,086 INFO HandlerThread:112 [system_monitor.py:start():194] Starting system monitor 2024-06-03 11:59:11,086 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:59:11,087 INFO SystemMonitor:112 [system_monitor.py:_start():158] Starting system asset monitoring threads 2024-06-03 11:59:11,087 INFO SystemMonitor:112 [interfaces.py:start():188] Started cpu monitoring 2024-06-03 11:59:11,087 INFO SystemMonitor:112 [interfaces.py:start():188] Started disk monitoring 2024-06-03 11:59:11,088 INFO SystemMonitor:112 [interfaces.py:start():188] Started gpu monitoring 2024-06-03 11:59:11,088 INFO SystemMonitor:112 [interfaces.py:start():188] Started memory monitoring 2024-06-03 11:59:11,089 INFO SystemMonitor:112 [interfaces.py:start():188] Started network monitoring 2024-06-03 11:59:13,486 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: pause 2024-06-03 11:59:13,486 INFO HandlerThread:112 [handler.py:handle_request_pause():724] stopping system metrics thread 2024-06-03 11:59:13,486 INFO HandlerThread:112 [system_monitor.py:finish():203] Stopping system monitor 2024-06-03 11:59:13,487 INFO HandlerThread:112 [interfaces.py:finish():200] Joined cpu monitor 2024-06-03 11:59:13,487 INFO HandlerThread:112 [interfaces.py:finish():200] Joined disk monitor 2024-06-03 11:59:13,488 DEBUG SystemMonitor:112 [system_monitor.py:_start():172] Starting system metrics aggregation loop 2024-06-03 11:59:13,488 DEBUG SystemMonitor:112 [system_monitor.py:_start():179] Finished system metrics aggregation loop 2024-06-03 11:59:13,488 DEBUG SystemMonitor:112 [system_monitor.py:_start():183] Publishing last batch of metrics 2024-06-03 11:59:13,496 INFO HandlerThread:112 [interfaces.py:finish():200] Joined gpu monitor 2024-06-03 11:59:13,496 INFO HandlerThread:112 [interfaces.py:finish():200] Joined memory monitor 2024-06-03 11:59:13,496 INFO HandlerThread:112 [interfaces.py:finish():200] Joined network monitor 2024-06-03 11:59:13,497 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 11:59:15,498 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:59:20,504 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:59:21,508 INFO Thread-12 :112 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240603_111947-zd4tutif/files/config.yaml 2024-06-03 11:59:22,852 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:59:22,852 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:59:25,967 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:59:30,969 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:59:35,970 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:59:37,852 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:59:37,853 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:59:41,921 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:59:46,922 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:59:51,924 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 11:59:52,852 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 11:59:52,853 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 11:59:56,992 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 12:00:01,993 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 12:00:06,994 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 12:00:07,852 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 12:00:07,853 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 12:00:12,017 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 12:00:17,018 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 12:00:22,020 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 12:00:22,852 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 12:00:22,853 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 12:00:27,020 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 12:00:32,021 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 12:00:37,023 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 12:00:37,853 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 12:00:37,853 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 12:00:42,944 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 12:00:47,945 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 12:00:52,853 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 12:00:52,854 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 12:00:52,980 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 12:00:57,982 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 12:01:02,983 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 12:01:07,853 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 12:01:07,854 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 12:01:08,014 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 12:01:13,015 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 12:01:18,016 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 12:01:22,853 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 12:01:22,854 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 12:01:23,928 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 12:01:28,929 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 12:01:33,930 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 12:01:37,854 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 12:01:37,854 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 12:01:38,963 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 12:01:42,197 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: resume 2024-06-03 12:01:42,197 INFO HandlerThread:112 [handler.py:handle_request_resume():715] starting system metrics thread 2024-06-03 12:01:42,197 INFO HandlerThread:112 [system_monitor.py:start():194] Starting system monitor 2024-06-03 12:01:42,197 INFO SystemMonitor:112 [system_monitor.py:_start():158] Starting system asset monitoring threads 2024-06-03 12:01:42,198 INFO SystemMonitor:112 [interfaces.py:start():188] Started cpu monitoring 2024-06-03 12:01:42,199 INFO SystemMonitor:112 [interfaces.py:start():188] Started disk monitoring 2024-06-03 12:01:42,199 INFO SystemMonitor:112 [interfaces.py:start():188] Started gpu monitoring 2024-06-03 12:01:42,201 INFO SystemMonitor:112 [interfaces.py:start():188] Started memory monitoring 2024-06-03 12:01:42,202 INFO SystemMonitor:112 [interfaces.py:start():188] Started network monitoring 2024-06-03 12:01:42,426 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: pause 2024-06-03 12:01:42,426 INFO HandlerThread:112 [handler.py:handle_request_pause():724] stopping system metrics thread 2024-06-03 12:01:42,426 INFO HandlerThread:112 [system_monitor.py:finish():203] Stopping system monitor 2024-06-03 12:01:42,426 DEBUG SystemMonitor:112 [system_monitor.py:_start():172] Starting system metrics aggregation loop 2024-06-03 12:01:42,426 INFO HandlerThread:112 [interfaces.py:finish():200] Joined cpu monitor 2024-06-03 12:01:42,427 INFO HandlerThread:112 [interfaces.py:finish():200] Joined disk monitor 2024-06-03 12:01:42,427 DEBUG SystemMonitor:112 [system_monitor.py:_start():179] Finished system metrics aggregation loop 2024-06-03 12:01:42,427 DEBUG SystemMonitor:112 [system_monitor.py:_start():183] Publishing last batch of metrics 2024-06-03 12:01:42,434 INFO HandlerThread:112 [interfaces.py:finish():200] Joined gpu monitor 2024-06-03 12:01:42,434 INFO HandlerThread:112 [interfaces.py:finish():200] Joined memory monitor 2024-06-03 12:01:42,434 INFO HandlerThread:112 [interfaces.py:finish():200] Joined network monitor 2024-06-03 12:01:42,434 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 12:01:44,435 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 12:01:49,436 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 12:01:52,854 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 12:01:52,854 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 12:01:54,944 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 12:01:59,946 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 12:02:04,947 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 12:02:07,854 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 12:02:07,855 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 12:02:09,955 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 12:02:14,956 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 12:02:19,958 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 12:02:22,854 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 12:02:22,855 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 12:02:25,004 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 12:02:30,005 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 12:02:35,006 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 12:02:37,854 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 12:02:37,855 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 12:02:40,014 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 12:02:45,015 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 12:02:50,016 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 12:02:52,854 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 12:02:52,855 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 12:02:55,960 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 12:03:00,961 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 12:03:05,963 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 12:03:07,855 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 12:03:07,855 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 12:03:10,991 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 12:03:15,992 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 12:03:20,994 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 12:03:22,855 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 12:03:22,855 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 12:03:26,971 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 12:03:31,972 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 12:03:34,286 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: resume 2024-06-03 12:03:34,286 INFO HandlerThread:112 [handler.py:handle_request_resume():715] starting system metrics thread 2024-06-03 12:03:34,286 INFO HandlerThread:112 [system_monitor.py:start():194] Starting system monitor 2024-06-03 12:03:34,286 INFO SystemMonitor:112 [system_monitor.py:_start():158] Starting system asset monitoring threads 2024-06-03 12:03:34,287 INFO SystemMonitor:112 [interfaces.py:start():188] Started cpu monitoring 2024-06-03 12:03:34,288 INFO SystemMonitor:112 [interfaces.py:start():188] Started disk monitoring 2024-06-03 12:03:34,289 INFO SystemMonitor:112 [interfaces.py:start():188] Started gpu monitoring 2024-06-03 12:03:34,290 INFO SystemMonitor:112 [interfaces.py:start():188] Started memory monitoring 2024-06-03 12:03:34,290 INFO SystemMonitor:112 [interfaces.py:start():188] Started network monitoring 2024-06-03 12:03:34,466 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: pause 2024-06-03 12:03:34,466 INFO HandlerThread:112 [handler.py:handle_request_pause():724] stopping system metrics thread 2024-06-03 12:03:34,466 INFO HandlerThread:112 [system_monitor.py:finish():203] Stopping system monitor 2024-06-03 12:03:34,467 DEBUG SystemMonitor:112 [system_monitor.py:_start():172] Starting system metrics aggregation loop 2024-06-03 12:03:34,467 INFO HandlerThread:112 [interfaces.py:finish():200] Joined cpu monitor 2024-06-03 12:03:34,467 DEBUG SystemMonitor:112 [system_monitor.py:_start():179] Finished system metrics aggregation loop 2024-06-03 12:03:34,467 INFO HandlerThread:112 [interfaces.py:finish():200] Joined disk monitor 2024-06-03 12:03:34,467 DEBUG SystemMonitor:112 [system_monitor.py:_start():183] Publishing last batch of metrics 2024-06-03 12:03:34,474 INFO HandlerThread:112 [interfaces.py:finish():200] Joined gpu monitor 2024-06-03 12:03:34,475 INFO HandlerThread:112 [interfaces.py:finish():200] Joined memory monitor 2024-06-03 12:03:34,475 INFO HandlerThread:112 [interfaces.py:finish():200] Joined network monitor 2024-06-03 12:03:34,475 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 12:03:37,476 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 12:03:37,855 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 12:03:37,856 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 12:03:38,981 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: resume 2024-06-03 12:03:38,981 INFO HandlerThread:112 [handler.py:handle_request_resume():715] starting system metrics thread 2024-06-03 12:03:38,981 INFO HandlerThread:112 [system_monitor.py:start():194] Starting system monitor 2024-06-03 12:03:38,982 INFO SystemMonitor:112 [system_monitor.py:_start():158] Starting system asset monitoring threads 2024-06-03 12:03:38,982 INFO SystemMonitor:112 [interfaces.py:start():188] Started cpu monitoring 2024-06-03 12:03:38,983 INFO SystemMonitor:112 [interfaces.py:start():188] Started disk monitoring 2024-06-03 12:03:38,984 INFO SystemMonitor:112 [interfaces.py:start():188] Started gpu monitoring 2024-06-03 12:03:38,985 INFO SystemMonitor:112 [interfaces.py:start():188] Started memory monitoring 2024-06-03 12:03:38,985 INFO SystemMonitor:112 [interfaces.py:start():188] Started network monitoring 2024-06-03 12:03:39,116 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: pause 2024-06-03 12:03:39,116 INFO HandlerThread:112 [handler.py:handle_request_pause():724] stopping system metrics thread 2024-06-03 12:03:39,117 INFO HandlerThread:112 [system_monitor.py:finish():203] Stopping system monitor 2024-06-03 12:03:39,117 DEBUG SystemMonitor:112 [system_monitor.py:_start():172] Starting system metrics aggregation loop 2024-06-03 12:03:39,117 DEBUG SystemMonitor:112 [system_monitor.py:_start():179] Finished system metrics aggregation loop 2024-06-03 12:03:39,117 DEBUG SystemMonitor:112 [system_monitor.py:_start():183] Publishing last batch of metrics 2024-06-03 12:03:39,118 INFO HandlerThread:112 [interfaces.py:finish():200] Joined cpu monitor 2024-06-03 12:03:39,118 INFO HandlerThread:112 [interfaces.py:finish():200] Joined disk monitor 2024-06-03 12:03:39,124 INFO HandlerThread:112 [interfaces.py:finish():200] Joined gpu monitor 2024-06-03 12:03:39,125 INFO HandlerThread:112 [interfaces.py:finish():200] Joined memory monitor 2024-06-03 12:03:39,125 INFO HandlerThread:112 [interfaces.py:finish():200] Joined network monitor 2024-06-03 12:03:39,125 DEBUG SenderThread:112 [sender.py:send():378] send: stats 2024-06-03 12:03:41,617 INFO Thread-12 :112 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240603_111947-zd4tutif/files/output.log 2024-06-03 12:03:43,126 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 12:03:45,291 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: resume 2024-06-03 12:03:45,292 INFO HandlerThread:112 [handler.py:handle_request_resume():715] starting system metrics thread 2024-06-03 12:03:45,292 INFO HandlerThread:112 [system_monitor.py:start():194] Starting system monitor 2024-06-03 12:03:45,292 INFO SystemMonitor:112 [system_monitor.py:_start():158] Starting system asset monitoring threads 2024-06-03 12:03:45,292 INFO SystemMonitor:112 [interfaces.py:start():188] Started cpu monitoring 2024-06-03 12:03:45,293 INFO SystemMonitor:112 [interfaces.py:start():188] Started disk monitoring 2024-06-03 12:03:45,294 INFO SystemMonitor:112 [interfaces.py:start():188] Started gpu monitoring 2024-06-03 12:03:45,296 INFO SystemMonitor:112 [interfaces.py:start():188] Started memory monitoring 2024-06-03 12:03:45,297 INFO SystemMonitor:112 [interfaces.py:start():188] Started network monitoring 2024-06-03 12:03:48,127 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 12:03:52,855 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 12:03:52,856 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 12:03:53,996 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 12:03:58,997 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 12:04:03,999 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 12:04:07,856 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 12:04:07,856 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status 2024-06-03 12:04:09,025 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 12:04:14,027 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 12:04:19,027 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: status_report 2024-06-03 12:04:22,857 DEBUG HandlerThread:112 [handler.py:handle_request():158] handle_request: stop_status 2024-06-03 12:04:22,857 DEBUG SenderThread:112 [sender.py:send_request():405] send_request: stop_status