2023-11-22 23:41:40,886 INFO StreamThr :3236 [internal.py:wandb_internal():86] W&B internal server running at pid: 3236, started at: 2023-11-22 23:41:40.884904 2023-11-22 23:41:40,888 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status 2023-11-22 23:41:40,951 INFO WriterThread:3236 [datastore.py:open_for_write():85] open: /mnt/e/wandb/run-20231122_234140-blvl5h7j/run-blvl5h7j.wandb 2023-11-22 23:41:40,953 DEBUG SenderThread:3236 [sender.py:send():380] send: header 2023-11-22 23:41:40,954 DEBUG SenderThread:3236 [sender.py:send():380] send: run 2023-11-22 23:41:42,000 INFO SenderThread:3236 [dir_watcher.py:__init__():211] watching files in: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files 2023-11-22 23:41:42,006 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: check_version 2023-11-22 23:41:42,010 INFO SenderThread:3236 [sender.py:_start_run_threads():1124] run started: blvl5h7j with start time 1700689300.888471 2023-11-22 23:41:42,011 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: check_version 2023-11-22 23:41:42,072 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: run_start 2023-11-22 23:41:42,126 DEBUG HandlerThread:3236 [system_info.py:__init__():32] System info init 2023-11-22 23:41:42,126 DEBUG HandlerThread:3236 [system_info.py:__init__():47] System info init done 2023-11-22 23:41:42,126 INFO HandlerThread:3236 [system_monitor.py:start():194] Starting system monitor 2023-11-22 23:41:42,126 INFO SystemMonitor:3236 [system_monitor.py:_start():158] Starting system asset monitoring threads 2023-11-22 23:41:42,127 INFO HandlerThread:3236 [system_monitor.py:probe():214] Collecting system info 2023-11-22 23:41:42,132 INFO SystemMonitor:3236 [interfaces.py:start():190] Started cpu monitoring 2023-11-22 23:41:42,133 INFO SystemMonitor:3236 [interfaces.py:start():190] Started disk monitoring 2023-11-22 23:41:42,134 INFO SystemMonitor:3236 [interfaces.py:start():190] Started gpu monitoring 2023-11-22 23:41:42,139 INFO SystemMonitor:3236 [interfaces.py:start():190] Started memory monitoring 2023-11-22 23:41:42,140 INFO SystemMonitor:3236 [interfaces.py:start():190] Started network monitoring 2023-11-22 23:41:42,157 DEBUG HandlerThread:3236 [system_info.py:probe():196] Probing system 2023-11-22 23:41:42,272 DEBUG HandlerThread:3236 [gitlib.py:_init_repo():56] git repository is invalid 2023-11-22 23:41:42,272 DEBUG HandlerThread:3236 [system_info.py:probe():244] Probing system done 2023-11-22 23:41:42,273 DEBUG HandlerThread:3236 [system_monitor.py:probe():223] {'os': 'Linux-5.15.123.1-microsoft-standard-WSL2-x86_64-with-glibc2.35', 'python': '3.10.13', 'heartbeatAt': '2023-11-22T21:41:42.157485', 'startedAt': '2023-11-22T21:41:40.813315', 'docker': None, 'cuda': None, 'args': ('--local_rank=0', '--deepspeed=ds_config.json', '--model_name_or_path=openai/whisper-large-v3', '--train_dataset_names', 'mozilla-foundation/common_voice_11_0', 'mozilla-foundation/common_voice_11_0', 'google/fleurs', 'Finnish-NLP/css10_audio_processed', 'Finnish-NLP/youtube_fi_110k_preprocessed', 'Finnish-NLP/voxpopuli_fi_audio_processed', 'Finnish-NLP/parliament_audio_processed', 'Finnish-NLP/aalto_eduskunta_asr_audio_processed', 'Finnish-NLP/fbc_dialog_audio_processed', 'Finnish-NLP/fbc_monolog_audio_processed', '--train_dataset_config_names', 'fi', 'fi', 'fi_fi', '', '', '', '', '', '', '', '--train_dataset_split_name', 'train', 'validation', 'train', 'train', 'train', 'train', 'train', 'train', 'train', 'train', '--train_dataset_text_column_names', 'sentence', 'sentence', 'raw_transcription', 'sentence', 'texts', 'sentence', 'sentence', 'sentence', 'sentence', 'sentence', '--eval_dataset_name=mozilla-foundation/common_voice_11_0', '--eval_dataset_config_name=fi', '--eval_split_name=test', '--language=finnish', '--model_index_name=WhisperLargeV3Finnish', '--max_steps=60000', '--output_dir=./', '--per_device_train_batch_size=8', '--per_device_eval_batch_size=4', '--gradient_accumulation_steps=2', '--logging_steps=20', '--learning_rate=3e-6', '--warmup_steps=1000', '--evaluation_strategy=steps', '--eval_steps=1000', '--save_strategy=steps', '--save_steps=1000', '--generation_max_length=225', '--length_column_name=input_length', '--max_duration_in_seconds=30', '--text_column_name=sentence', '--freeze_feature_encoder=False', '--report_to=wandb', '--metric_for_best_model=wer', '--greater_is_better=False', '--load_best_model_at_end', '--gradient_checkpointing', '--fp16', '--torch_compile', '--overwrite_output_dir', '--do_train', '--do_eval', '--predict_with_generate', '--do_normalize_eval', '--use_auth_token', '--push_to_hub', '--push_to_hub_model_id=WhisperLargeFinnishV3', '--resume_from_checkpoint=./checkpoint-28000'), 'state': 'running', 'program': '/mnt/e/run_speech_recognition_seq2seq_streaming.py', 'codePathLocal': 'run_speech_recognition_seq2seq_streaming.py', 'codePath': 'run_speech_recognition_seq2seq_streaming.py', 'host': 'DESKTOP-59O9VN1', 'username': 'rasmus', 'executable': '/home/rasmus/miniconda3/envs/WhisperFinetuneEnv/bin/python', 'cpu_count': 4, 'cpu_count_logical': 8, 'cpu_freq': {'current': 3911.9949999999994, 'min': 0.0, 'max': 0.0}, 'cpu_freq_per_core': [{'current': 3911.995, 'min': 0.0, 'max': 0.0}, {'current': 3911.995, 'min': 0.0, 'max': 0.0}, {'current': 3911.995, 'min': 0.0, 'max': 0.0}, {'current': 3911.995, 'min': 0.0, 'max': 0.0}, {'current': 3911.995, 'min': 0.0, 'max': 0.0}, {'current': 3911.995, 'min': 0.0, 'max': 0.0}, {'current': 3911.995, 'min': 0.0, 'max': 0.0}, {'current': 3911.995, 'min': 0.0, 'max': 0.0}], 'disk': {'/': {'total': 1006.853931427002, 'used': 37.587677001953125}}, 'gpu': 'NVIDIA GeForce RTX 4080', 'gpu_count': 1, 'gpu_devices': [{'name': 'NVIDIA GeForce RTX 4080', 'memory_total': 17171480576}], 'memory': {'total': 54.92573928833008}} 2023-11-22 23:41:42,282 INFO HandlerThread:3236 [system_monitor.py:probe():224] Finished collecting system info 2023-11-22 23:41:42,282 INFO HandlerThread:3236 [system_monitor.py:probe():227] Publishing system info 2023-11-22 23:41:42,282 DEBUG HandlerThread:3236 [system_info.py:_save_pip():52] Saving list of pip packages installed into the current environment 2023-11-22 23:41:42,285 DEBUG HandlerThread:3236 [system_info.py:_save_pip():68] Saving pip packages done 2023-11-22 23:41:42,285 DEBUG HandlerThread:3236 [system_info.py:_save_conda():75] Saving list of conda packages installed into the current environment 2023-11-22 23:41:42,998 INFO Thread-12 :3236 [dir_watcher.py:_on_file_created():271] file/dir created: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/requirements.txt 2023-11-22 23:41:42,999 INFO Thread-12 :3236 [dir_watcher.py:_on_file_created():271] file/dir created: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/conda-environment.yaml 2023-11-22 23:41:47,011 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/conda-environment.yaml 2023-11-22 23:41:47,102 DEBUG HandlerThread:3236 [system_info.py:_save_conda():87] Saving conda packages done 2023-11-22 23:41:47,105 INFO HandlerThread:3236 [system_monitor.py:probe():229] Finished publishing system info 2023-11-22 23:41:47,133 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:41:47,133 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-22 23:41:47,133 DEBUG SenderThread:3236 [sender.py:send():380] send: files 2023-11-22 23:41:47,134 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-metadata.json with policy now 2023-11-22 23:41:47,152 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:41:47,153 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:41:47,164 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:41:47,424 DEBUG SenderThread:3236 [sender.py:send():380] send: telemetry 2023-11-22 23:41:47,425 DEBUG SenderThread:3236 [sender.py:send():380] send: config 2023-11-22 23:41:47,426 DEBUG SenderThread:3236 [sender.py:send():380] send: metric 2023-11-22 23:41:47,426 DEBUG SenderThread:3236 [sender.py:send():380] send: telemetry 2023-11-22 23:41:47,426 DEBUG SenderThread:3236 [sender.py:send():380] send: metric 2023-11-22 23:41:47,427 WARNING SenderThread:3236 [sender.py:send_metric():1331] Seen metric with glob (shouldn't happen) 2023-11-22 23:41:47,427 DEBUG SenderThread:3236 [sender.py:send():380] send: telemetry 2023-11-22 23:41:47,782 INFO wandb-upload_0:3236 [upload_job.py:push():131] Uploaded file /tmp/tmpezhpmgrqwandb/ycihc3wj-wandb-metadata.json 2023-11-22 23:41:48,015 INFO Thread-12 :3236 [dir_watcher.py:_on_file_created():271] file/dir created: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-metadata.json 2023-11-22 23:41:48,016 INFO Thread-12 :3236 [dir_watcher.py:_on_file_created():271] file/dir created: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-22 23:41:50,023 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-22 23:41:51,201 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:41:54,044 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-22 23:41:56,051 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-22 23:41:57,173 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:42:02,152 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:42:02,153 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:42:02,163 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:42:02,386 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:42:07,387 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:42:12,391 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:42:13,131 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/config.yaml 2023-11-22 23:42:17,152 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:42:17,153 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:42:17,200 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:42:18,344 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:42:23,345 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:42:28,346 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:42:32,152 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:42:32,153 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:42:32,200 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:42:33,369 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:42:38,370 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:42:42,141 DEBUG SystemMonitor:3236 [system_monitor.py:_start():172] Starting system metrics aggregation loop 2023-11-22 23:42:42,142 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-22 23:42:44,143 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:42:47,153 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:42:47,153 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:42:47,200 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:42:49,344 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:42:54,345 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:42:59,346 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:43:02,153 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:43:02,153 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:43:02,210 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:43:04,371 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:43:09,372 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:43:12,143 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-22 23:43:15,144 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:43:17,153 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:43:17,154 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:43:17,200 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:43:20,327 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:43:25,327 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:43:30,328 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:43:32,153 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:43:32,154 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:43:32,200 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:43:35,405 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:43:40,406 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:43:42,143 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-22 23:43:46,144 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:43:47,153 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:43:47,154 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:43:47,200 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:43:51,403 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:43:56,403 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:44:01,404 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:44:02,153 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:44:02,154 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:44:02,210 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:44:06,414 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:44:11,414 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:44:12,143 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-22 23:44:13,661 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-22 23:44:17,144 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:44:17,153 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:44:17,154 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:44:17,165 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:44:22,352 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:44:27,352 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:44:32,153 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:44:32,154 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:44:32,200 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:44:32,412 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:44:37,412 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:44:42,145 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-22 23:44:43,146 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:44:47,154 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:44:47,154 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:44:47,200 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:44:48,422 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:44:53,423 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:44:58,425 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:45:02,154 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:45:02,154 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:45:02,210 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:45:04,321 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:45:09,321 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:45:12,144 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-22 23:45:15,145 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:45:17,160 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:45:17,160 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:45:17,210 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:45:20,325 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:45:25,326 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:45:30,327 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:45:32,160 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:45:32,161 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:45:32,210 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:45:35,387 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:45:40,388 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:45:42,154 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-22 23:45:46,155 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:45:47,160 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:45:47,161 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:45:47,210 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:45:51,412 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:45:56,412 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:46:01,413 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:46:02,160 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:46:02,161 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:46:02,210 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:46:07,378 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:46:12,155 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-22 23:46:13,155 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:46:17,160 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:46:17,161 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:46:17,210 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:46:18,391 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:46:23,396 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:46:28,397 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:46:32,160 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:46:32,161 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:46:32,210 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:46:34,330 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:46:39,330 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:46:42,155 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-22 23:46:45,156 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:46:47,160 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:46:47,161 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:46:47,220 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:46:50,388 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:46:55,389 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:47:00,389 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:47:02,161 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:47:02,161 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:47:02,210 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:47:06,400 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:47:11,400 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:47:12,156 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-22 23:47:17,157 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:47:17,161 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:47:17,161 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:47:17,210 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:47:22,414 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:47:27,414 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:47:32,168 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:47:32,168 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:47:32,210 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:47:33,375 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:47:38,376 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:47:42,156 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-22 23:47:44,157 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:47:47,167 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:47:47,167 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:47:47,210 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:47:49,415 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:47:54,420 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:47:59,420 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:48:02,171 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:48:02,177 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:48:02,177 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:48:05,333 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:48:10,334 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:48:12,165 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-22 23:48:16,166 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:48:17,171 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:48:17,177 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:48:17,177 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:48:21,381 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:48:26,382 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:48:31,382 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:48:32,171 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:48:32,177 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:48:32,177 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:48:37,324 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:48:42,166 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-22 23:48:43,167 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:48:47,172 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:48:47,178 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:48:47,178 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:48:48,330 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:48:53,331 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:48:58,332 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:49:02,172 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:49:02,178 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:49:02,178 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:49:03,398 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:49:08,399 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:49:12,167 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-22 23:49:14,168 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:49:17,172 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:49:17,178 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:49:17,178 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:49:19,337 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:49:24,338 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:49:29,339 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:49:32,172 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:49:32,178 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:49:32,178 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:49:34,416 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:49:39,416 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:49:42,167 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-22 23:49:45,168 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:49:47,172 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:49:47,178 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:49:47,178 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:49:50,381 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:49:55,382 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:50:00,383 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:50:02,173 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:50:02,178 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:50:02,178 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:50:05,423 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:50:10,423 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:50:12,175 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-22 23:50:16,180 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:50:17,172 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:50:17,181 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:50:17,182 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:50:21,347 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:50:26,348 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:50:31,349 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:50:32,172 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:50:32,181 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:50:32,182 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:50:36,407 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:50:41,408 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:50:42,175 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-22 23:50:47,170 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:50:47,172 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:50:47,181 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:50:47,182 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:50:52,350 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:50:57,351 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:51:02,172 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:51:02,181 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:51:02,182 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:51:03,334 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:51:08,339 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:51:12,189 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-22 23:51:14,190 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:51:17,172 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:51:17,181 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:51:17,182 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:51:19,413 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:51:24,430 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:51:29,431 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:51:32,189 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:51:32,189 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:51:32,190 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:51:35,406 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:51:40,407 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:51:42,191 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-22 23:51:46,192 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:51:47,180 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:51:47,189 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:51:47,189 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:51:51,338 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:51:56,344 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:52:01,345 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:52:02,180 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:52:02,189 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:52:02,189 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:52:06,464 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:52:11,465 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:52:12,191 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-22 23:52:17,170 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:52:17,180 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:52:17,189 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:52:17,189 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:52:22,472 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:52:27,480 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:52:32,180 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:52:32,189 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:52:32,190 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:52:33,372 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:52:38,373 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:52:42,191 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-22 23:52:44,192 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:52:47,181 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:52:47,191 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:52:47,192 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:52:49,392 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:52:54,393 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:52:59,394 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:53:02,181 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:53:02,189 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:53:02,190 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:53:04,421 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:53:09,422 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:53:12,192 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-22 23:53:15,193 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:53:17,181 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:53:17,189 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:53:17,189 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:53:20,364 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:53:25,365 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:53:30,366 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:53:32,181 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:53:32,189 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:53:32,190 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:53:35,439 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:53:40,439 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:53:42,203 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-22 23:53:46,204 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:53:47,181 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:53:47,189 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:53:47,190 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:53:51,358 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:53:56,359 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:54:01,367 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:54:02,181 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:54:02,189 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:54:02,190 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:54:07,349 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:54:12,193 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-22 23:54:13,193 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:54:17,181 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:54:17,198 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:54:17,199 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:54:18,361 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:54:23,361 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:54:28,362 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:54:32,181 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:54:32,198 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:54:32,199 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:54:33,395 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:54:38,396 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:54:42,193 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-22 23:54:44,194 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:54:47,182 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:54:47,199 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:54:47,199 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:54:49,361 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:54:54,362 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:54:59,363 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:55:02,182 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:55:02,199 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:55:02,199 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:55:04,378 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:55:09,379 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:55:12,193 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-22 23:55:15,194 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:55:17,182 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:55:17,199 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:55:17,199 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:55:20,393 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:55:25,394 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:55:30,394 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:55:32,182 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:55:32,209 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:55:32,210 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:55:36,385 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:55:41,386 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:55:42,194 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-22 23:55:47,172 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:55:47,182 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:55:47,210 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:55:47,210 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:55:52,398 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:55:57,398 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:56:02,182 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:56:02,210 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:56:02,210 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:56:02,471 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:56:07,472 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:56:12,195 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-22 23:56:13,195 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:56:17,182 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:56:17,210 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:56:17,210 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:56:18,426 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:56:23,426 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:56:27,308 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-22 23:56:29,023 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:56:32,182 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:56:32,210 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:56:32,210 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:56:34,462 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:56:39,462 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:56:42,195 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-22 23:56:45,196 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:56:47,182 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:56:47,210 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:56:47,211 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:56:50,455 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:56:55,456 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:57:00,457 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:57:02,182 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:57:02,210 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:57:02,210 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:57:06,388 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:57:11,389 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:57:12,195 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-22 23:57:17,172 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:57:17,182 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:57:17,210 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:57:17,211 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:57:22,382 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:57:27,382 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:57:32,183 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:57:32,210 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:57:32,211 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:57:32,419 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:57:37,424 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:57:42,196 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-22 23:57:43,197 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:57:47,183 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:57:47,220 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:57:47,220 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:57:48,424 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:57:53,424 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:57:58,425 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:58:02,183 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:58:02,230 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:58:02,231 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:58:03,469 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:58:08,470 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:58:12,196 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-22 23:58:14,197 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:58:17,183 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:58:17,230 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:58:17,231 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:58:19,404 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:58:24,410 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:58:29,410 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:58:32,183 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:58:32,230 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:58:32,231 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:58:34,438 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:58:39,444 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:58:42,197 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-22 23:58:45,198 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:58:47,183 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:58:47,230 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:58:47,231 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:58:50,420 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:58:55,420 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:59:00,421 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:59:02,183 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:59:02,230 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:59:02,231 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:59:05,467 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:59:10,470 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:59:12,197 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-22 23:59:16,198 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:59:17,183 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:59:17,231 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:59:17,231 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:59:21,431 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:59:26,432 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:59:31,433 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:59:32,183 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:59:32,231 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:59:32,231 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:59:36,471 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:59:41,471 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:59:42,198 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-22 23:59:47,173 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:59:47,183 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-22 23:59:47,231 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-22 23:59:47,231 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-22 23:59:52,383 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:59:57,384 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-22 23:59:59,017 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 00:00:02,184 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:00:02,231 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:00:02,243 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:00:02,445 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:00:07,445 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:00:12,198 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:00:13,199 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:00:17,201 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:00:17,231 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:00:17,232 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:00:18,471 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:00:23,480 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:00:28,481 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:00:32,201 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:00:32,231 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:00:32,231 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:00:34,461 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:00:39,461 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:00:42,199 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:00:45,200 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:00:47,201 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:00:47,231 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:00:47,232 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:00:50,487 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:00:55,488 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:01:00,489 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:01:02,201 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:01:02,233 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:01:02,233 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:01:06,417 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:01:11,417 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:01:12,199 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:01:17,200 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:01:17,202 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:01:17,232 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:01:17,233 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:01:22,440 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:01:27,441 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:01:32,202 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:01:32,232 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:01:32,233 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:01:33,428 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:01:38,428 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:01:42,200 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:01:44,201 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:01:47,202 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:01:47,233 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:01:47,233 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:01:49,485 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:01:54,486 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:01:59,486 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:02:02,202 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:02:02,233 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:02:02,233 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:02:05,461 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:02:10,462 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:02:12,200 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:02:16,202 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:02:17,202 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:02:17,233 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:02:17,233 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:02:21,445 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:02:26,446 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:02:31,447 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:02:32,202 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:02:32,233 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:02:32,233 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:02:36,483 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:02:41,484 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:02:42,201 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:02:47,202 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:02:47,203 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:02:47,233 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:02:47,234 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:02:52,442 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:02:57,450 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:03:02,202 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:03:02,233 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:03:02,234 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:03:02,487 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:03:07,480 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:03:12,202 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:03:13,202 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:03:17,202 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:03:17,243 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:03:17,243 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:03:18,399 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:03:23,400 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:03:28,400 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:03:32,203 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:03:32,243 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:03:32,243 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:03:33,458 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:03:38,469 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:03:42,202 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:03:44,203 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:03:47,203 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:03:47,243 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:03:47,244 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:03:49,498 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:03:54,499 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:03:59,500 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:04:02,203 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:04:02,243 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:04:02,244 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:04:05,431 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:04:10,432 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:04:12,202 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:04:16,203 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:04:17,203 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:04:17,243 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:04:17,244 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:04:21,455 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:04:26,455 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:04:31,456 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:04:32,203 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:04:32,243 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:04:32,244 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:04:36,467 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:04:41,468 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:04:42,203 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:04:47,203 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:04:47,203 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:04:47,243 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:04:47,244 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:04:52,413 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:04:57,414 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:05:02,203 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:05:02,244 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:05:02,244 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:05:02,466 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:05:07,467 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:05:12,204 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:05:13,204 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:05:17,203 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:05:17,250 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:05:17,250 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:05:18,499 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:05:23,500 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:05:28,501 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:05:32,204 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:05:32,250 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:05:32,250 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:05:33,520 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:05:38,521 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:05:42,204 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:05:44,205 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:05:47,204 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:05:47,250 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:05:47,251 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:05:49,501 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:05:54,508 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:05:59,509 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:06:02,210 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:06:02,250 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:06:02,251 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:06:05,500 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:06:10,501 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:06:12,204 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:06:16,206 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:06:17,210 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:06:17,250 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:06:17,251 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:06:21,644 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:06:26,645 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:06:31,646 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:06:32,210 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:06:32,250 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:06:32,251 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:06:37,409 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:06:42,205 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:06:43,206 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:06:47,210 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:06:47,250 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:06:47,251 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:06:48,502 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:06:53,503 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:06:58,504 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:07:02,210 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:07:02,250 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:07:02,251 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:07:04,493 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:07:09,494 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:07:12,206 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:07:15,207 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:07:17,218 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:07:17,251 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:07:17,251 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:07:20,467 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:07:25,468 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:07:30,468 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:07:32,218 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:07:32,251 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:07:32,251 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:07:35,491 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:07:40,491 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:07:42,206 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:07:46,207 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:07:47,218 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:07:47,252 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:07:47,253 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:07:51,437 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:07:56,440 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:08:01,447 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:08:02,218 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:08:02,252 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:08:02,253 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:08:07,422 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:08:12,207 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:08:13,207 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:08:17,218 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:08:17,252 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:08:17,253 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:08:18,420 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:08:23,421 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:08:28,422 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:08:32,218 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:08:32,252 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:08:32,253 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:08:33,527 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:08:38,528 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:08:42,207 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:08:44,208 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:08:47,220 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:08:47,253 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:08:47,254 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:08:49,423 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:08:54,424 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:08:59,425 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:09:02,219 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:09:02,253 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:09:02,253 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:09:04,429 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:09:09,429 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:09:12,238 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:09:15,239 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:09:17,219 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:09:17,253 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:09:17,253 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:09:20,434 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:09:26,080 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 00:09:26,166 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:09:31,167 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:09:32,221 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:09:32,253 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:09:32,253 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:09:36,410 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:09:41,410 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:09:42,208 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:09:47,209 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:09:47,221 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:09:47,253 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:09:47,253 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:09:52,470 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:09:57,471 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:10:02,222 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:10:02,253 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:10:02,254 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:10:03,444 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:10:08,445 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:10:12,208 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:10:14,210 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:10:17,222 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:10:17,253 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:10:17,254 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:10:19,456 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:10:24,456 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:10:29,457 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:10:32,222 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:10:32,253 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:10:32,254 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:10:34,501 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:10:39,502 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:10:42,219 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:10:45,220 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:10:47,222 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:10:47,253 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:10:47,254 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:10:50,468 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:10:55,469 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:11:00,480 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:11:02,222 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:11:02,253 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:11:02,254 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:11:05,524 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:11:10,524 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:11:12,220 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:11:16,221 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:11:17,222 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:11:17,254 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:11:17,254 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:11:21,451 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:11:26,460 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:11:31,460 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:11:32,222 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:11:32,254 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:11:32,254 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:11:37,430 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:11:42,220 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:11:43,221 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:11:47,222 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:11:47,254 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:11:47,254 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:11:48,469 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:11:53,470 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:11:58,470 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:12:02,222 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:12:02,254 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:12:02,255 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:12:04,446 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:12:09,447 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:12:12,220 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:12:15,222 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:12:17,222 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:12:17,254 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:12:17,255 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:12:20,482 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:12:25,483 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:12:30,490 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:12:32,222 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:12:32,254 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:12:32,255 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:12:35,502 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:12:40,502 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:12:42,221 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:12:46,222 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:12:47,222 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:12:47,258 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:12:47,258 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:12:51,449 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:12:56,450 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:13:01,451 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:13:02,223 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:13:02,254 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:13:02,255 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:13:07,437 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:13:12,221 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:13:13,222 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:13:17,223 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:13:17,255 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:13:17,255 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:13:18,517 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:13:23,518 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:13:28,518 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:13:32,223 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:13:32,255 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:13:32,255 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:13:34,428 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:13:39,429 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:13:42,222 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:13:45,223 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:13:47,223 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:13:47,255 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:13:47,255 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:13:50,412 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:13:55,413 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:14:00,414 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:14:02,223 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:14:02,255 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:14:02,255 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:14:05,418 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:14:10,419 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:14:12,222 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:14:16,223 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:14:17,234 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:14:17,255 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:14:17,255 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:14:21,465 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:14:26,466 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:14:31,467 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:14:32,234 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:14:32,255 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:14:32,256 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:14:36,492 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:14:41,493 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:14:42,223 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:14:47,218 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:14:47,234 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:14:47,255 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:14:47,256 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:14:52,437 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:14:57,438 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:15:02,234 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:15:02,256 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:15:02,256 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:15:02,493 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:15:07,494 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:15:12,232 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:15:13,233 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:15:17,234 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:15:17,256 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:15:17,256 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:15:18,507 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:15:23,517 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:15:28,518 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:15:32,234 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:15:32,256 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:15:32,256 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:15:34,504 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:15:39,504 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:15:42,224 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:15:45,225 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:15:47,234 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:15:47,256 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:15:47,256 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:15:50,424 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:15:55,424 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:16:00,425 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:16:02,235 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:16:02,260 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:16:02,260 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:16:05,464 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:16:10,464 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:16:12,224 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:16:16,225 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:16:17,235 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:16:17,260 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:16:17,261 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:16:21,426 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:16:26,426 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:16:31,427 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:16:32,235 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:16:32,260 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:16:32,260 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:16:36,438 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:16:41,439 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:16:42,225 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:16:47,219 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:16:47,250 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:16:47,260 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:16:47,260 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:16:52,514 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:16:57,981 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:16:58,743 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 00:17:02,250 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:17:02,260 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:17:02,261 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:17:03,432 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:17:08,433 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:17:12,225 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:17:14,226 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:17:17,250 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:17:17,260 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:17:17,261 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:17:19,456 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:17:24,473 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:17:29,474 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:17:32,250 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:17:32,260 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:17:32,261 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:17:34,479 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:17:39,480 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:17:42,226 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:17:45,227 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:17:47,250 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:17:47,260 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:17:47,261 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:17:50,445 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:17:55,446 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:18:00,447 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:18:02,250 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:18:02,270 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:18:02,270 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:18:05,454 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:18:10,455 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:18:12,227 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:18:16,228 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:18:17,250 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:18:17,270 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:18:17,270 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:18:21,451 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:18:26,452 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:18:31,452 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:18:32,250 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:18:32,270 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:18:32,271 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:18:36,533 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:18:41,534 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:18:42,227 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:18:47,219 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:18:47,251 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:18:47,270 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:18:47,271 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:18:52,490 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:18:57,492 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:19:02,251 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:19:02,270 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:19:02,271 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:19:03,470 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:19:08,470 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:19:12,228 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:19:14,228 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:19:17,251 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:19:17,270 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:19:17,271 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:19:19,461 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:19:24,462 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:19:29,462 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:19:32,251 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:19:32,270 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:19:32,271 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:19:34,472 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:19:39,473 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:19:42,228 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:19:45,229 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:19:47,251 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:19:47,271 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:19:47,271 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:19:50,496 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:19:55,497 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:20:00,498 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:20:02,251 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:20:02,271 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:20:02,271 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:20:05,526 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:20:10,526 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:20:12,229 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:20:16,230 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:20:17,251 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:20:17,271 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:20:17,271 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:20:21,520 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:20:26,520 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:20:31,521 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:20:32,251 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:20:32,271 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:20:32,271 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:20:37,420 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:20:42,229 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:20:43,230 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:20:47,260 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:20:47,271 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:20:47,271 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:20:48,467 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:20:53,467 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:20:58,470 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:21:02,260 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:21:02,271 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:21:02,272 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:21:03,503 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:21:08,504 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:21:12,230 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:21:14,231 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:21:17,260 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:21:17,299 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:21:17,300 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:21:19,520 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:21:24,521 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:21:29,522 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:21:32,260 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:21:32,271 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:21:32,272 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:21:35,468 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:21:40,469 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:21:42,231 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:21:46,232 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:21:47,260 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:21:47,271 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:21:47,272 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:21:51,530 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:21:56,531 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:22:01,532 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:22:02,260 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:22:02,271 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:22:02,275 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:22:07,471 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:22:12,236 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:22:13,237 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:22:17,260 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:22:17,272 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:22:17,272 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:22:18,501 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:22:23,501 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:22:29,025 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 00:22:29,488 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:22:32,261 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:22:32,272 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:22:32,272 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:22:34,518 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:22:39,520 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:22:42,237 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:22:45,237 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:22:47,261 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:22:47,272 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:22:47,273 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:22:50,488 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:22:55,488 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:23:00,489 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:23:02,261 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:23:02,272 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:23:02,272 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:23:06,471 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:23:11,471 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:23:12,238 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:23:17,221 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:23:17,261 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:23:17,280 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:23:17,280 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:23:22,550 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:23:27,560 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:23:32,262 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:23:32,280 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:23:32,280 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:23:33,445 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:23:38,446 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:23:42,240 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:23:44,241 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:23:47,261 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:23:47,280 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:23:47,280 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:23:49,497 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:23:54,498 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:23:59,499 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:24:02,262 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:24:02,280 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:24:02,281 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:24:04,523 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:24:09,524 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:24:12,239 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:24:15,247 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:24:17,262 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:24:17,280 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:24:17,281 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:24:20,464 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:24:25,465 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:24:30,480 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:24:32,262 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:24:32,280 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:24:32,281 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:24:35,490 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:24:40,490 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:24:42,239 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:24:46,240 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:24:47,262 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:24:47,280 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:24:47,281 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:24:51,469 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:24:56,469 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:25:01,470 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:25:02,262 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:25:02,280 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:25:02,281 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:25:06,489 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:25:11,490 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:25:12,239 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:25:17,223 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:25:17,262 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:25:17,281 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:25:17,281 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:25:22,469 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:25:27,470 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:25:32,262 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:25:32,281 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:25:32,281 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:25:32,500 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:25:37,501 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:25:42,240 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:25:43,241 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:25:47,262 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:25:47,281 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:25:47,281 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:25:48,486 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:25:53,490 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:25:58,491 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:26:02,263 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:26:02,281 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:26:02,281 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:26:03,546 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:26:08,546 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:26:12,240 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:26:14,241 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:26:17,263 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:26:17,290 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:26:17,290 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:26:19,590 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:26:24,590 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:26:29,591 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:26:32,263 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:26:32,288 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:26:32,289 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:26:35,540 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:26:40,540 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:26:42,241 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:26:46,242 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:26:47,263 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:26:47,289 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:26:47,289 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:26:51,496 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:26:56,497 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:27:01,497 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:27:02,263 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:27:02,289 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:27:02,289 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:27:07,489 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:27:12,242 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:27:13,242 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:27:17,263 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:27:17,300 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:27:17,300 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:27:18,552 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:27:23,553 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:27:28,554 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:27:32,263 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:27:32,300 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:27:32,301 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:27:34,516 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:27:39,517 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:27:42,242 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:27:45,243 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:27:47,263 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:27:47,300 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:27:47,301 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:27:50,505 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:27:55,506 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:28:00,507 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:28:02,263 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:28:02,300 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:28:02,301 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:28:05,523 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:28:10,524 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:28:12,242 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:28:16,244 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:28:17,263 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:28:17,300 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:28:17,301 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:28:21,497 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:28:26,498 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:28:31,499 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:28:32,264 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:28:32,300 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:28:32,301 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:28:36,568 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:28:41,569 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:28:42,243 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:28:47,231 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:28:47,264 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:28:47,300 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:28:47,301 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:28:52,532 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:28:57,540 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:29:02,264 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:29:02,300 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:29:02,301 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:29:03,465 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:29:08,466 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:29:12,243 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:29:14,244 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:29:17,264 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:29:17,301 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:29:17,301 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:29:19,528 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:29:24,528 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:29:29,529 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:29:32,264 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:29:32,301 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:29:32,302 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:29:35,518 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:29:40,519 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:29:42,244 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:29:46,245 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:29:47,264 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:29:47,301 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:29:47,301 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:29:51,507 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:29:56,508 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:30:01,508 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:30:02,264 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:30:02,301 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:30:02,301 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:30:07,506 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:30:12,244 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:30:13,245 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:30:17,264 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:30:17,301 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:30:17,301 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:30:18,530 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:30:23,538 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:30:28,539 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:30:32,264 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:30:32,301 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:30:32,302 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:30:34,457 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:30:39,457 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:30:42,245 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:30:45,246 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:30:47,265 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:30:47,301 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:30:47,302 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:30:50,526 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:30:55,527 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:31:00,528 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:31:02,265 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:31:02,301 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:31:02,302 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:31:05,537 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:31:10,540 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:31:12,245 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:31:16,247 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:31:17,274 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:31:17,302 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:31:17,302 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:31:21,559 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:31:26,560 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:31:31,560 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:31:32,274 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:31:32,302 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:31:32,302 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:31:37,472 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:31:42,246 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:31:43,247 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:31:47,274 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:31:47,302 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:31:47,302 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:31:48,501 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:31:53,502 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:31:58,502 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:32:02,274 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:32:02,302 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:32:02,302 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:32:03,512 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:32:08,513 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:32:12,247 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:32:14,247 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:32:17,274 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:32:17,302 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:32:17,303 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:32:19,497 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:32:24,498 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:32:29,508 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:32:32,274 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:32:32,302 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:32:32,303 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:32:35,495 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:32:40,495 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:32:42,247 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:32:46,248 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:32:47,274 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:32:47,302 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:32:47,303 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:32:51,559 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:32:56,560 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:33:01,561 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:33:02,274 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:33:02,302 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:33:02,303 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:33:06,575 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:33:11,576 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:33:12,247 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:33:17,233 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:33:17,274 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:33:17,302 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:33:17,303 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:33:22,504 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:33:27,505 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:33:32,275 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:33:32,302 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:33:32,303 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:33:33,462 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:33:38,463 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:33:42,248 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:33:44,249 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:33:47,275 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:33:47,303 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:33:47,303 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:33:49,565 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:33:54,566 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:33:59,567 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:34:02,275 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:34:02,303 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:34:02,303 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:34:04,638 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:34:09,639 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:34:12,249 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:34:15,250 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:34:17,275 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:34:17,303 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:34:17,303 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:34:20,479 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:34:25,479 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:34:30,480 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:34:32,275 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:34:32,303 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:34:32,304 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:34:36,458 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:34:41,459 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:34:42,249 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:34:47,233 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:34:47,275 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:34:47,303 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:34:47,303 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:34:49,805 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 00:34:52,866 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:34:57,865 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:35:02,275 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:35:02,303 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:35:02,304 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:35:03,496 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:35:08,497 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:35:12,250 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:35:14,250 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:35:17,275 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:35:17,303 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:35:17,304 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:35:19,496 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:35:24,497 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:35:27,955 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 00:35:29,736 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:35:32,276 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:35:32,303 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:35:32,304 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:35:35,583 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:35:40,590 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:35:42,250 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:35:46,251 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:35:47,276 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:35:47,303 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:35:47,304 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:35:51,463 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:35:56,464 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:36:01,465 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:36:02,276 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:36:02,303 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:36:02,304 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:36:06,487 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:36:11,488 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:36:12,251 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:36:17,234 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:36:17,276 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:36:17,303 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:36:17,304 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:36:22,495 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:36:27,496 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:36:32,276 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:36:32,304 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:36:32,304 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:36:32,514 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:36:37,514 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:36:42,251 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:36:43,252 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:36:47,276 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:36:47,304 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:36:47,304 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:36:48,605 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:36:53,606 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:36:58,606 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:37:02,276 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:37:02,304 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:37:02,310 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:37:04,540 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:37:09,540 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:37:12,252 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:37:15,252 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:37:17,276 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:37:17,304 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:37:17,304 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:37:20,601 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:37:25,602 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:37:30,602 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:37:32,276 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:37:32,304 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:37:32,304 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:37:36,498 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:37:41,498 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:37:42,252 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:37:47,235 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:37:47,276 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:37:47,304 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:37:47,305 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:37:52,559 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:37:57,560 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:38:02,277 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:38:02,304 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:38:02,305 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:38:03,530 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:38:08,531 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:38:12,253 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:38:14,253 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:38:17,277 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:38:17,304 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:38:17,305 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:38:19,471 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:38:24,472 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:38:29,480 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:38:32,277 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:38:32,304 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:38:32,305 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:38:35,476 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:38:40,477 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:38:42,253 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:38:46,254 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:38:47,277 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:38:47,305 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:38:47,305 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:38:51,548 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:38:56,548 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:39:01,560 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:39:02,277 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:39:02,305 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:39:02,305 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:39:07,469 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:39:12,264 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:39:13,264 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:39:17,277 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:39:17,305 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:39:17,305 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:39:18,530 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:39:23,531 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:39:28,539 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:39:32,277 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:39:32,305 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:39:32,305 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:39:34,498 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:39:39,499 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:39:42,254 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:39:45,255 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:39:47,277 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:39:47,305 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:39:47,305 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:39:50,499 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:39:55,500 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:40:00,501 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:40:02,277 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:40:02,305 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:40:02,305 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:40:05,562 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:40:10,563 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:40:12,255 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:40:16,257 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:40:17,277 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:40:17,305 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:40:17,305 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:40:21,552 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:40:26,560 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:40:31,562 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:40:32,278 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:40:32,312 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:40:32,315 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:40:37,548 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:40:42,255 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:40:43,256 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:40:47,278 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:40:47,305 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:40:47,315 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:40:48,562 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:40:53,563 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:40:58,564 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:41:02,278 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:41:02,305 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:41:02,306 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:41:04,522 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:41:09,523 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:41:12,256 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:41:15,257 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:41:17,278 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:41:17,305 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:41:17,306 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:41:20,532 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:41:25,533 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:41:30,534 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:41:32,278 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:41:32,306 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:41:32,306 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:41:35,553 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:41:40,554 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:41:42,256 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:41:46,258 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:41:47,278 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:41:47,306 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:41:47,306 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:41:51,518 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:41:56,519 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:42:01,530 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:42:02,279 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:42:02,306 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:42:02,306 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:42:06,556 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:42:11,557 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:42:12,257 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:42:17,240 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:42:17,279 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:42:17,306 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:42:17,306 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:42:22,556 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:42:27,557 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:42:32,279 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:42:32,307 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:42:32,307 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:42:33,527 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:42:38,528 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:42:42,257 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:42:44,258 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:42:47,279 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:42:47,306 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:42:47,307 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:42:49,542 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:42:54,543 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:42:59,543 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:43:02,279 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:43:02,306 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:43:02,307 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:43:05,482 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:43:10,483 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:43:12,269 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:43:16,270 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:43:17,279 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:43:17,306 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:43:17,307 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:43:21,560 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:43:26,561 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:43:31,561 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:43:32,279 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:43:32,307 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:43:32,307 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:43:36,565 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:43:41,566 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:43:42,269 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:43:47,241 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:43:47,280 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:43:47,307 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:43:47,308 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:43:52,542 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:43:57,550 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:44:02,280 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:44:02,307 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:44:02,308 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:44:03,522 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:44:08,523 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:44:12,269 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:44:14,270 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:44:17,280 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:44:17,307 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:44:17,308 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:44:19,559 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:44:24,560 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:44:29,560 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:44:32,280 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:44:32,307 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:44:32,308 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:44:35,518 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:44:40,526 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:44:42,270 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:44:46,271 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:44:47,280 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:44:47,307 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:44:47,308 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:44:51,528 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:44:56,529 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:45:01,530 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:45:02,280 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:45:02,307 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:45:02,308 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:45:07,472 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:45:12,271 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:45:13,271 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:45:17,280 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:45:17,308 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:45:17,308 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:45:18,560 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:45:23,561 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:45:28,570 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:45:32,280 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:45:32,308 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:45:32,308 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:45:34,555 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:45:39,556 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:45:42,271 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:45:45,271 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:45:47,281 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:45:47,320 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:45:47,320 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:45:50,527 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:45:55,528 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:46:00,537 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:46:02,280 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:46:02,320 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:46:02,321 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:46:05,583 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:46:10,584 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:46:12,271 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:46:16,273 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:46:17,281 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:46:17,320 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:46:17,321 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:46:21,522 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:46:26,523 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:46:31,523 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:46:32,281 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:46:32,320 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:46:32,321 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:46:36,559 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:46:41,560 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:46:42,272 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:46:47,243 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:46:47,281 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:46:47,321 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:46:47,321 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:46:52,568 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:46:57,569 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:47:02,281 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:47:02,321 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:47:02,322 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:47:02,733 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:47:07,740 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:47:12,272 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:47:13,273 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:47:17,281 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:47:17,321 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:47:17,322 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:47:18,536 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:47:23,537 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:47:28,537 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:47:32,281 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:47:32,321 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:47:32,322 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:47:34,489 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:47:39,490 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:47:42,273 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:47:45,273 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:47:47,281 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:47:47,321 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:47:47,322 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:47:50,513 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:47:55,514 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:48:00,514 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:48:02,281 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:48:02,322 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:48:02,322 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:48:05,560 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:48:10,561 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:48:12,273 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:48:16,274 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:48:17,281 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:48:17,322 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:48:17,323 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:48:21,575 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:48:26,575 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:48:31,576 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:48:32,281 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:48:32,322 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:48:32,323 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:48:37,514 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:48:42,274 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:48:42,587 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:48:44,907 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 00:48:47,281 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:48:47,322 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:48:47,323 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:48:48,501 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:48:53,504 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:48:58,505 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:49:02,282 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:49:02,322 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:49:02,322 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:49:03,559 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:49:08,560 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:49:12,274 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:49:14,275 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:49:17,282 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:49:17,322 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:49:17,323 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:49:19,533 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:49:24,534 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:49:29,535 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:49:32,282 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:49:32,322 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:49:32,323 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:49:35,485 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:49:40,486 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:49:42,275 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:49:46,276 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:49:47,282 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:49:47,322 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:49:47,323 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:49:51,555 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:49:56,556 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:50:01,564 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:50:02,282 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:50:02,322 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:50:02,323 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:50:07,488 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:50:12,275 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:50:13,276 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:50:17,282 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:50:17,322 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:50:17,323 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:50:18,508 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:50:23,509 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:50:28,527 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:50:32,290 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:50:32,323 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:50:32,323 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:50:33,547 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:50:38,548 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:50:42,276 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:50:44,277 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:50:47,290 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:50:47,323 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:50:47,323 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:50:49,577 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:50:54,578 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:50:59,578 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:51:02,290 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:51:02,323 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:51:02,323 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:51:05,541 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:51:10,541 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:51:12,277 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:51:16,278 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:51:17,291 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:51:17,323 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:51:17,324 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:51:21,504 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:51:26,510 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:51:31,510 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:51:32,290 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:51:32,323 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:51:32,324 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:51:36,579 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:51:41,589 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:51:42,277 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:51:47,270 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:51:47,291 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:51:47,323 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:51:47,324 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:51:52,539 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:51:52,620 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 00:51:57,540 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:52:02,291 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:52:02,323 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:52:02,324 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:52:02,563 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:52:07,570 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:52:12,277 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:52:13,278 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:52:17,298 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:52:17,323 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:52:17,324 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:52:18,544 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:52:23,550 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:52:28,550 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:52:32,299 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:52:32,323 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:52:32,324 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:52:33,557 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:52:38,557 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:52:42,278 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:52:44,279 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:52:47,299 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:52:47,324 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:52:47,324 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:52:49,510 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:52:54,520 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:52:59,520 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:53:02,299 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:53:02,324 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:53:02,324 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:53:04,570 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:53:09,571 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:53:12,278 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:53:15,279 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:53:17,299 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:53:17,324 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:53:17,324 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:53:20,582 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:53:25,583 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:53:30,584 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:53:32,299 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:53:32,324 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:53:32,324 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:53:36,556 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:53:41,557 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:53:42,279 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:53:47,271 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:53:47,299 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:53:47,324 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:53:47,324 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:53:52,590 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:53:57,591 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:54:02,299 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:54:02,324 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:54:02,325 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:54:03,578 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:54:08,579 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:54:12,290 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:54:14,291 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:54:17,299 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:54:17,324 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:54:17,325 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:54:19,514 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:54:24,515 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:54:29,515 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:54:32,300 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:54:32,324 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:54:32,325 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:54:35,484 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:54:40,484 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:54:42,280 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:54:46,281 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:54:47,300 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:54:47,324 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:54:47,325 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:54:51,495 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:54:56,495 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:55:01,496 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:55:02,300 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:55:02,325 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:55:02,325 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:55:06,551 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:55:11,552 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:55:12,280 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:55:17,272 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:55:17,300 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:55:17,325 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:55:17,325 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:55:22,524 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:55:27,525 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:55:32,300 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:55:32,325 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:55:32,325 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:55:32,580 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:55:37,580 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:55:42,281 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:55:43,281 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:55:47,300 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:55:47,325 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:55:47,325 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:55:48,508 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:55:53,508 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:55:58,509 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:56:02,300 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:56:02,325 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:56:02,325 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:56:03,561 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:56:08,561 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:56:12,281 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:56:14,282 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:56:17,300 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:56:17,325 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:56:17,325 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:56:19,584 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:56:24,595 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:56:29,595 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:56:32,300 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:56:32,325 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:56:32,326 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:56:35,512 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:56:40,512 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:56:42,282 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:56:46,283 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:56:47,301 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:56:47,335 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:56:47,336 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:56:51,584 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:56:56,584 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:57:01,585 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:57:02,301 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:57:02,335 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:57:02,336 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:57:07,515 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:57:12,283 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:57:13,284 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:57:17,301 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:57:17,336 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:57:17,336 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:57:18,504 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:57:23,505 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:57:28,506 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:57:32,301 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:57:32,335 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:57:32,336 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:57:33,515 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:57:38,516 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:57:42,283 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:57:44,284 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:57:47,301 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:57:47,335 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:57:47,336 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:57:49,534 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:57:54,534 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:57:59,540 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:58:02,301 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:58:02,335 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:58:02,336 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:58:05,538 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:58:10,539 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:58:12,283 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:58:16,284 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:58:17,301 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:58:17,336 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:58:17,336 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:58:21,550 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:58:26,554 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:58:31,555 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:58:32,301 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:58:32,336 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:58:32,336 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:58:37,550 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:58:42,284 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:58:43,284 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:58:47,301 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:58:47,336 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:58:47,337 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:58:48,594 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:58:53,595 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:58:58,595 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:59:02,301 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:59:02,336 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:59:02,337 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:59:04,519 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:59:09,520 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:59:12,284 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:59:15,285 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:59:17,301 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:59:17,336 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:59:17,337 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:59:20,545 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:59:25,546 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:59:30,547 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:59:32,302 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:59:32,336 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:59:32,336 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:59:36,531 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:59:41,531 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:59:42,284 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 00:59:47,273 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:59:47,302 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 00:59:47,336 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 00:59:47,337 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 00:59:52,541 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 00:59:57,541 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:00:02,302 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:00:02,337 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:00:02,338 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:00:02,574 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:00:07,574 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:00:12,285 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:00:13,286 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:00:17,302 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:00:17,337 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:00:17,338 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:00:18,563 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:00:23,564 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:00:28,564 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:00:32,310 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:00:32,338 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:00:32,338 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:00:34,499 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:00:39,500 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:00:42,286 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:00:45,286 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:00:47,310 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:00:47,338 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:00:47,338 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:00:50,515 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:00:55,516 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:01:00,516 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:01:02,310 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:01:02,338 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:01:02,338 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:01:05,577 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:01:10,577 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:01:12,286 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:01:16,287 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:01:17,310 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:01:17,338 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:01:17,340 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:01:21,559 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:01:26,559 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:01:31,560 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:01:32,310 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:01:32,338 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:01:32,339 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:01:37,491 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:01:42,286 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:01:43,287 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:01:47,310 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:01:47,338 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:01:47,339 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:01:48,505 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:01:54,231 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:01:54,800 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 01:01:59,231 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:02:02,310 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:02:02,338 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:02:02,339 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:02:04,492 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:02:09,492 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:02:12,287 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:02:15,288 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:02:17,310 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:02:17,338 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:02:17,339 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:02:20,506 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:02:25,506 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:02:30,507 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:02:32,311 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:02:32,338 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:02:32,339 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:02:35,544 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:02:40,544 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:02:42,287 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:02:46,289 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:02:47,311 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:02:47,339 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:02:47,339 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:02:51,501 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:02:56,502 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:03:01,503 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:03:02,311 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:03:02,339 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:03:02,339 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:03:06,545 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:03:11,550 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:03:12,288 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:03:17,284 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:03:17,311 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:03:17,349 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:03:17,349 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:03:22,576 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:03:27,576 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:03:32,311 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:03:32,349 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:03:32,349 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:03:32,605 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:03:37,606 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:03:42,289 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:03:43,289 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:03:47,311 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:03:47,349 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:03:47,350 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:03:48,515 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:03:53,515 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:03:58,516 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:04:02,311 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:04:02,349 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:04:02,350 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:04:03,608 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:04:08,609 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:04:12,289 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:04:14,290 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:04:17,311 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:04:17,349 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:04:17,350 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:04:19,591 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:04:24,592 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:04:29,592 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:04:32,312 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:04:32,350 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:04:32,350 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:04:35,566 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:04:40,566 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:04:42,289 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:04:46,290 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:04:47,312 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:04:47,350 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:04:47,350 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:04:51,541 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:04:56,542 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:05:01,542 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:05:02,312 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:05:02,350 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:05:02,350 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:05:06,599 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:05:11,600 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:05:12,290 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:05:17,285 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:05:17,312 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:05:17,350 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:05:17,350 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:05:22,601 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:05:27,602 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:05:32,312 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:05:32,350 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:05:32,350 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:05:32,604 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:05:37,613 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:05:42,291 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:05:43,291 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:05:47,312 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:05:47,350 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:05:47,350 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:05:48,609 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:05:53,610 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:05:58,611 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:06:02,312 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:06:02,350 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:06:02,350 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:06:04,598 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:06:09,598 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:06:12,291 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:06:15,292 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:06:17,312 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:06:17,350 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:06:17,351 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:06:20,575 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:06:25,576 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:06:30,576 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:06:32,313 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:06:32,350 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:06:32,351 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:06:35,612 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:06:40,612 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:06:42,292 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:06:46,293 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:06:47,320 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:06:47,350 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:06:47,351 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:06:51,532 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:06:56,533 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:07:01,534 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:07:02,320 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:07:02,350 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:07:02,351 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:07:06,612 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:07:11,613 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:07:12,292 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:07:17,293 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:07:17,320 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:07:17,351 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:07:17,351 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:07:22,557 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:07:27,558 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:07:32,320 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:07:32,351 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:07:32,351 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:07:32,601 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:07:37,602 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:07:42,292 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:07:43,293 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:07:47,320 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:07:47,351 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:07:47,351 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:07:48,621 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:07:53,622 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:07:58,622 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:08:02,322 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:08:02,351 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:08:02,352 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:08:04,605 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:08:09,606 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:08:12,293 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:08:15,294 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:08:17,322 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:08:17,351 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:08:17,351 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:08:20,567 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:08:25,576 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:08:30,580 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:08:32,323 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:08:32,351 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:08:32,352 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:08:36,527 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:08:41,528 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:08:42,293 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:08:46,209 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 01:08:47,296 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:08:47,323 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:08:47,354 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:08:47,354 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:08:52,577 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:08:57,577 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:09:02,323 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:09:02,354 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:09:02,354 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:09:03,575 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:09:08,575 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:09:12,305 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:09:14,305 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:09:17,323 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:09:17,354 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:09:17,354 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:09:19,538 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:09:24,539 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:09:29,540 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:09:32,323 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:09:32,354 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:09:32,355 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:09:34,565 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:09:39,574 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:09:42,294 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:09:45,295 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:09:47,323 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:09:47,360 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:09:47,360 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:09:50,530 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:09:55,530 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:10:00,538 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:10:02,323 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:10:02,360 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:10:02,360 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:10:05,563 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:10:10,564 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:10:12,295 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:10:16,296 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:10:17,323 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:10:17,360 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:10:17,361 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:10:21,625 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:10:26,626 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:10:31,627 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:10:32,323 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:10:32,360 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:10:32,361 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:10:37,593 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:10:42,295 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:10:43,296 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:10:47,324 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:10:47,360 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:10:47,361 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:10:48,618 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:10:53,618 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:10:58,619 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:11:02,324 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:11:02,360 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:11:02,361 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:11:04,576 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:11:09,578 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:11:12,296 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:11:15,296 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:11:17,324 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:11:17,360 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:11:17,361 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:11:20,614 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:11:25,614 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:11:30,615 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:11:32,324 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:11:32,361 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:11:32,361 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:11:36,535 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:11:41,535 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:11:42,296 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:11:47,297 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:11:47,324 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:11:47,361 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:11:47,361 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:11:52,581 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:11:57,581 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:12:02,324 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:12:02,361 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:12:02,361 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:12:02,584 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:12:07,585 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:12:12,296 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:12:13,297 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:12:17,324 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:12:17,361 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:12:17,361 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:12:18,622 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:12:23,623 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:12:28,636 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:12:32,324 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:12:32,361 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:12:32,362 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:12:34,609 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:12:39,610 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:12:42,297 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:12:45,299 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:12:47,324 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:12:47,361 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:12:47,362 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:12:50,536 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:12:55,536 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:13:00,537 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:13:02,324 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:13:02,362 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:13:02,362 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:13:05,600 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:13:10,600 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:13:12,298 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:13:16,299 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:13:17,333 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:13:17,361 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:13:17,362 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:13:21,557 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:13:26,557 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:13:31,558 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:13:32,333 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:13:32,362 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:13:32,363 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:13:37,535 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:13:42,298 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:13:43,299 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:13:47,333 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:13:47,362 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:13:47,362 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:13:48,552 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:13:53,553 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:13:58,554 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:14:02,333 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:14:02,362 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:14:02,362 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:14:04,545 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:14:09,546 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:14:12,299 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:14:15,299 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:14:17,333 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:14:17,362 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:14:17,363 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:14:20,627 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:14:25,628 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:14:30,436 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 01:14:31,523 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:14:32,333 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:14:32,362 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:14:32,362 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:14:36,565 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:14:41,566 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:14:42,299 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:14:47,300 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:14:47,334 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:14:47,362 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:14:47,362 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:14:52,540 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:14:57,540 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:15:02,334 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:15:02,362 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:15:02,363 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:15:02,577 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:15:07,578 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:15:12,300 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:15:13,300 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:15:17,334 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:15:17,362 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:15:17,363 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:15:18,572 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:15:23,573 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:15:28,573 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:15:32,334 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:15:32,363 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:15:32,363 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:15:34,522 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:15:39,530 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:15:42,300 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:15:45,301 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:15:47,334 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:15:47,370 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:15:47,370 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:15:50,588 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:15:55,589 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:16:00,590 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:16:02,334 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:16:02,370 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:16:02,370 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:16:05,613 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:16:10,614 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:16:12,300 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:16:16,301 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:16:17,334 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:16:17,370 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:16:17,370 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:16:21,568 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:16:26,568 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:16:31,588 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:16:32,334 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:16:32,370 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:16:32,371 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:16:37,559 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:16:42,301 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:16:43,302 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:16:47,334 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:16:47,370 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:16:47,371 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:16:48,530 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:16:53,531 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:16:58,532 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:17:02,335 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:17:02,370 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:17:02,371 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:17:03,572 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:17:08,573 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:17:12,302 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:17:14,302 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:17:17,335 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:17:17,370 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:17:17,371 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:17:19,541 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:17:24,541 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:17:29,542 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:17:32,335 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:17:32,371 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:17:32,371 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:17:34,560 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:17:39,560 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:17:42,310 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:17:45,311 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:17:47,335 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:17:47,371 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:17:47,371 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:17:50,580 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:17:55,581 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:18:00,581 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:18:02,335 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:18:02,371 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:18:02,371 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:18:06,580 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:18:11,580 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:18:12,311 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:18:17,307 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:18:17,335 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:18:17,371 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:18:17,371 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:18:22,586 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:18:27,587 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:18:32,335 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:18:32,371 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:18:32,371 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:18:33,581 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:18:38,582 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:18:42,311 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:18:44,312 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:18:47,335 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:18:47,371 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:18:47,372 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:18:49,594 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:18:54,595 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:18:59,596 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:19:02,335 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:19:02,371 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:19:02,372 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:19:04,617 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:19:09,618 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:19:12,312 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:19:15,312 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:19:17,335 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:19:17,371 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:19:17,372 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:19:20,625 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:19:25,625 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:19:30,639 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:19:32,336 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:19:32,371 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:19:32,372 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:19:36,618 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:19:41,619 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:19:42,313 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:19:47,308 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:19:47,336 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:19:47,371 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:19:47,372 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:19:52,601 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:19:57,602 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:20:02,336 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:20:02,372 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:20:02,372 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:20:03,585 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:20:08,586 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:20:12,313 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:20:14,314 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:20:17,336 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:20:17,372 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:20:17,372 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:20:19,610 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:20:24,611 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:20:29,612 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:20:32,336 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:20:32,372 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:20:32,372 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:20:35,549 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:20:40,549 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:20:42,313 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:20:46,325 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:20:47,336 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:20:47,372 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:20:47,372 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:20:51,607 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:20:56,608 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:21:01,608 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:21:02,336 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:21:02,372 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:21:02,372 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:21:06,632 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:21:11,633 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:21:12,314 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:21:17,309 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:21:17,336 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:21:17,372 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:21:17,372 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:21:22,788 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:21:27,788 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:21:32,336 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:21:32,372 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:21:32,372 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:21:33,620 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:21:38,620 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:21:42,314 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:21:44,315 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:21:47,336 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:21:47,372 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:21:47,373 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:21:49,547 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:21:54,550 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:21:59,550 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:22:02,336 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:22:02,372 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:22:02,373 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:22:04,566 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:22:09,567 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:22:12,324 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:22:15,325 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:22:17,337 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:22:17,372 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:22:17,373 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:22:20,554 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:22:25,555 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:22:30,556 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:22:32,337 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:22:32,373 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:22:32,373 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:22:36,539 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:22:41,560 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:22:42,325 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:22:47,309 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:22:47,337 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:22:47,373 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:22:47,373 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:22:52,620 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:22:57,620 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:23:02,337 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:23:02,373 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:23:02,373 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:23:03,570 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:23:08,571 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:23:12,325 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:23:14,325 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:23:17,337 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:23:17,373 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:23:17,373 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:23:19,592 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:23:24,592 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:23:29,593 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:23:32,337 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:23:32,373 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:23:32,374 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:23:34,616 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:23:39,617 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:23:42,325 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:23:45,327 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:23:47,346 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:23:47,373 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:23:47,374 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:23:50,566 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:23:55,567 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:24:00,568 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:24:02,346 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:24:02,373 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:24:02,374 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:24:05,593 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:24:10,594 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:24:12,326 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:24:16,327 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:24:17,346 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:24:17,373 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:24:17,374 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:24:21,597 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:24:26,610 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:24:31,626 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:24:32,346 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:24:32,373 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:24:32,374 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:24:37,554 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:24:42,326 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:24:43,327 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:24:47,347 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:24:47,374 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:24:47,374 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:24:48,580 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:24:53,600 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:24:58,590 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:25:02,347 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:25:02,374 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:25:02,374 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:25:03,607 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:25:08,608 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:25:12,337 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:25:14,338 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:25:17,347 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:25:17,374 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:25:17,374 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:25:19,580 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:25:24,581 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:25:29,581 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:25:32,347 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:25:32,374 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:25:32,374 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:25:34,584 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:25:39,584 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:25:42,327 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:25:45,328 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:25:47,347 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:25:47,374 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:25:47,374 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:25:50,608 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:25:55,609 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:25:57,034 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 01:25:59,041 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 01:26:01,420 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:26:02,347 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:26:02,374 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:26:02,375 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:26:06,579 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:26:11,580 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:26:12,328 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:26:17,311 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:26:17,347 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:26:17,390 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:26:17,390 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:26:22,610 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:26:27,611 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:26:32,347 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:26:32,390 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:26:32,390 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:26:33,600 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:26:38,601 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:26:42,328 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:26:44,329 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:26:47,347 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:26:47,390 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:26:47,390 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:26:49,615 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:26:54,616 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:26:59,617 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:27:02,347 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:27:02,390 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:27:02,391 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:27:05,583 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:27:10,583 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:27:12,329 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:27:16,330 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:27:17,347 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:27:17,390 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:27:17,391 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:27:21,326 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 01:27:22,210 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:27:27,210 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:27:32,211 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:27:32,351 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:27:32,391 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:27:32,391 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:27:37,562 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:27:42,329 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:27:43,330 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:27:47,351 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:27:47,391 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:27:47,391 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:27:48,648 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:27:53,649 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:27:58,650 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:28:02,350 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:28:02,391 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:28:02,392 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:28:04,642 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:28:09,642 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:28:12,330 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:28:15,331 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:28:17,351 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:28:17,391 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:28:17,392 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:28:20,627 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:28:25,628 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:28:30,629 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:28:32,351 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:28:32,391 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:28:32,392 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:28:36,585 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:28:41,586 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:28:42,330 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:28:47,321 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:28:47,351 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:28:47,391 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:28:47,392 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:28:52,604 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:28:57,604 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:29:02,351 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:29:02,391 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:29:02,392 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:29:02,641 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:29:07,642 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:29:12,330 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:29:13,331 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:29:17,351 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:29:17,391 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:29:17,392 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:29:18,616 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:29:23,617 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:29:28,618 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:29:32,351 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:29:32,391 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:29:32,392 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:29:33,657 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:29:38,658 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:29:42,331 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:29:44,333 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:29:47,351 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:29:47,392 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:29:47,392 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:29:49,600 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:29:54,601 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:29:59,602 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:30:02,351 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:30:02,392 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:30:02,392 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:30:05,592 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:30:10,603 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:30:12,332 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:30:16,333 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:30:17,351 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:30:17,392 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:30:17,392 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:30:21,645 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:30:26,646 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:30:31,646 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:30:32,351 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:30:32,392 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:30:32,392 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:30:37,567 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:30:42,332 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:30:43,333 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:30:47,352 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:30:47,392 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:30:47,393 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:30:48,566 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:30:53,570 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:30:58,570 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:31:02,352 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:31:02,392 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:31:02,393 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:31:03,648 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:31:08,649 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:31:12,332 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:31:14,333 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:31:17,352 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:31:17,392 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:31:17,393 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:31:19,618 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:31:24,619 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:31:29,620 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:31:32,352 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:31:32,393 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:31:32,393 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:31:34,628 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:31:39,628 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:31:42,333 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:31:45,334 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:31:47,352 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:31:47,393 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:31:47,393 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:31:50,603 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:31:55,604 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:32:00,605 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:32:02,352 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:32:02,393 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:32:02,394 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:32:05,618 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:32:10,618 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:32:12,334 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:32:16,335 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:32:17,352 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:32:17,393 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:32:17,393 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:32:21,594 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:32:26,595 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:32:31,604 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:32:32,352 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:32:32,393 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:32:32,393 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:32:36,661 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:32:41,662 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:32:42,334 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:32:47,323 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:32:47,352 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:32:47,393 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:32:47,394 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:32:52,630 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:32:57,631 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:33:02,352 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:33:02,393 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:33:02,394 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:33:03,589 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:33:08,590 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:33:12,335 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:33:14,336 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:33:17,353 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:33:17,393 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:33:17,394 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:33:19,573 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:33:24,580 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:33:29,580 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:33:32,353 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:33:32,393 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:33:32,394 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:33:34,623 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:33:39,623 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:33:42,335 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:33:45,336 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:33:47,353 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:33:47,394 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:33:47,394 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:33:50,614 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:33:55,620 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:34:00,620 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:34:02,353 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:34:02,393 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:34:02,394 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:34:05,665 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:34:10,669 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:34:12,336 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:34:16,337 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:34:17,353 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:34:17,394 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:34:17,394 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:34:21,561 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:34:26,562 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:34:31,563 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:34:32,353 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:34:32,394 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:34:32,394 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:34:36,624 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:34:41,624 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:34:42,336 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:34:47,324 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:34:47,353 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:34:47,394 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:34:47,394 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:34:52,647 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:34:57,648 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:35:02,353 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:35:02,394 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:35:02,394 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:35:03,617 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:35:08,618 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:35:12,336 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:35:14,337 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:35:17,355 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:35:17,394 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:35:17,395 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:35:19,580 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:35:24,581 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:35:29,582 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:35:32,355 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:35:32,400 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:35:32,400 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:35:34,631 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:35:39,632 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:35:42,337 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:35:45,339 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:35:47,355 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:35:47,400 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:35:47,401 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:35:50,582 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:35:55,583 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:36:00,583 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:36:02,355 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:36:02,400 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:36:02,400 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:36:05,593 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:36:10,597 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:36:12,338 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:36:16,339 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:36:17,355 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:36:17,400 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:36:17,400 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:36:21,646 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:36:26,646 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:36:31,647 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:36:32,355 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:36:32,400 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:36:32,401 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:36:37,607 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:36:42,338 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:36:43,339 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:36:47,355 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:36:47,401 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:36:47,401 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:36:48,647 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:36:53,648 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:36:58,648 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:37:02,355 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:37:02,401 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:37:02,402 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:37:04,565 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:37:09,566 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:37:12,339 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:37:15,339 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:37:17,355 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:37:17,401 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:37:17,402 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:37:20,648 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:37:25,649 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:37:30,650 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:37:32,355 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:37:32,401 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:37:32,402 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:37:36,644 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:37:41,645 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:37:42,339 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:37:47,334 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:37:47,356 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:37:47,401 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:37:47,402 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:37:52,656 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:37:57,657 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:38:02,356 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:38:02,401 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:38:02,402 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:38:02,663 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:38:07,664 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:38:12,339 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:38:13,340 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:38:17,356 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:38:17,402 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:38:17,402 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:38:18,616 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:38:23,617 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:38:28,617 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:38:32,360 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:38:32,402 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:38:32,402 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:38:33,664 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:38:38,665 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:38:42,340 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:38:44,341 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:38:47,360 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:38:47,402 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:38:47,402 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:38:49,622 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:38:54,623 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:38:59,624 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:39:02,360 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:39:02,402 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:39:02,403 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:39:05,577 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:39:10,578 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:39:12,340 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:39:16,341 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:39:17,360 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:39:17,402 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:39:17,402 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:39:21,568 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:39:26,569 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:39:31,570 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:39:32,360 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:39:32,402 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:39:32,403 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:39:36,585 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:39:41,586 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:39:42,341 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:39:47,335 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:39:47,360 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:39:47,402 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:39:47,403 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:39:52,636 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:39:57,644 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:40:02,360 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:40:02,402 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:40:02,403 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:40:03,583 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:40:09,333 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:40:10,072 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 01:40:12,341 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:40:14,342 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:40:17,361 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:40:17,402 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:40:17,403 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:40:19,652 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:40:24,653 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:40:29,654 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:40:32,361 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:40:32,402 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:40:32,403 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:40:35,599 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:40:40,599 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:40:42,342 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:40:46,343 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:40:47,361 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:40:47,403 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:40:47,404 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:40:51,603 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:40:56,604 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:41:01,610 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:41:02,361 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:41:02,403 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:41:02,404 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:41:06,619 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:41:11,620 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:41:12,342 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:41:17,336 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:41:17,361 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:41:17,403 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:41:17,403 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:41:22,659 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:41:27,660 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:41:32,361 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:41:32,410 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:41:32,411 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:41:33,600 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:41:38,601 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:41:42,343 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:41:44,350 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:41:47,361 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:41:47,410 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:41:47,411 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:41:49,634 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:41:54,635 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:41:59,636 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:42:02,361 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:42:02,410 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:42:02,411 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:42:05,629 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:42:10,630 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:42:12,343 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:42:16,344 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:42:17,361 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:42:17,410 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:42:17,411 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:42:21,727 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:42:26,732 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:42:31,733 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:42:32,362 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:42:32,411 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:42:32,411 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:42:37,567 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:42:42,344 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:42:43,344 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:42:47,362 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:42:47,411 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:42:47,411 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:42:48,661 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:42:53,662 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:42:58,662 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:43:02,362 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:43:02,411 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:43:02,411 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:43:04,644 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:43:09,658 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:43:12,344 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:43:15,345 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:43:17,362 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:43:17,411 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:43:17,411 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:43:19,840 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 01:43:20,439 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:43:21,845 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 01:43:25,440 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:43:30,441 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:43:32,370 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:43:32,411 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:43:32,411 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:43:35,613 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:43:40,620 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:43:42,345 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:43:46,350 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:43:47,370 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:43:47,411 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:43:47,412 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:43:51,649 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:43:56,649 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:44:01,660 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:44:02,370 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:44:02,411 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:44:02,412 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:44:07,630 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:44:12,345 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:44:13,346 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:44:17,370 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:44:17,411 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:44:17,412 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:44:18,620 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:44:23,621 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:44:28,622 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:44:32,370 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:44:32,411 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:44:32,412 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:44:33,680 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:44:38,681 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:44:42,360 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:44:44,361 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:44:47,370 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:44:47,411 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:44:47,412 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:44:49,636 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:44:54,636 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:44:59,637 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:45:02,370 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:45:02,412 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:45:02,412 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:45:05,616 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:45:10,617 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:45:12,361 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:45:16,362 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:45:17,380 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:45:17,412 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:45:17,412 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:45:21,625 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:45:26,626 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:45:31,630 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:45:32,380 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:45:32,420 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:45:32,420 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:45:36,657 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:45:41,660 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:45:42,361 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:45:47,339 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:45:47,380 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:45:47,420 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:45:47,420 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:45:52,668 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:45:57,668 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:46:02,380 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:46:02,420 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:46:02,421 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:46:03,643 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:46:08,644 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:46:12,362 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:46:14,371 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:46:17,380 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:46:17,420 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:46:17,420 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:46:19,600 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:46:24,600 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:46:29,601 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:46:32,380 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:46:32,420 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:46:32,421 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:46:35,596 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:46:40,597 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:46:42,362 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:46:46,363 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:46:47,380 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:46:47,420 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:46:47,421 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:46:51,637 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:46:56,638 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:47:01,639 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:47:02,387 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:47:02,420 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:47:02,421 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:47:06,644 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:47:11,644 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:47:12,362 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:47:17,339 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:47:17,387 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:47:17,420 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:47:17,421 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:47:22,657 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:47:27,658 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:47:32,395 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:47:32,421 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:47:32,421 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:47:33,577 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:47:38,580 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:47:42,363 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:47:44,364 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:47:47,395 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:47:47,421 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:47:47,421 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:47:49,625 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:47:54,626 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:47:59,627 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:48:02,395 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:48:02,421 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:48:02,422 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:48:04,670 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:48:09,671 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:48:12,363 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:48:15,364 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:48:17,395 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:48:17,421 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:48:17,421 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:48:20,596 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:48:25,597 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:48:30,598 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:48:32,396 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:48:32,421 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:48:32,421 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:48:35,613 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:48:40,613 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:48:42,363 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:48:46,364 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:48:47,396 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:48:47,421 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:48:47,422 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:48:51,616 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:48:56,617 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:49:01,618 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:49:02,396 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:49:02,421 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:49:02,422 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:49:06,677 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:49:11,678 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:49:12,370 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:49:17,340 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:49:17,396 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:49:17,421 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:49:17,422 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:49:22,681 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:49:27,690 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:49:32,396 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:49:32,421 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:49:32,422 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:49:33,587 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:49:38,605 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:49:42,371 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:49:44,372 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:49:47,396 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:49:47,422 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:49:47,422 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:49:49,609 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:49:54,610 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:49:59,610 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:50:02,396 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:50:02,422 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:50:02,422 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:50:04,655 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:50:09,656 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:50:12,371 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:50:15,372 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:50:17,396 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:50:17,422 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:50:17,422 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:50:20,597 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:50:25,598 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:50:30,598 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:50:32,396 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:50:32,422 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:50:32,423 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:50:35,614 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:50:40,615 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:50:42,372 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:50:46,373 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:50:47,397 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:50:47,422 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:50:47,423 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:50:51,592 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:50:56,592 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:51:01,600 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:51:02,397 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:51:02,422 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:51:02,423 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:51:06,672 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:51:11,679 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:51:12,372 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:51:17,348 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:51:17,397 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:51:17,430 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:51:17,430 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:51:22,614 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:51:27,615 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:51:32,397 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:51:32,440 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:51:32,440 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:51:33,610 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:51:38,611 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:51:42,373 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:51:44,374 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:51:47,397 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:51:47,448 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:51:47,449 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:51:49,667 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:51:54,668 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:51:59,669 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:52:02,397 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:52:02,449 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:52:02,449 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:52:05,647 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:52:10,648 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:52:12,373 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:52:16,374 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:52:17,397 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:52:17,449 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:52:17,449 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:52:21,704 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:52:26,710 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:52:31,710 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:52:32,397 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:52:32,449 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:52:32,449 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:52:37,630 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:52:42,374 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:52:43,374 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:52:47,398 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:52:47,449 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:52:47,449 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:52:48,635 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:52:53,636 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:52:58,637 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:53:02,397 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:53:02,449 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:53:02,449 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:53:03,674 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:53:08,675 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:53:12,374 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:53:14,377 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:53:14,854 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 01:53:17,398 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:53:17,449 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:53:17,449 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:53:19,612 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:53:24,613 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:53:29,613 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:53:32,398 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:53:32,457 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:53:32,457 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:53:34,623 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:53:39,624 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:53:42,375 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:53:45,375 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:53:47,398 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:53:47,457 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:53:47,457 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:53:50,664 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:53:55,665 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:54:00,665 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:54:02,398 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:54:02,457 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:54:02,457 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:54:05,681 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:54:10,682 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:54:12,375 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:54:16,376 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:54:17,398 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:54:17,457 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:54:17,458 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:54:21,613 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:54:26,614 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:54:31,614 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:54:32,398 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:54:32,457 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:54:32,458 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:54:36,648 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:54:41,649 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:54:42,406 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:54:47,349 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:54:47,398 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:54:47,467 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:54:47,468 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:54:52,707 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:54:57,707 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:55:02,398 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:55:02,467 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:55:02,468 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:55:03,661 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:55:08,662 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:55:12,376 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:55:14,377 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:55:17,398 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:55:17,467 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:55:17,468 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:55:19,661 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:55:24,662 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:55:29,662 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:55:32,398 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:55:32,468 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:55:32,468 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:55:35,628 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:55:40,629 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:55:42,376 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:55:46,377 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:55:47,399 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:55:47,467 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:55:47,468 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:55:51,641 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:55:56,641 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:56:01,642 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:56:02,399 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:56:02,468 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:56:02,468 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:56:06,718 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:56:11,718 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:56:12,377 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:56:17,350 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:56:17,399 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:56:17,468 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:56:17,468 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:56:22,678 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:56:27,686 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:56:32,399 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:56:32,468 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:56:32,468 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:56:33,651 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:56:38,652 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:56:42,378 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:56:44,378 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:56:47,399 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:56:47,468 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:56:47,468 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:56:49,671 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:56:54,672 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:56:59,675 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:57:02,399 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:57:02,468 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:57:02,468 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:57:05,641 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:57:10,642 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:57:12,378 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:57:16,379 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:57:17,399 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:57:17,468 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:57:17,469 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:57:21,722 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:57:26,722 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:57:31,723 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:57:32,399 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:57:32,468 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:57:32,471 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:57:37,650 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:57:42,378 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:57:43,379 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:57:47,400 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:57:47,468 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:57:47,469 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:57:48,674 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:57:53,675 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:57:58,676 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:58:02,400 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:58:02,468 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:58:02,469 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:58:03,715 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:58:08,715 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:58:12,379 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:58:14,380 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:58:17,400 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:58:17,469 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:58:17,469 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:58:19,707 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:58:24,707 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:58:29,708 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:58:32,400 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:58:32,469 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:58:32,469 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:58:34,712 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:58:39,713 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:58:42,379 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:58:45,380 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:58:47,400 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:58:47,469 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:58:47,469 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:58:50,704 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:58:55,704 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:59:00,705 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:59:02,400 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:59:02,469 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:59:02,469 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:59:06,689 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:59:11,689 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:59:12,380 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:59:17,351 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:59:17,400 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:59:17,469 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:59:17,469 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:59:22,678 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:59:27,679 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:59:31,229 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 01:59:32,400 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:59:32,469 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:59:32,470 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:59:32,709 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:59:37,709 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:59:42,380 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 01:59:43,381 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:59:47,400 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 01:59:47,469 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 01:59:47,470 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 01:59:48,671 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:59:53,671 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 01:59:58,672 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:00:02,400 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:00:02,469 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:00:02,470 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:00:03,730 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:00:08,736 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:00:12,380 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:00:14,381 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:00:17,401 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:00:17,470 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:00:17,471 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:00:19,698 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:00:24,699 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:00:29,700 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:00:32,401 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:00:32,470 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:00:32,471 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:00:35,655 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:00:40,656 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:00:42,381 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:00:46,384 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:00:47,401 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:00:47,470 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:00:47,470 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:00:51,653 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:00:56,653 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:01:01,654 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:01:02,401 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:01:02,470 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:01:02,470 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:01:06,657 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:01:11,657 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:01:12,381 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:01:17,352 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:01:17,401 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:01:17,470 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:01:17,471 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:01:22,700 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:01:27,701 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:01:32,401 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:01:32,470 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:01:32,471 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:01:32,709 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:01:37,709 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:01:42,382 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:01:43,382 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:01:47,401 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:01:47,470 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:01:47,471 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:01:48,637 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:01:53,638 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:01:58,646 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:02:02,401 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:02:02,470 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:02:02,471 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:02:03,685 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:02:08,686 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:02:12,382 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:02:14,383 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:02:17,401 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:02:17,470 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:02:17,471 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:02:19,706 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:02:24,707 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:02:29,708 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:02:32,401 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:02:32,471 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:02:32,471 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:02:34,711 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:02:39,712 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:02:42,383 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:02:45,384 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:02:47,402 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:02:47,471 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:02:47,471 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:02:50,704 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:02:55,705 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:03:00,705 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:03:02,402 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:03:02,471 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:03:02,471 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:03:06,690 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:03:11,690 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:03:12,383 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:03:17,353 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:03:17,402 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:03:17,471 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:03:17,471 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:03:22,653 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:03:27,654 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:03:32,402 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:03:32,471 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:03:32,472 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:03:32,723 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:03:37,724 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:03:42,384 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:03:43,385 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:03:47,402 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:03:47,471 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:03:47,472 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:03:48,669 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:03:53,669 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:03:58,670 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:04:02,402 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:04:02,471 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:04:02,471 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:04:03,728 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:04:08,729 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:04:12,384 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:04:14,385 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:04:17,402 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:04:17,473 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:04:17,473 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:04:19,697 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:04:24,703 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:04:29,704 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:04:32,402 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:04:32,473 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:04:32,473 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:04:35,685 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:04:40,686 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:04:42,385 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:04:46,385 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:04:47,403 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:04:47,473 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:04:47,473 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:04:51,731 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:04:56,732 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:05:01,733 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:05:02,403 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:05:02,473 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:05:02,474 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:05:07,727 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:05:12,385 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:05:13,386 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:05:17,403 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:05:17,473 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:05:17,474 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:05:18,665 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:05:23,666 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:05:28,666 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:05:32,403 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:05:32,473 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:05:32,474 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:05:34,648 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:05:39,648 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:05:42,386 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:05:45,387 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:05:47,403 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:05:47,473 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:05:47,474 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:05:50,679 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:05:55,680 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:06:00,681 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:06:02,403 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:06:02,473 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:06:02,474 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:06:05,690 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:06:10,691 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:06:12,386 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:06:16,387 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:06:17,403 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:06:17,474 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:06:17,474 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:06:17,698 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 02:06:21,670 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:06:26,671 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:06:31,672 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:06:32,403 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:06:32,474 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:06:32,474 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:06:36,726 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:06:41,734 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:06:42,387 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:06:47,363 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:06:47,403 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:06:47,474 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:06:47,475 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:06:52,736 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:06:57,736 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:07:02,403 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:07:02,474 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:07:02,475 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:07:03,702 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:07:08,710 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:07:12,388 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:07:14,388 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:07:17,410 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:07:17,474 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:07:17,475 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:07:19,682 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:07:24,684 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:07:29,684 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:07:32,410 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:07:32,474 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:07:32,475 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:07:34,720 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:07:39,721 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:07:42,388 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:07:45,389 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:07:47,410 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:07:47,474 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:07:47,475 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:07:50,733 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:07:55,734 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:08:00,735 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:08:02,410 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:08:02,474 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:08:02,475 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:08:06,712 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:08:11,712 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:08:12,389 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:08:17,363 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:08:17,410 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:08:17,474 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:08:17,475 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:08:22,690 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:08:27,691 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:08:32,410 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:08:32,475 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:08:32,475 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:08:33,633 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:08:38,634 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:08:42,389 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:08:44,390 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:08:47,415 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:08:47,475 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:08:47,475 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:08:49,656 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:08:54,657 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:08:59,657 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:09:02,415 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:09:02,475 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:09:02,476 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:09:05,645 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:09:10,645 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:09:12,389 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:09:16,391 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:09:17,416 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:09:17,475 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:09:17,475 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:09:21,668 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:09:26,669 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:09:31,670 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:09:32,416 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:09:32,475 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:09:32,475 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:09:36,730 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:09:41,730 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:09:42,390 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:09:47,364 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:09:47,416 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:09:47,475 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:09:47,476 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:09:52,724 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:09:57,725 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:10:02,416 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:10:02,475 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:10:02,476 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:10:03,705 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:10:08,706 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:10:12,401 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:10:14,401 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:10:17,416 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:10:17,475 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:10:17,476 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:10:19,676 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:10:24,676 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:10:29,677 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:10:32,416 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:10:32,476 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:10:32,476 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:10:35,661 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:10:40,662 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:10:42,400 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:10:46,400 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:10:47,416 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:10:47,476 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:10:47,476 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:10:51,714 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:10:56,715 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:11:01,715 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:11:02,416 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:11:02,476 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:11:02,476 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:11:07,654 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:11:12,400 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:11:13,401 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:11:17,416 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:11:17,476 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:11:17,476 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:11:18,674 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:11:23,674 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:11:28,675 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:11:32,416 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:11:32,476 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:11:32,476 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:11:34,647 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:11:39,647 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:11:42,401 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:11:45,402 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:11:47,417 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:11:47,476 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:11:47,477 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:11:50,673 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:11:55,673 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:12:00,674 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:12:02,419 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:12:02,476 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:12:02,477 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:12:06,670 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:12:11,678 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:12:12,401 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:12:17,394 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:12:17,420 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:12:17,486 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:12:17,487 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:12:22,659 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:12:27,659 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:12:32,420 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:12:32,486 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:12:32,487 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:12:32,662 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:12:37,662 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:12:42,402 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:12:43,402 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:12:47,420 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:12:47,487 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:12:47,487 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:12:48,726 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:12:53,727 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:12:58,728 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:13:02,420 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:13:02,487 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:13:02,488 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:13:04,710 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:13:09,710 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:13:12,402 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:13:15,403 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:13:17,420 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:13:17,488 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:13:17,488 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:13:20,694 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:13:25,694 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:13:30,695 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:13:32,420 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:13:32,488 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:13:32,488 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:13:36,666 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:13:41,667 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:13:42,403 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:13:47,384 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:13:47,420 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:13:47,490 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:13:47,490 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:13:52,710 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:13:57,710 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:14:02,420 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:14:02,490 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:14:02,490 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:14:03,672 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:14:08,673 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:14:12,403 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:14:14,404 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:14:17,420 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:14:17,490 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:14:17,490 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:14:19,761 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:14:24,762 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:14:29,762 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:14:32,420 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:14:32,490 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:14:32,491 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:14:35,755 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:14:40,756 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:14:42,404 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:14:46,404 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:14:47,420 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:14:47,490 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:14:47,491 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:14:51,722 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:14:56,722 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:15:01,723 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:15:02,421 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:15:02,490 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:15:02,491 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:15:06,743 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:15:11,744 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:15:12,404 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:15:17,387 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:15:17,430 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:15:17,500 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:15:17,501 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:15:22,710 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:15:27,712 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:15:32,430 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:15:32,500 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:15:32,501 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:15:32,735 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:15:37,735 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:15:42,405 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:15:43,406 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:15:47,430 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:15:47,500 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:15:47,501 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:15:48,760 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:15:53,761 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:15:58,762 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:16:02,430 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:16:02,501 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:16:02,501 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:16:04,684 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:16:09,684 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:16:11,789 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 02:16:12,405 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:16:13,794 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 02:16:15,406 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:16:17,430 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:16:17,501 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:16:17,501 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:16:20,728 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:16:25,729 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:16:30,730 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:16:32,430 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:16:32,501 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:16:32,501 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:16:35,750 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:16:40,751 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:16:42,406 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:16:46,415 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:16:47,430 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:16:47,501 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:16:47,501 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:16:51,759 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:16:56,760 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:17:01,761 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:17:02,430 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:17:02,501 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:17:02,502 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:17:07,757 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:17:12,407 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:17:13,407 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:17:17,431 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:17:17,501 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:17:17,502 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:17:18,717 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:17:23,718 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:17:28,719 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:17:32,431 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:17:32,501 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:17:32,502 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:17:34,674 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:17:39,674 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:17:42,407 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:17:45,408 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:17:47,431 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:17:47,501 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:17:47,502 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:17:50,669 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:17:55,670 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:18:00,680 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:18:02,431 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:18:02,501 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:18:02,502 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:18:05,743 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:18:10,744 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:18:12,407 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:18:16,408 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:18:17,431 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:18:17,502 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:18:17,502 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:18:21,703 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:18:26,709 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:18:31,710 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:18:32,431 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:18:32,502 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:18:32,502 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:18:37,669 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:18:42,408 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:18:43,408 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:18:47,431 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:18:47,502 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:18:47,510 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:18:48,736 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:18:53,750 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:18:58,750 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:19:02,431 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:19:02,502 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:19:02,503 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:19:04,744 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:19:09,744 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:19:12,408 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:19:15,409 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:19:17,431 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:19:17,502 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:19:17,502 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:19:18,552 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 02:19:20,669 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:19:25,673 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:19:30,675 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:19:32,432 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:19:32,502 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:19:32,502 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:19:35,756 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:19:40,756 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:19:42,409 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:19:46,410 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:19:47,432 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:19:47,502 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:19:47,503 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:19:51,707 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:19:56,710 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:20:01,710 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:20:02,432 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:20:02,502 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:20:02,503 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:20:06,767 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:20:11,768 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:20:12,409 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:20:17,391 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:20:17,432 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:20:17,502 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:20:17,503 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:20:22,689 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:20:27,690 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:20:32,432 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:20:32,502 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:20:32,503 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:20:32,767 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:20:37,768 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:20:42,410 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:20:43,411 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:20:47,432 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:20:47,509 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:20:47,510 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:20:48,685 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:20:53,686 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:20:58,686 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:21:02,432 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:21:02,510 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:21:02,510 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:21:03,717 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:21:08,717 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:21:12,410 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:21:14,411 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:21:17,432 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:21:17,510 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:21:17,510 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:21:19,699 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:21:24,700 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:21:29,701 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:21:32,432 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:21:32,510 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:21:32,511 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:21:34,743 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:21:39,743 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:21:42,413 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:21:45,414 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:21:47,432 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:21:47,510 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:21:47,510 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:21:50,719 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:21:55,720 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:22:00,720 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:22:02,440 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:22:02,510 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:22:02,510 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:22:06,709 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:22:11,710 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:22:12,412 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:22:17,392 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:22:17,440 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:22:17,510 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:22:17,511 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:22:22,704 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:22:27,705 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:22:32,440 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:22:32,510 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:22:32,511 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:22:32,722 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:22:37,723 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:22:42,412 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:22:43,413 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:22:47,440 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:22:47,511 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:22:47,511 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:22:48,661 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:22:53,671 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:22:58,671 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:23:02,440 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:23:02,519 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:23:02,519 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:23:03,772 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:23:08,773 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:23:12,413 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:23:14,413 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:23:17,440 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:23:17,519 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:23:17,519 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:23:19,770 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:23:24,770 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:23:29,771 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:23:32,440 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:23:32,519 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:23:32,519 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:23:35,743 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:23:40,744 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:23:42,413 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:23:46,420 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:23:47,441 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:23:47,519 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:23:47,520 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:23:51,751 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:23:56,751 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:24:01,752 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:24:02,440 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:24:02,519 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:24:02,520 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:24:06,766 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:24:11,767 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:24:12,415 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:24:17,401 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:24:17,441 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:24:17,519 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:24:17,520 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:24:22,743 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:24:27,743 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:24:32,441 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:24:32,524 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:24:32,525 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:24:33,693 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:24:38,700 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:24:42,415 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:24:44,416 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:24:47,441 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:24:47,524 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:24:47,525 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:24:49,726 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:24:54,726 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:24:59,727 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:25:02,441 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:25:02,524 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:25:02,525 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:25:04,771 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:25:09,772 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:25:12,427 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:25:15,428 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:25:17,441 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:25:17,524 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:25:17,531 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:25:20,776 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:25:25,776 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:25:30,777 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:25:32,442 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:25:32,525 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:25:32,525 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:25:36,762 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:25:41,768 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:25:42,416 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:25:47,402 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:25:47,441 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:25:47,525 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:25:47,525 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:25:52,692 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:25:57,693 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:26:02,442 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:26:02,525 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:26:02,525 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:26:02,704 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:26:07,710 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:26:12,417 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:26:13,417 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:26:17,442 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:26:17,525 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:26:17,525 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:26:18,732 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:26:23,733 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:26:28,740 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:26:32,442 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:26:32,525 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:26:32,525 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:26:34,699 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:26:39,700 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:26:42,417 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:26:45,418 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:26:47,442 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:26:47,525 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:26:47,526 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:26:50,768 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:26:55,769 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:27:00,769 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:27:02,442 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:27:02,525 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:27:02,526 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:27:06,729 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:27:11,730 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:27:12,418 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:27:17,403 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:27:17,442 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:27:17,525 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:27:17,526 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:27:22,766 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:27:27,767 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:27:32,442 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:27:32,525 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:27:32,526 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:27:32,776 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:27:37,777 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:27:42,418 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:27:43,419 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:27:47,442 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:27:47,526 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:27:47,526 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:27:48,702 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:27:53,702 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:27:58,703 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:28:02,442 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:28:02,526 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:28:02,526 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:28:03,719 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:28:08,720 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:28:12,418 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:28:14,419 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:28:17,442 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:28:17,530 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:28:17,530 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:28:19,766 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:28:24,766 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:28:29,800 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:28:32,443 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:28:32,530 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:28:32,531 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:28:35,712 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:28:40,712 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:28:42,419 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:28:46,420 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:28:47,443 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:28:47,530 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:28:47,530 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:28:51,789 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:28:56,790 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:29:01,791 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:29:02,443 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:29:02,530 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:29:02,531 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:29:07,761 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:29:12,420 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:29:13,421 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:29:17,443 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:29:17,530 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:29:17,531 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:29:18,742 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:29:23,742 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:29:28,743 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:29:32,443 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:29:32,530 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:29:32,531 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:29:34,697 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:29:39,698 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:29:42,420 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:29:45,421 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:29:47,443 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:29:47,531 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:29:47,531 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:29:50,782 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:29:55,783 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:30:00,784 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:30:02,443 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:30:02,531 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:30:02,531 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:30:05,784 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:30:10,785 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:30:12,420 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:30:16,422 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:30:17,443 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:30:17,531 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:30:17,532 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:30:21,778 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:30:26,779 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:30:31,780 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:30:32,443 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:30:32,531 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:30:32,531 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:30:37,717 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:30:42,421 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:30:43,422 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:30:47,443 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:30:47,531 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:30:47,532 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:30:48,718 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:30:53,718 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:30:58,719 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:31:02,444 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:31:02,531 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:31:02,532 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:31:04,694 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:31:09,695 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:31:12,421 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:31:15,422 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:31:17,444 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:31:17,531 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:31:17,532 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:31:20,741 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:31:25,742 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:31:30,759 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:31:32,444 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:31:32,531 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:31:32,532 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:31:36,723 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:31:41,723 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:31:42,422 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:31:47,404 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:31:47,444 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:31:47,531 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:31:47,532 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:31:52,736 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:31:57,750 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:32:02,444 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:32:02,531 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:32:02,532 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:32:02,757 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:32:07,757 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:32:12,422 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:32:13,423 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:32:17,445 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:32:17,535 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:32:17,536 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:32:18,713 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:32:23,713 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:32:28,720 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:32:32,445 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:32:32,535 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:32:32,536 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:32:33,466 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 02:32:33,776 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:32:38,777 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:32:42,423 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:32:44,424 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:32:47,445 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:32:47,536 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:32:47,536 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:32:49,791 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:32:54,792 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:32:59,792 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:33:02,445 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:33:02,536 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:33:02,536 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:33:05,743 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:33:10,744 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:33:12,423 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:33:16,424 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:33:17,445 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:33:17,536 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:33:17,536 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:33:21,757 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:33:26,758 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:33:31,758 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:33:32,458 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:33:32,536 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:33:32,536 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:33:36,787 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:33:41,788 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:33:42,424 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:33:47,405 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:33:47,453 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:33:47,536 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:33:47,536 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:33:52,720 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:33:57,720 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:34:02,453 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:34:02,536 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:34:02,537 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:34:03,691 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:34:07,802 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 02:34:08,736 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:34:12,424 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:34:14,425 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:34:17,453 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:34:17,536 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:34:17,537 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:34:19,696 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:34:24,696 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:34:29,697 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:34:32,454 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:34:32,536 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:34:32,537 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:34:34,759 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:34:39,760 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:34:42,425 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:34:45,426 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:34:47,454 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:34:47,537 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:34:47,537 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:34:50,742 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:34:55,743 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:35:00,744 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:35:02,454 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:35:02,537 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:35:02,540 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:35:05,761 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:35:10,761 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:35:12,425 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:35:16,427 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:35:17,454 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:35:17,537 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:35:17,537 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:35:21,702 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:35:26,710 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:35:31,710 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:35:32,454 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:35:32,537 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:35:32,538 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:35:36,736 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:35:41,737 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:35:42,426 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:35:47,406 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:35:47,454 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:35:47,537 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:35:47,538 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:35:52,755 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:35:57,756 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:36:02,454 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:36:02,537 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:36:02,538 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:36:03,725 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:36:08,725 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:36:12,430 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:36:14,431 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:36:17,454 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:36:17,537 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:36:17,538 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:36:19,727 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:36:24,728 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:36:29,729 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:36:32,454 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:36:32,537 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:36:32,538 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:36:34,748 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:36:39,748 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:36:42,430 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:36:45,431 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:36:47,454 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:36:47,537 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:36:47,538 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:36:50,745 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:36:55,746 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:37:00,747 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:37:02,454 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:37:02,538 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:37:02,538 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:37:06,747 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:37:11,747 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:37:12,431 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:37:17,407 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:37:17,455 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:37:17,538 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:37:17,539 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:37:22,808 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:37:27,808 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:37:32,455 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:37:32,539 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:37:32,540 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:37:33,718 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:37:38,719 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:37:42,431 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:37:44,432 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:37:47,455 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:37:47,539 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:37:47,540 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:37:49,761 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:37:54,762 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:37:59,763 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:38:02,455 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:38:02,540 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:38:02,540 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:38:04,764 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:38:09,765 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:38:12,432 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:38:15,433 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:38:17,455 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:38:17,540 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:38:17,540 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:38:20,786 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:38:25,786 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:38:30,787 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:38:32,456 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:38:32,540 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:38:32,541 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:38:36,705 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:38:41,706 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:38:42,433 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:38:47,407 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:38:47,455 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:38:47,540 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:38:47,540 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:38:52,722 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:38:57,722 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:39:02,456 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:39:02,544 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:39:02,544 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:39:02,802 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:39:07,803 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:39:12,433 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:39:13,434 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:39:17,460 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:39:17,544 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:39:17,544 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:39:18,785 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:39:23,786 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:39:28,786 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:39:32,460 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:39:32,544 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:39:32,544 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:39:34,779 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:39:39,780 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:39:42,433 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:39:45,434 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:39:47,460 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:39:47,544 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:39:47,544 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:39:50,727 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:39:55,740 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:40:00,740 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:40:02,460 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:40:02,544 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:40:02,545 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:40:06,717 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:40:11,717 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:40:12,436 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:40:17,418 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:40:17,460 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:40:17,544 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:40:17,545 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:40:22,848 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:40:27,849 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:40:32,460 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:40:32,544 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:40:32,545 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:40:33,817 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:40:38,825 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:40:42,435 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:40:44,435 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:40:47,460 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:40:47,544 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:40:47,545 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:40:49,797 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:40:54,798 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:40:59,799 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:41:02,460 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:41:02,550 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:41:02,550 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:41:05,775 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:41:10,780 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:41:12,435 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:41:16,437 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:41:17,461 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:41:17,550 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:41:17,550 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:41:21,809 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:41:26,810 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:41:31,811 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:41:32,461 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:41:32,550 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:41:32,551 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:41:37,725 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:41:42,435 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:41:43,436 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:41:47,461 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:41:47,550 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:41:47,550 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:41:48,804 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:41:53,805 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:41:58,806 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:42:02,461 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:42:02,550 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:42:02,551 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:42:04,758 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:42:09,759 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:42:12,436 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:42:15,437 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:42:17,461 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:42:17,550 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:42:17,551 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:42:20,712 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:42:25,730 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:42:30,737 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:42:32,461 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:42:32,550 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:42:32,551 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:42:35,760 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:42:40,761 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:42:42,437 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:42:46,438 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:42:47,462 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:42:47,551 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:42:47,551 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:42:51,764 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:42:56,765 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:43:01,765 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:43:02,462 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:43:02,551 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:43:02,551 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:43:07,729 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:43:12,437 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:43:13,438 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:43:17,462 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:43:17,551 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:43:17,551 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:43:18,730 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:43:23,730 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:43:28,731 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:43:32,463 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:43:32,551 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:43:32,551 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:43:33,827 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:43:38,827 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:43:42,438 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:43:44,439 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:43:47,463 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:43:47,551 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:43:47,551 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:43:49,800 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:43:54,801 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:43:59,801 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:44:02,463 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:44:02,551 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:44:02,551 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:44:05,765 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:44:10,765 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:44:12,438 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:44:16,439 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:44:17,463 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:44:17,551 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:44:17,552 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:44:21,794 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:44:26,800 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:44:31,800 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:44:32,463 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:44:32,551 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:44:32,552 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:44:37,794 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:44:42,439 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:44:43,439 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:44:47,463 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:44:47,551 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:44:47,552 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:44:48,781 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:44:53,782 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:44:58,783 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:45:02,120 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 02:45:02,463 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:45:02,551 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:45:02,552 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:45:03,804 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:45:08,805 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:45:12,439 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:45:14,440 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:45:17,463 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:45:17,551 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:45:17,552 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:45:19,774 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:45:24,774 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:45:29,775 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:45:32,463 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:45:32,551 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:45:32,552 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:45:35,751 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:45:40,754 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:45:42,440 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:45:46,441 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:45:47,463 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:45:47,552 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:45:47,552 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:45:51,796 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:45:56,797 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:46:01,797 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:46:02,464 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:46:02,552 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:46:02,552 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:46:07,754 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:46:12,440 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:46:13,441 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:46:17,464 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:46:17,552 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:46:17,552 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:46:18,767 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:46:23,768 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:46:28,769 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:46:32,464 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:46:32,552 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:46:32,552 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:46:33,800 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:46:38,801 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:46:42,441 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:46:44,441 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:46:47,464 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:46:47,552 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:46:47,553 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:46:49,748 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:46:54,749 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:46:59,750 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:47:02,464 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:47:02,552 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:47:02,553 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:47:04,806 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:47:09,807 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:47:12,441 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:47:15,442 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:47:17,464 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:47:17,552 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:47:17,553 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:47:20,796 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:47:25,797 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:47:30,798 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:47:32,475 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:47:32,552 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:47:32,553 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:47:36,798 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:47:41,800 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:47:42,442 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:47:47,443 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:47:47,474 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:47:47,552 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:47:47,553 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:47:52,805 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:47:57,806 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:48:02,474 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:48:02,553 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:48:02,553 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:48:03,764 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:48:08,765 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:48:12,442 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:48:14,443 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:48:17,475 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:48:17,553 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:48:17,553 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:48:19,748 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:48:24,760 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:48:29,761 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:48:32,475 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:48:32,553 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:48:32,554 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:48:34,812 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:48:39,813 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:48:42,443 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:48:45,443 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:48:47,475 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:48:47,553 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:48:47,553 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:48:50,780 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:48:55,781 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:49:00,781 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:49:02,475 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:49:02,553 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:49:02,554 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:49:05,800 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:49:10,801 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:49:12,443 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:49:16,444 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:49:17,475 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:49:17,553 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:49:17,554 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:49:21,757 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:49:26,758 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:49:31,759 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:49:32,475 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:49:32,560 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:49:32,560 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:49:37,722 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:49:42,443 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:49:43,444 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:49:47,475 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:49:47,560 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:49:47,560 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:49:48,728 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:49:53,728 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:49:58,729 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:50:02,475 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:50:02,560 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:50:02,561 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:50:03,812 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:50:08,812 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:50:12,444 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:50:14,445 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:50:17,475 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:50:17,560 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:50:17,561 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:50:19,770 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:50:24,771 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:50:29,772 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:50:32,476 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:50:32,560 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:50:32,561 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:50:35,736 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:50:40,737 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:50:42,444 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:50:46,445 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:50:47,476 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:50:47,560 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:50:47,561 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:50:51,788 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:50:56,789 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:51:01,790 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:51:02,476 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:51:02,561 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:51:02,561 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:51:07,746 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:51:12,445 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:51:13,445 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:51:16,497 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 02:51:17,476 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:51:17,570 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:51:17,570 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:51:18,728 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:51:23,729 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:51:28,729 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:51:32,476 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:51:32,570 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:51:32,570 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:51:33,814 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:51:38,815 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:51:42,445 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:51:44,456 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:51:47,476 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:51:47,570 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:51:47,570 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:51:49,819 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:51:54,820 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:51:59,828 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:52:02,487 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:52:02,570 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:52:02,571 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:52:05,818 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:52:10,820 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:52:12,446 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:52:16,446 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:52:17,487 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:52:17,570 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:52:17,571 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:52:21,764 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:52:26,764 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:52:31,765 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:52:32,487 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:52:32,570 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:52:32,571 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:52:37,745 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:52:42,446 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:52:43,446 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:52:47,487 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:52:47,570 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:52:47,571 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:52:48,840 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:52:53,841 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:52:58,843 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:53:02,488 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:53:02,571 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:53:02,571 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:53:04,826 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:53:09,826 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:53:12,447 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:53:15,447 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:53:17,487 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:53:17,571 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:53:17,572 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:53:20,791 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:53:25,792 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:53:30,793 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:53:32,488 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:53:32,571 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:53:32,571 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:53:35,819 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:53:40,819 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:53:42,447 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:53:46,448 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:53:47,488 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:53:47,571 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:53:47,571 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:53:51,806 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:53:56,807 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:54:01,808 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:54:02,488 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:54:02,571 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:54:02,571 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:54:07,760 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:54:12,447 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:54:13,448 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:54:17,488 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:54:17,571 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:54:17,571 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:54:18,853 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:54:23,853 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:54:28,854 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:54:32,488 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:54:32,571 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:54:32,572 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:54:34,784 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:54:39,785 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:54:42,448 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:54:45,449 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:54:47,488 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:54:47,571 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:54:47,572 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:54:50,864 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:54:55,864 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:55:00,865 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:55:02,488 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:55:02,571 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:55:02,572 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:55:06,796 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:55:11,797 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:55:12,448 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:55:17,449 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:55:17,488 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:55:17,571 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:55:17,572 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:55:22,819 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:55:27,820 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:55:32,488 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:55:32,572 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:55:32,572 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:55:33,777 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:55:38,790 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:55:42,459 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:55:44,460 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:55:47,488 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:55:47,572 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:55:47,572 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:55:49,752 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:55:54,760 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:55:59,760 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:56:02,489 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:56:02,572 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:56:02,572 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:56:04,795 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:56:09,796 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:56:12,449 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:56:15,450 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:56:17,489 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:56:17,572 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:56:17,572 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:56:20,801 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:56:25,802 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:56:30,803 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:56:32,489 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:56:32,572 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:56:32,573 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:56:36,768 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:56:41,768 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:56:42,450 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:56:47,452 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:56:47,489 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:56:47,572 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:56:47,573 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:56:52,834 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:56:57,835 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:57:02,489 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:57:02,572 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:57:02,573 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:57:03,785 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:57:08,785 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:57:12,450 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:57:14,451 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:57:17,489 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:57:17,572 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:57:17,573 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:57:19,812 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:57:24,819 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:57:29,820 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:57:32,489 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:57:32,573 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:57:32,573 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:57:35,818 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:57:40,819 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:57:42,451 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:57:46,452 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:57:47,489 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:57:47,573 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:57:47,573 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:57:51,743 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:57:56,744 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:58:01,745 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:58:02,489 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:58:02,573 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:58:02,574 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:58:06,758 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:58:11,759 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:58:12,451 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:58:17,077 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:58:17,490 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:58:17,573 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:58:17,573 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:58:19,046 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 02:58:22,820 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:58:27,820 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:58:32,489 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:58:32,573 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:58:32,573 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:58:33,741 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:58:38,742 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:58:42,452 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:58:44,453 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:58:47,490 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:58:47,573 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:58:47,574 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:58:49,765 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:58:54,766 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:58:59,767 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:59:02,490 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:59:02,573 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:59:02,574 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:59:05,726 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:59:10,726 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:59:12,452 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:59:16,453 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:59:17,490 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:59:17,573 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:59:17,574 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:59:21,745 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:59:26,749 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:59:31,750 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:59:32,490 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:59:32,573 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:59:32,574 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:59:37,745 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:59:42,453 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 02:59:43,453 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:59:47,490 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 02:59:47,573 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 02:59:47,575 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 02:59:48,826 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:59:53,827 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 02:59:58,828 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:00:02,501 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:00:02,580 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:00:02,580 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:00:04,777 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:00:09,778 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:00:12,453 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:00:15,454 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:00:17,491 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:00:17,580 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:00:17,580 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:00:20,772 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:00:25,773 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:00:30,773 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:00:32,491 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:00:32,580 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:00:32,581 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:00:35,816 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:00:40,817 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:00:42,454 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:00:46,455 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:00:47,492 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:00:47,590 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:00:47,591 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:00:51,826 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:00:56,827 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:01:01,828 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:01:02,492 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:01:02,590 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:01:02,591 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:01:07,781 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:01:12,454 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:01:13,455 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:01:17,492 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:01:17,590 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:01:17,591 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:01:18,809 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:01:23,810 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:01:28,811 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:01:32,492 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:01:32,590 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:01:32,591 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:01:33,830 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:01:38,830 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:01:42,454 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:01:44,455 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:01:47,492 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:01:47,591 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:01:47,591 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:01:49,839 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:01:54,839 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:01:59,840 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:02:02,492 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:02:02,591 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:02:02,591 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:02:05,808 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:02:10,809 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:02:12,455 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:02:16,455 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:02:17,492 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:02:17,591 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:02:17,591 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:02:21,751 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:02:26,760 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:02:31,760 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:02:32,492 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:02:32,591 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:02:32,591 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:02:37,755 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:02:42,455 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:02:43,456 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:02:47,492 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:02:47,591 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:02:47,592 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:02:48,815 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:02:53,816 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:02:58,817 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:03:02,492 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:03:02,591 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:03:02,592 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:03:04,774 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:03:09,775 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:03:12,456 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:03:15,457 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:03:17,493 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:03:17,591 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:03:17,592 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:03:20,843 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:03:25,843 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:03:30,844 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:03:32,493 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:03:32,591 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:03:32,592 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:03:36,750 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:03:41,751 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:03:42,456 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:03:47,458 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:03:47,493 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:03:47,600 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:03:47,601 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:03:52,847 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:03:57,848 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:04:02,493 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:04:02,600 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:04:02,601 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:04:03,753 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:04:08,754 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:04:12,457 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:04:14,458 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:04:17,493 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:04:17,601 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:04:17,601 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:04:19,843 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:04:24,844 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:04:29,844 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:04:32,493 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:04:32,601 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:04:32,601 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:04:35,755 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:04:40,755 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:04:42,460 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:04:46,461 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:04:47,493 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:04:47,607 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:04:47,608 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:04:51,811 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:04:56,812 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:05:01,812 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:05:02,493 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:05:02,608 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:05:02,608 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:05:06,837 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:05:11,838 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:05:12,461 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:05:17,462 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:05:17,493 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:05:17,608 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:05:17,608 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:05:22,826 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:05:27,827 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:05:32,494 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:05:32,608 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:05:32,608 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:05:33,788 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:05:38,789 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:05:42,461 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:05:44,462 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:05:47,494 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:05:47,608 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:05:47,608 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:05:49,879 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:05:54,880 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:05:59,881 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:06:02,494 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:06:02,608 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:06:02,609 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:06:05,785 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:06:10,785 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:06:12,461 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:06:16,463 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:06:17,494 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:06:17,608 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:06:17,609 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:06:21,844 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:06:26,844 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:06:31,845 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:06:32,494 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:06:32,608 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:06:32,609 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:06:36,855 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:06:41,856 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:06:42,462 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:06:47,470 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:06:47,494 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:06:47,608 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:06:47,609 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:06:52,771 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:06:57,771 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:07:02,494 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:07:02,608 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:07:02,609 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:07:02,776 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:07:07,777 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:07:12,463 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:07:13,463 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:07:17,494 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:07:17,608 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:07:17,609 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:07:18,841 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:07:23,842 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:07:28,842 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:07:32,494 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:07:32,609 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:07:32,609 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:07:34,789 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:07:39,790 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:07:41,999 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 03:07:42,463 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:07:44,005 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 03:07:45,464 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:07:47,494 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:07:47,609 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:07:47,609 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:07:50,780 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:07:55,781 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:08:00,782 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:08:02,495 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:08:02,609 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:08:02,609 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:08:05,801 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:08:10,801 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:08:12,463 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:08:16,464 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:08:17,494 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:08:17,609 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:08:17,609 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:08:21,867 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:08:26,868 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:08:31,868 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:08:32,495 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:08:32,609 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:08:32,609 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:08:37,789 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:08:42,464 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:08:43,465 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:08:47,495 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:08:47,609 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:08:47,609 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:08:48,860 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:08:53,860 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:08:58,861 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:09:02,495 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:09:02,609 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:09:02,610 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:09:04,858 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:09:09,858 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:09:12,464 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:09:15,465 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:09:17,495 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:09:17,609 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:09:17,610 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:09:20,899 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:09:25,900 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:09:30,900 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:09:32,495 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:09:32,609 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:09:32,610 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:09:36,799 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:09:41,800 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:09:42,465 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:09:47,466 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:09:47,500 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:09:47,609 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:09:47,610 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:09:52,852 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:09:57,860 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:10:02,500 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:10:02,610 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:10:02,610 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:10:03,787 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:10:08,788 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:10:12,466 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:10:14,467 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:10:17,500 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:10:17,610 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:10:17,610 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:10:19,846 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:10:24,846 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:10:29,847 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:10:32,500 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:10:32,610 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:10:32,610 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:10:35,802 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:10:40,803 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:10:42,476 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:10:46,454 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:10:47,500 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:10:47,610 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:10:47,610 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:10:47,635 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 03:10:51,828 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:10:56,829 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:11:01,829 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:11:02,501 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:11:02,610 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:11:02,610 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:11:06,869 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:11:11,870 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:11:12,466 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:11:17,470 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:11:17,500 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:11:17,610 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:11:17,617 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:11:22,833 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:11:27,834 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:11:32,500 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:11:32,610 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:11:32,611 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:11:32,872 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:11:37,880 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:11:42,467 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:11:43,468 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:11:47,500 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:11:47,610 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:11:47,611 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:11:48,825 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:11:53,825 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:11:58,830 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:12:02,501 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:12:02,610 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:12:02,611 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:12:03,876 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:12:08,877 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:12:12,467 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:12:14,468 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:12:17,501 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:12:17,610 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:12:17,611 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:12:19,855 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:12:24,856 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:12:29,856 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:12:32,501 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:12:32,611 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:12:32,611 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:12:35,789 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:12:40,790 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:12:42,468 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:12:46,469 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:12:47,501 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:12:47,611 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:12:47,611 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:12:51,808 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:12:56,809 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:13:01,810 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:13:02,501 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:13:02,611 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:13:02,611 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:13:07,801 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:13:12,468 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:13:13,469 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:13:17,501 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:13:17,611 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:13:17,612 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:13:18,835 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:13:23,836 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:13:28,836 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:13:32,501 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:13:32,611 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:13:32,612 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:13:34,797 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:13:39,797 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:13:42,469 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:13:45,470 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:13:47,501 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:13:47,620 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:13:47,620 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:13:50,803 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:13:55,804 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:14:00,804 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:14:02,502 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:14:02,620 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:14:02,620 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:14:05,886 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:14:10,887 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:14:12,469 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:14:16,470 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:14:17,502 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:14:17,620 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:14:17,620 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:14:21,839 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:14:26,847 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:14:31,841 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:14:32,502 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:14:32,620 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:14:32,620 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:14:36,856 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:14:41,857 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:14:42,470 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:14:47,471 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:14:47,502 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:14:47,620 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:14:47,621 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:14:52,847 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:14:57,848 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:15:02,502 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:15:02,620 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:15:02,621 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:15:02,862 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:15:07,862 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:15:12,470 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:15:13,471 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:15:17,502 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:15:17,620 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:15:17,621 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:15:18,807 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:15:23,808 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:15:28,809 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:15:32,502 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:15:32,620 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:15:32,621 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:15:33,850 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:15:38,851 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:15:42,471 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:15:44,472 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:15:47,502 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:15:47,621 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:15:47,621 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:15:49,806 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:15:54,807 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:15:59,808 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:16:02,510 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:16:02,621 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:16:02,621 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:16:04,867 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:16:09,867 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:16:12,471 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:16:15,473 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:16:17,502 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:16:17,621 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:16:17,621 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:16:20,832 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:16:25,832 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:16:30,840 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:16:32,503 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:16:32,621 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:16:32,621 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:16:35,849 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:16:40,850 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:16:42,472 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:16:46,473 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:16:47,503 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:16:47,621 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:16:47,621 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:16:51,864 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:16:56,865 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:17:01,866 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:17:02,503 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:17:02,621 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:17:02,621 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:17:07,793 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:17:12,472 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:17:13,474 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:17:17,503 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:17:17,621 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:17:17,622 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:17:18,854 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:17:23,854 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:17:28,855 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:17:32,504 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:17:32,621 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:17:32,622 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:17:33,861 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:17:38,862 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:17:42,473 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:17:44,474 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:17:47,503 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:17:47,621 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:17:47,622 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:17:49,876 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:17:54,877 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:17:59,877 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:18:02,503 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:18:02,622 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:18:02,622 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:18:05,778 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:18:10,779 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:18:12,473 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:18:16,474 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:18:17,503 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:18:17,622 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:18:17,622 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:18:21,916 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:18:26,917 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:18:31,917 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:18:32,503 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:18:32,622 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:18:32,622 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:18:37,840 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:18:42,474 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:18:43,475 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:18:47,503 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:18:47,622 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:18:47,623 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:18:48,794 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:18:53,794 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:18:58,795 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:19:02,503 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:19:02,622 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:19:02,623 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:19:03,841 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:19:08,841 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:19:12,474 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:19:14,475 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:19:17,504 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:19:17,622 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:19:17,623 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:19:19,848 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:19:24,848 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:19:29,849 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:19:32,504 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:19:32,622 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:19:32,623 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:19:35,811 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:19:40,812 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:19:42,475 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:19:46,476 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:19:47,504 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:19:47,622 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:19:47,623 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:19:51,859 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:19:56,860 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:20:01,860 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:20:02,504 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:20:02,622 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:20:02,623 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:20:07,826 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:20:12,475 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:20:13,476 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:20:17,504 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:20:17,623 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:20:17,624 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:20:18,834 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:20:23,835 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:20:28,835 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:20:32,511 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:20:32,623 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:20:32,623 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:20:33,877 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:20:38,880 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:20:42,482 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:20:44,482 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:20:47,504 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:20:47,623 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:20:47,624 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:20:49,795 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:20:54,796 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:20:59,797 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:21:02,504 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:21:02,623 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:21:02,623 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:21:04,841 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:21:09,841 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:21:12,476 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:21:15,477 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:21:17,505 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:21:17,623 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:21:17,623 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:21:20,878 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:21:25,887 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:21:30,888 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:21:32,505 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:21:32,623 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:21:32,624 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:21:36,839 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:21:41,840 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:21:42,478 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:21:47,479 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:21:47,505 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:21:47,623 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:21:47,624 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:21:52,844 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:21:57,845 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:22:02,505 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:22:02,623 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:22:02,624 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:22:03,803 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:22:08,803 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:22:12,477 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:22:14,478 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:22:17,505 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:22:17,632 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:22:17,633 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:22:19,870 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:22:24,871 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:22:29,871 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:22:32,505 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:22:32,633 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:22:32,633 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:22:35,851 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:22:40,852 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:22:42,484 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:22:46,485 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:22:47,505 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:22:47,633 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:22:47,633 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:22:51,869 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:22:56,870 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:23:01,870 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:23:02,505 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:23:02,633 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:23:02,633 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:23:07,858 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:23:12,478 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:23:13,479 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:23:17,509 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:23:17,633 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:23:17,633 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:23:18,880 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:23:23,881 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:23:28,881 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:23:32,509 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:23:32,633 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:23:32,634 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:23:34,837 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:23:38,310 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 03:23:39,976 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:23:40,314 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 03:23:42,479 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:23:45,487 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:23:47,509 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:23:47,633 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:23:47,634 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:23:50,859 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:23:55,860 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:24:00,860 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:24:02,509 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:24:02,633 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:24:02,634 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:24:06,804 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:24:11,804 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:24:12,479 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:24:16,426 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 03:24:16,917 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:24:17,509 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:24:17,633 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:24:17,634 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:24:22,862 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:24:27,869 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:24:32,509 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:24:32,633 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:24:32,634 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:24:33,820 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:24:38,821 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:24:42,480 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:24:44,481 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:24:47,510 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:24:47,634 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:24:47,634 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:24:49,858 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:24:54,869 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:24:59,869 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:25:02,510 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:25:02,634 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:25:02,634 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:25:04,872 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:25:09,873 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:25:12,491 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:25:15,496 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:25:17,510 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:25:17,634 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:25:17,634 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:25:20,875 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:25:25,876 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:25:30,877 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:25:32,510 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:25:32,634 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:25:32,634 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:25:36,836 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:25:41,837 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:25:42,481 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:25:47,482 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:25:47,510 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:25:47,634 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:25:47,634 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:25:52,865 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:25:57,865 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:26:02,510 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:26:02,634 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:26:02,634 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:26:03,831 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:26:08,832 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:26:12,481 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:26:14,482 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:26:17,510 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:26:17,634 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:26:17,634 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:26:19,852 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:26:24,857 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:26:29,858 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:26:32,510 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:26:32,640 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:26:32,641 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:26:35,806 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:26:40,807 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:26:42,482 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:26:46,483 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:26:47,510 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:26:47,640 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:26:47,640 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:26:51,803 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:26:56,804 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:27:01,805 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:27:02,511 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:27:02,640 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:27:02,640 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:27:06,895 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:27:11,896 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:27:12,483 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:27:17,484 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:27:17,511 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:27:17,640 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:27:17,641 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:27:22,834 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:27:27,836 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:27:32,511 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:27:32,649 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:27:32,649 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:27:32,849 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:27:37,850 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:27:42,483 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:27:43,483 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:27:47,511 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:27:47,649 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:27:47,650 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:27:48,870 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:27:53,878 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:27:58,879 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:28:02,511 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:28:02,649 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:28:02,650 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:28:03,896 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:28:08,897 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:28:12,483 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:28:14,484 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:28:17,511 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:28:17,649 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:28:17,650 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:28:19,855 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:28:24,856 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:28:29,856 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:28:32,511 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:28:32,650 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:28:32,650 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:28:35,830 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:28:40,830 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:28:42,484 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:28:46,485 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:28:47,511 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:28:47,650 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:28:47,650 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:28:51,885 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:28:56,886 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:29:01,886 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:29:02,511 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:29:02,650 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:29:02,650 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:29:06,907 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:29:11,908 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:29:12,484 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:29:17,485 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:29:17,511 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:29:17,650 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:29:17,650 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:29:22,871 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:29:27,872 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:29:32,512 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:29:32,650 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:29:32,650 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:29:32,902 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:29:37,903 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:29:42,485 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:29:43,485 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:29:47,512 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:29:47,650 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:29:47,651 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:29:48,825 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:29:53,826 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:29:58,827 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:30:02,512 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:30:02,650 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:30:02,651 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:30:03,866 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:30:08,866 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:30:12,485 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:30:14,486 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:30:17,512 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:30:17,651 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:30:17,651 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:30:19,864 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:30:24,870 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:30:29,870 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:30:32,512 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:30:32,650 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:30:32,651 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:30:34,878 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:30:39,879 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:30:42,486 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:30:45,487 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:30:47,512 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:30:47,651 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:30:47,651 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:30:50,813 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:30:55,814 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:31:00,815 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:31:02,512 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:31:02,651 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:31:02,653 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:31:05,902 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:31:10,902 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:31:12,486 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:31:16,487 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:31:17,512 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:31:17,651 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:31:17,651 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:31:21,827 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:31:26,836 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:31:31,840 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:31:32,513 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:31:32,651 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:31:32,652 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:31:36,901 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:31:41,901 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:31:42,487 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:31:47,488 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:31:47,513 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:31:47,651 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:31:47,651 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:31:52,818 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:31:57,818 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:32:02,513 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:32:02,651 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:32:02,652 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:32:03,817 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:32:08,818 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:32:12,490 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:32:14,491 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:32:17,513 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:32:17,651 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:32:17,652 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:32:19,916 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:32:24,917 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:32:29,922 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:32:32,513 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:32:32,651 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:32:32,652 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:32:35,901 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:32:40,902 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:32:42,490 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:32:46,491 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:32:47,513 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:32:47,651 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:32:47,652 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:32:51,858 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:32:56,858 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:33:01,859 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:33:02,520 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:33:02,652 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:33:02,652 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:33:06,902 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:33:11,927 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:33:12,491 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:33:17,500 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:33:17,520 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:33:17,652 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:33:17,652 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:33:22,850 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:33:27,850 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:33:32,520 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:33:32,652 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:33:32,653 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:33:32,855 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:33:37,860 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:33:42,492 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:33:43,492 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:33:47,520 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:33:47,652 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:33:47,653 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:33:48,885 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:33:53,886 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:33:58,886 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:34:02,520 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:34:02,652 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:34:02,653 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:34:04,870 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:34:09,871 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:34:12,493 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:34:15,493 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:34:17,520 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:34:17,652 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:34:17,653 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:34:20,836 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:34:25,837 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:34:30,838 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:34:32,521 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:34:32,652 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:34:32,653 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:34:35,885 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:34:40,885 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:34:42,493 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:34:46,493 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:34:47,521 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:34:47,653 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:34:47,654 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:34:51,920 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:34:56,930 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:35:01,930 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:35:02,522 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:35:02,652 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:35:02,653 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:35:07,880 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:35:12,493 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:35:13,493 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:35:17,522 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:35:17,652 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:35:17,653 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:35:18,916 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:35:23,917 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:35:28,917 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:35:32,527 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:35:32,653 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:35:32,653 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:35:34,858 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:35:39,859 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:35:42,493 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:35:45,494 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:35:47,535 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:35:47,653 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:35:47,653 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:35:50,847 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:35:55,850 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:36:00,850 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:36:02,535 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:36:02,653 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:36:02,653 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:36:05,871 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:36:10,871 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:36:12,494 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:36:16,495 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:36:17,536 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:36:17,653 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:36:17,653 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:36:18,988 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 03:36:21,910 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:36:26,910 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:36:31,911 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:36:32,536 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:36:32,653 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:36:32,654 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:36:37,813 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:36:42,494 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:36:43,495 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:36:47,536 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:36:47,653 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:36:47,654 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:36:48,878 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:36:53,879 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:36:58,880 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:37:02,536 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:37:02,653 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:37:02,654 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:37:04,832 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:37:09,840 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:37:12,495 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:37:15,495 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:37:17,536 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:37:17,654 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:37:17,654 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:37:20,849 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:37:25,857 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:37:30,860 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:37:32,536 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:37:32,654 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:37:32,654 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:37:36,828 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:37:41,828 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:37:42,495 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:37:47,496 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:37:47,536 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:37:47,654 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:37:47,654 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:37:52,890 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:37:57,890 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:38:02,536 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:38:02,654 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:38:02,655 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:38:02,900 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:38:07,901 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:38:12,496 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:38:13,496 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:38:17,536 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:38:17,654 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:38:17,654 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:38:18,821 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:38:23,822 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:38:28,822 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:38:32,537 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:38:32,660 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:38:32,660 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:38:33,824 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:38:38,825 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:38:42,496 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:38:44,497 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:38:47,537 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:38:47,660 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:38:47,660 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:38:49,883 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:38:54,884 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:38:59,884 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:39:02,537 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:39:02,660 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:39:02,661 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:39:05,844 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:39:10,845 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:39:12,498 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:39:16,510 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:39:17,537 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:39:17,670 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:39:17,671 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:39:21,841 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:39:26,842 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:39:31,843 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:39:32,537 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:39:32,670 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:39:32,671 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:39:36,892 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:39:41,893 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:39:42,497 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:39:47,507 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:39:47,537 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:39:47,670 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:39:47,671 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:39:52,875 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:39:57,876 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:40:02,537 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:40:02,671 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:40:02,671 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:40:02,888 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:40:07,889 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:40:12,498 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:40:13,499 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:40:17,537 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:40:17,671 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:40:17,671 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:40:18,920 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:40:23,920 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:40:28,921 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:40:32,537 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:40:32,671 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:40:32,671 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:40:34,868 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:40:39,869 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:40:42,529 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:40:45,530 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:40:47,537 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:40:47,671 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:40:47,671 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:40:50,863 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:40:55,864 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:41:00,865 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:41:02,538 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:41:02,671 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:41:02,672 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:41:06,844 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:41:11,845 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:41:12,499 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:41:17,500 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:41:17,538 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:41:17,671 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:41:17,672 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:41:22,848 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:41:27,849 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:41:32,538 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:41:32,680 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:41:32,680 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:41:32,941 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:41:37,941 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:41:42,499 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:41:43,237 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 03:41:43,501 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:41:47,538 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:41:47,680 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:41:47,680 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:41:48,933 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:41:53,940 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:41:58,948 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:42:02,538 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:42:02,680 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:42:02,681 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:42:04,901 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:42:09,910 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:42:12,499 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:42:15,500 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:42:17,538 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:42:17,680 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:42:17,680 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:42:20,941 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:42:25,942 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:42:30,943 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:42:32,538 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:42:32,680 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:42:32,681 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:42:36,859 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:42:41,859 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:42:42,500 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:42:47,501 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:42:47,538 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:42:47,680 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:42:47,681 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:42:53,002 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:42:58,003 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:43:02,539 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:43:02,680 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:43:02,681 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:43:03,894 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:43:08,895 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:43:12,500 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:43:14,501 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:43:17,539 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:43:17,681 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:43:17,681 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:43:19,890 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:43:24,890 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:43:29,891 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:43:32,539 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:43:32,681 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:43:32,681 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:43:34,935 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:43:39,935 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:43:42,501 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:43:45,502 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:43:47,539 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:43:47,681 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:43:47,681 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:43:50,847 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:43:55,847 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:44:00,848 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:44:02,540 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:44:02,681 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:44:02,681 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:44:05,886 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:44:10,887 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:44:12,502 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:44:16,503 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:44:17,540 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:44:17,683 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:44:17,684 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:44:21,896 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:44:26,897 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:44:31,900 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:44:32,540 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:44:32,681 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:44:32,682 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:44:36,938 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:44:41,939 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:44:42,502 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:44:47,503 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:44:47,540 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:44:47,681 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:44:47,682 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:44:52,917 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:44:57,918 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:45:02,541 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:45:02,681 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:45:02,682 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:45:03,898 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:45:08,899 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:45:12,502 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:45:14,503 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:45:17,541 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:45:17,681 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:45:17,682 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:45:19,870 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:45:24,871 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:45:29,880 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:45:32,541 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:45:32,681 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:45:32,682 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:45:35,871 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:45:40,871 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:45:42,503 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:45:46,504 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:45:47,541 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:45:47,690 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:45:47,691 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:45:51,957 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:45:56,958 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:46:01,959 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:46:02,541 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:46:02,690 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:46:02,691 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:46:07,922 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:46:12,503 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:46:13,504 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:46:17,541 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:46:17,691 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:46:17,691 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:46:18,947 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:46:23,948 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:46:28,948 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:46:32,541 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:46:32,700 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:46:32,700 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:46:34,918 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:46:39,919 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:46:42,504 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:46:45,504 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:46:47,541 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:46:47,700 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:46:47,701 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:46:50,987 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:46:55,988 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:47:00,988 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:47:02,541 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:47:02,700 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:47:02,700 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:47:06,923 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:47:11,923 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:47:12,504 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:47:17,505 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:47:17,542 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:47:17,700 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:47:17,701 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:47:23,003 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:47:28,004 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:47:32,542 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:47:32,700 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:47:32,701 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:47:33,888 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:47:38,888 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:47:42,504 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:47:44,505 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:47:47,542 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:47:47,700 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:47:47,701 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:47:49,997 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:47:54,998 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:47:59,999 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:48:02,542 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:48:02,700 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:48:02,701 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:48:05,988 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:48:10,998 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:48:12,505 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:48:16,506 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:48:17,542 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:48:17,701 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:48:17,701 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:48:22,013 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:48:27,014 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:48:32,014 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:48:32,543 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:48:32,701 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:48:32,701 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:48:38,010 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:48:42,506 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:48:43,506 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:48:47,550 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:48:47,701 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:48:47,701 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:48:48,984 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:48:53,985 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:48:58,986 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:49:02,550 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:49:02,701 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:49:02,702 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:49:04,985 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:49:09,986 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:49:12,506 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:49:15,507 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:49:17,550 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:49:17,701 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:49:17,701 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:49:19,839 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 03:49:21,234 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:49:21,844 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 03:49:26,249 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:49:31,241 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:49:32,550 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:49:32,701 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:49:32,702 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:49:36,965 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:49:41,966 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:49:42,506 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:49:47,507 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:49:47,550 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:49:47,701 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:49:47,702 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:49:52,933 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:49:57,934 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:50:02,550 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:50:02,701 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:50:02,702 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:50:02,943 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:50:07,944 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:50:12,507 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:50:13,508 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:50:17,550 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:50:17,702 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:50:17,702 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:50:18,968 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:50:23,980 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:50:28,980 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:50:32,550 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:50:32,702 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:50:32,702 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:50:34,951 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:50:39,952 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:50:42,508 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:50:45,508 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:50:47,551 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:50:47,702 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:50:47,702 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:50:50,883 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:50:55,890 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:51:00,891 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:51:02,560 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:51:02,702 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:51:02,702 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:51:05,927 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:51:10,927 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:51:12,508 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:51:16,510 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:51:17,560 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:51:17,702 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:51:17,703 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:51:21,966 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:51:26,967 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:51:31,968 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:51:32,560 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:51:32,702 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:51:32,703 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:51:37,920 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:51:42,509 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:51:43,509 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:51:47,560 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:51:47,702 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:51:47,703 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:51:48,981 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:51:53,981 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:51:58,982 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:52:02,560 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:52:02,702 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:52:02,703 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:52:04,964 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:52:09,965 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:52:12,509 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:52:15,510 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:52:17,560 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:52:17,702 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:52:17,703 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:52:20,959 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:52:25,960 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:52:30,960 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:52:32,561 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:52:32,702 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:52:32,703 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:52:36,934 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:52:41,935 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:52:42,510 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:52:47,511 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:52:47,561 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:52:47,703 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:52:47,703 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:52:52,960 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:52:57,963 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:53:02,561 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:53:02,703 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:53:02,703 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:53:03,954 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:53:08,954 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:53:12,510 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:53:14,511 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:53:17,561 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:53:17,703 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:53:17,703 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:53:19,872 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:53:24,873 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:53:29,874 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:53:32,561 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:53:32,703 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:53:32,703 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:53:34,909 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:53:39,910 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:53:42,511 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:53:45,511 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:53:47,561 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:53:47,711 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:53:47,711 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:53:50,895 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:53:55,895 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:54:00,896 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:54:02,561 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:54:02,711 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:54:02,711 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:54:05,932 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:54:10,932 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:54:12,511 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:54:16,516 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:54:17,561 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:54:17,711 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:54:17,711 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:54:21,916 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:54:26,930 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:54:31,930 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:54:32,569 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:54:32,711 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:54:32,711 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:54:37,903 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:54:42,512 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:54:43,512 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:54:47,569 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:54:47,717 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:54:47,717 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:54:48,932 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:54:53,933 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:54:58,933 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:55:02,569 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:55:02,711 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:55:02,711 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:55:04,933 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:55:09,934 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:55:12,512 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:55:15,513 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:55:17,569 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:55:17,720 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:55:17,720 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:55:20,959 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:55:25,969 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:55:30,970 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:55:32,569 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:55:32,720 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:55:32,721 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:55:36,910 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:55:41,911 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:55:42,513 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:55:47,514 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:55:47,569 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:55:47,720 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:55:47,720 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:55:52,914 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:55:57,915 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:56:02,580 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:56:02,723 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:56:02,724 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:56:03,880 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:56:08,881 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:56:12,565 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:56:14,566 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:56:17,580 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:56:17,730 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:56:17,730 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:56:19,959 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:56:24,959 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:56:29,960 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:56:32,580 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:56:32,730 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:56:32,730 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:56:35,930 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:56:40,930 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:56:42,514 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:56:46,515 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:56:47,580 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:56:47,730 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:56:47,730 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:56:51,938 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:56:56,939 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:57:01,939 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:57:02,580 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:57:02,730 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:57:02,730 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:57:07,891 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:57:12,514 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:57:13,515 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:57:17,581 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:57:17,730 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:57:17,731 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:57:18,939 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:57:23,940 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:57:28,941 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:57:32,587 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:57:32,730 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:57:32,730 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:57:34,876 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:57:39,880 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:57:42,515 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:57:45,516 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:57:47,587 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:57:47,740 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:57:47,741 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:57:50,987 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:57:55,987 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:57:58,760 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 03:58:01,133 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:58:02,587 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:58:02,738 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:58:02,739 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:58:06,921 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:58:11,922 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:58:12,516 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:58:17,516 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:58:17,588 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:58:17,738 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:58:17,739 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:58:22,899 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:58:27,900 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:58:32,588 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:58:32,739 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:58:32,739 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:58:32,993 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:58:37,994 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:58:42,516 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:58:43,517 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:58:47,588 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:58:47,739 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:58:47,739 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:58:48,983 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:58:53,983 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:58:58,984 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:59:02,588 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:59:02,739 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:59:02,740 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:59:04,958 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:59:09,959 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:59:12,517 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:59:15,526 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:59:17,588 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:59:17,739 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:59:17,739 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:59:20,952 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:59:25,953 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:59:30,954 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:59:32,588 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:59:32,739 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:59:32,740 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:59:35,965 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:59:40,970 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:59:42,517 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 03:59:46,518 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:59:47,588 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 03:59:47,739 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 03:59:47,740 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 03:59:51,918 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 03:59:56,919 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:00:01,920 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:00:02,589 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:00:02,739 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:00:02,740 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:00:06,934 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:00:11,944 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:00:12,517 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:00:17,518 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:00:17,589 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:00:17,739 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:00:17,740 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:00:22,993 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:00:27,993 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:00:32,589 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:00:32,739 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:00:32,740 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:00:33,976 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:00:38,977 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:00:42,518 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:00:44,519 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:00:47,589 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:00:47,740 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:00:47,740 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:00:49,939 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:00:54,940 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:00:59,940 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:01:02,589 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:01:02,740 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:01:02,740 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:01:05,919 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:01:10,920 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:01:12,518 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:01:16,519 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:01:17,589 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:01:17,740 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:01:17,741 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:01:21,975 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:01:26,976 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:01:31,980 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:01:32,589 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:01:32,743 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:01:32,743 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:01:37,954 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:01:42,519 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:01:43,520 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:01:47,589 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:01:47,743 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:01:47,744 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:01:49,099 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:01:54,100 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:01:59,100 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:02:02,589 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:02:02,743 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:02:02,744 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:02:04,895 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:02:09,896 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:02:12,519 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:02:12,655 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 04:02:15,520 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:02:17,589 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:02:17,743 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:02:17,744 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:02:20,939 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:02:25,940 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:02:30,940 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:02:32,590 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:02:32,743 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:02:32,744 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:02:36,919 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:02:41,919 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:02:42,520 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:02:47,521 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:02:47,590 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:02:47,743 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:02:47,744 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:02:52,939 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:02:57,940 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:03:02,590 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:03:02,744 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:03:02,744 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:03:02,998 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:03:07,999 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:03:12,520 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:03:13,529 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:03:17,590 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:03:17,744 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:03:17,744 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:03:18,955 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:03:23,956 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:03:28,957 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:03:32,590 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:03:32,744 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:03:32,744 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:03:34,944 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:03:39,945 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:03:42,520 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:03:45,521 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:03:47,590 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:03:47,744 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:03:47,744 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:03:50,936 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:03:55,937 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:04:00,938 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:04:02,590 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:04:02,744 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:04:02,745 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:04:05,973 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:04:10,973 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:04:12,521 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:04:16,523 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:04:17,591 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:04:17,744 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:04:17,745 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:04:21,907 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:04:26,908 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:04:31,909 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:04:32,591 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:04:32,750 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:04:32,750 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:04:36,987 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:04:41,990 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:04:42,521 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:04:47,523 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:04:47,591 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:04:47,749 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:04:47,750 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:04:52,982 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:04:57,982 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:05:02,591 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:05:02,750 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:05:02,750 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:05:03,951 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:05:08,952 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:05:12,522 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:05:14,523 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:05:17,591 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:05:17,750 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:05:17,750 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:05:19,919 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:05:24,920 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:05:29,920 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:05:32,591 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:05:32,750 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:05:32,750 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:05:34,928 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:05:39,929 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:05:42,523 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:05:45,523 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:05:47,591 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:05:47,750 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:05:47,750 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:05:50,997 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:05:55,998 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:06:00,998 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:06:02,591 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:06:02,750 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:06:02,750 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:06:06,010 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:06:11,011 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:06:12,523 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:06:16,524 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:06:17,592 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:06:17,750 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:06:17,750 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:06:21,910 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:06:26,910 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:06:31,911 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:06:32,592 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:06:32,750 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:06:32,751 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:06:36,967 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:06:41,968 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:06:42,523 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:06:47,524 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:06:47,592 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:06:47,760 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:06:47,760 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:06:52,946 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:06:57,946 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:07:02,592 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:07:02,760 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:07:02,760 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:07:03,922 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:07:08,922 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:07:12,524 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:07:14,524 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:07:17,592 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:07:17,761 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:07:17,761 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:07:20,002 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:07:25,002 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:07:30,003 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:07:32,592 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:07:32,762 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:07:32,762 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:07:35,938 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:07:40,939 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:07:42,524 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:07:46,530 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:07:47,592 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:07:47,762 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:07:47,763 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:07:51,958 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:07:56,958 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:08:01,959 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:08:02,592 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:08:02,762 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:08:02,762 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:08:06,986 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:08:11,987 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:08:12,524 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:08:17,526 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:08:17,592 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:08:17,762 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:08:17,762 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:08:22,995 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:08:28,000 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:08:32,592 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:08:32,762 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:08:32,763 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:08:33,969 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:08:38,970 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:08:42,525 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:08:44,526 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:08:47,593 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:08:47,762 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:08:47,763 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:08:50,007 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:08:55,007 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:09:00,008 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:09:02,593 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:09:02,763 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:09:02,763 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:09:05,968 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:09:10,968 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:09:12,525 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:09:16,526 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:09:17,600 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:09:17,763 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:09:17,763 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:09:21,994 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:09:26,995 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:09:31,995 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:09:32,600 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:09:32,763 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:09:32,763 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:09:37,946 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:09:42,526 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:09:43,527 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:09:47,600 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:09:47,763 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:09:47,763 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:09:48,974 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:09:53,975 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:09:58,975 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:10:02,600 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:10:02,763 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:10:02,763 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:10:04,975 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:10:09,976 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:10:12,526 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:10:15,527 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:10:17,600 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:10:17,763 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:10:17,763 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:10:21,019 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:10:26,019 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:10:31,020 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:10:32,600 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:10:32,770 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:10:32,770 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:10:36,980 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:10:41,990 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:10:42,527 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:10:47,528 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:10:47,600 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:10:47,770 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:10:47,770 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:10:52,930 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:10:57,931 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:11:02,600 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:11:02,770 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:11:02,780 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:11:02,971 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:11:07,998 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:11:12,538 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:11:13,539 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:11:17,601 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:11:17,770 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:11:17,771 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:11:18,950 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:11:23,951 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:11:28,952 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:11:32,601 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:11:32,770 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:11:32,771 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:11:33,969 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:11:38,970 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:11:42,528 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:11:44,529 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:11:47,601 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:11:47,770 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:11:47,771 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:11:49,964 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:11:54,965 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:11:59,965 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:12:02,601 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:12:02,770 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:12:02,771 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:12:04,992 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:12:09,993 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:12:12,528 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:12:15,530 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:12:17,601 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:12:17,770 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:12:17,771 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:12:20,973 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:12:25,974 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:12:30,975 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:12:32,601 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:12:32,771 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:12:32,771 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:12:35,983 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:12:40,984 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:12:42,529 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:12:46,530 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:12:47,601 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:12:47,771 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:12:47,771 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:12:51,930 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:12:56,930 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:13:01,931 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:13:02,601 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:13:02,771 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:13:02,771 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:13:07,025 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:13:12,025 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:13:12,530 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:13:17,531 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:13:17,601 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:13:17,771 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:13:17,772 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:13:23,038 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:13:28,038 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:13:32,601 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:13:32,771 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:13:32,771 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:13:33,972 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:13:38,973 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:13:42,530 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:13:44,531 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:13:47,601 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:13:47,771 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:13:47,771 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:13:49,965 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:13:54,966 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:13:59,967 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:14:02,602 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:14:02,771 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:14:02,772 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:14:05,957 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:14:10,958 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:14:12,530 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:14:16,540 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:14:17,602 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:14:17,771 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:14:17,772 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:14:21,977 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:14:26,978 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:14:31,978 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:14:32,602 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:14:32,771 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:14:32,772 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:14:37,941 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:14:42,540 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:14:43,540 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:14:47,411 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 04:14:47,602 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:14:47,771 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:14:47,780 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:14:49,017 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:14:53,429 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 04:14:54,874 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:14:59,875 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:15:02,602 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:15:02,772 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:15:02,772 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:15:04,992 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:15:09,993 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:15:12,540 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:15:15,549 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:15:17,602 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:15:17,772 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:15:17,772 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:15:20,999 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:15:26,000 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:15:31,001 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:15:32,602 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:15:32,772 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:15:32,772 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:15:36,992 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:15:42,000 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:15:42,540 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:15:47,541 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:15:47,602 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:15:47,772 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:15:47,772 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:15:52,937 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:15:57,938 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:16:02,602 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:16:02,772 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:16:02,773 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:16:02,971 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:16:07,972 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:16:12,541 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:16:13,542 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:16:17,602 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:16:17,772 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:16:17,772 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:16:19,000 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:16:24,001 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:16:29,002 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:16:32,602 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:16:32,772 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:16:32,772 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:16:34,979 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:16:39,979 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:16:42,541 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:16:45,542 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:16:47,603 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:16:47,772 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:16:47,773 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:16:50,977 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:16:55,977 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:17:00,978 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:17:02,603 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:17:02,772 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:17:02,773 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:17:06,975 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:17:11,975 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:17:12,542 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:17:17,543 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:17:17,603 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:17:17,773 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:17:17,773 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:17:23,000 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:17:28,000 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:17:32,603 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:17:32,773 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:17:32,773 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:17:33,031 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:17:38,032 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:17:42,542 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:17:43,543 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:17:47,603 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:17:47,773 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:17:47,773 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:17:48,944 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:17:53,944 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:17:58,945 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:18:02,610 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:18:02,773 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:18:02,773 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:18:03,983 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:18:08,983 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:18:12,543 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:18:14,543 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:18:17,610 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:18:17,773 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:18:17,774 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:18:19,999 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:18:25,000 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:18:30,000 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:18:32,610 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:18:32,773 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:18:32,774 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:18:35,992 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:18:41,001 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:18:42,550 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:18:46,551 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:18:47,610 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:18:47,782 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:18:47,783 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:18:51,970 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:18:56,971 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:19:01,971 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:19:02,610 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:19:02,790 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:19:02,790 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:19:07,966 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:19:12,551 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:19:13,552 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:19:17,610 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:19:17,790 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:19:17,791 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:19:19,012 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:19:24,013 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:19:29,014 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:19:32,611 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:19:32,790 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:19:32,790 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:19:34,983 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:19:39,983 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:19:42,551 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:19:45,552 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:19:47,611 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:19:47,790 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:19:47,791 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:19:51,048 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:19:56,049 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:20:01,050 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:20:02,611 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:20:02,790 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:20:02,791 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:20:07,024 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:20:12,025 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:20:12,552 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:20:17,560 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:20:17,611 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:20:17,790 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:20:17,791 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:20:22,980 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:20:27,981 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:20:32,612 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:20:32,790 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:20:32,791 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:20:33,048 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:20:38,049 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:20:42,552 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:20:43,553 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:20:47,612 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:20:47,800 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:20:47,800 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:20:49,058 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:20:54,059 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:20:59,059 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:21:02,611 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:21:02,800 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:21:02,800 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:21:05,055 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:21:10,055 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:21:12,552 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:21:15,553 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:21:17,611 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:21:17,800 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:21:17,801 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:21:21,041 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:21:26,042 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:21:31,043 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:21:32,612 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:21:32,800 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:21:32,801 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:21:36,048 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:21:41,049 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:21:42,553 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:21:46,554 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:21:47,612 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:21:47,800 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:21:47,801 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:21:52,000 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:21:57,001 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:22:02,002 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:22:02,612 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:22:02,801 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:22:02,801 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:22:07,055 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:22:12,060 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:22:12,564 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:22:17,564 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:22:17,612 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:22:17,801 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:22:17,803 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:22:23,046 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:22:28,047 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:22:32,612 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:22:32,801 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:22:32,801 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:22:33,968 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:22:38,969 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:22:42,564 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:22:44,565 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:22:47,612 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:22:47,801 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:22:47,802 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:22:49,999 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:22:55,010 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:23:00,011 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:23:02,612 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:23:02,801 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:23:02,802 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:23:05,967 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:23:10,967 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:23:12,564 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:23:16,566 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:23:17,613 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:23:17,802 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:23:17,802 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:23:21,972 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:23:26,973 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:23:31,973 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:23:32,613 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:23:32,802 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:23:32,803 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:23:36,976 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:23:41,977 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:23:42,565 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:23:47,566 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:23:47,613 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:23:47,802 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:23:47,802 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:23:52,965 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:23:57,966 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:24:02,613 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:24:02,802 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:24:02,803 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:24:03,061 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:24:08,062 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:24:12,565 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:24:13,566 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:24:17,613 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:24:17,802 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:24:17,802 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:24:19,012 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:24:24,013 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:24:29,013 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:24:32,613 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:24:32,802 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:24:32,803 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:24:34,027 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:24:39,028 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:24:42,566 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:24:44,566 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:24:47,613 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:24:47,810 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:24:47,810 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:24:49,977 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:24:54,978 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:24:59,993 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:25:02,614 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:25:02,816 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:25:02,817 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:25:05,010 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:25:10,010 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:25:12,566 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:25:15,567 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:25:17,613 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:25:17,816 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:25:17,817 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:25:21,068 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:25:26,068 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:25:31,077 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:25:32,614 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:25:32,816 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:25:32,817 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:25:37,039 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:25:42,040 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:25:42,567 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:25:47,568 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:25:47,614 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:25:47,817 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:25:47,817 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:25:52,984 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:25:57,990 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:26:02,614 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:26:02,817 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:26:02,817 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:26:03,976 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:26:08,976 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:26:12,567 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:26:14,568 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:26:17,614 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:26:17,817 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:26:17,817 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:26:20,052 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:26:25,053 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:26:30,054 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:26:32,614 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:26:32,817 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:26:32,818 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:26:35,976 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:26:40,977 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:26:42,578 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:26:46,579 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:26:47,614 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:26:47,817 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:26:47,818 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:26:51,995 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:26:56,995 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:27:01,996 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:27:02,614 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:27:02,817 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:27:02,818 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:27:07,040 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:27:12,040 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:27:12,568 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:27:17,569 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:27:17,615 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:27:17,817 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:27:17,818 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:27:23,064 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:27:28,065 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:27:32,615 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:27:32,817 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:27:32,818 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:27:34,012 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:27:39,013 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:27:42,569 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:27:44,570 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:27:47,615 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:27:47,817 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:27:47,818 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:27:50,061 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:27:55,062 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:28:00,062 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:28:02,615 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:28:02,818 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:28:02,818 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:28:05,987 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:28:08,233 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 04:28:11,652 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:28:12,569 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:28:17,570 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:28:17,615 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:28:17,818 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:28:17,818 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:28:23,063 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:28:28,065 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:28:32,615 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:28:32,818 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:28:32,818 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:28:33,977 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:28:38,978 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:28:42,570 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:28:44,571 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:28:47,615 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:28:47,818 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:28:47,818 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:28:50,056 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:28:55,057 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:29:00,058 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:29:02,615 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:29:02,818 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:29:02,818 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:29:06,022 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:29:11,022 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:29:12,570 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:29:16,571 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:29:17,615 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:29:17,818 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:29:17,818 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:29:22,016 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:29:27,017 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:29:32,018 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:29:32,616 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:29:32,818 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:29:32,819 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:29:37,081 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:29:42,081 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:29:42,571 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:29:47,572 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:29:47,616 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:29:47,818 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:29:47,819 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:29:53,030 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:29:58,030 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:30:02,616 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:30:02,818 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:30:02,819 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:30:03,047 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:30:08,047 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:30:12,571 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:30:13,572 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:30:17,616 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:30:17,819 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:30:17,819 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:30:19,023 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:30:24,023 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:30:29,030 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:30:32,616 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:30:32,819 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:30:32,819 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:30:34,986 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:30:39,986 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:30:42,572 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:30:45,573 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:30:47,616 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:30:47,819 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:30:47,820 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:30:51,051 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:30:56,052 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:31:01,052 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:31:02,616 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:31:02,819 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:31:02,819 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:31:06,086 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:31:11,087 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:31:12,572 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:31:16,573 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:31:17,616 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:31:17,819 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:31:17,820 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:31:22,017 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:31:27,028 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:31:32,029 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:31:32,616 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:31:32,819 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:31:32,819 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:31:37,053 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:31:42,053 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:31:42,573 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:31:47,574 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:31:47,616 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:31:47,819 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:31:47,820 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:31:53,031 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:31:58,031 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:32:02,617 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:32:02,819 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:32:02,820 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:32:03,052 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:32:08,063 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:32:12,573 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:32:13,574 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:32:17,617 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:32:17,819 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:32:17,820 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:32:19,024 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:32:24,617 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:32:25,211 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 04:32:29,618 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:32:32,617 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:32:32,820 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:32:32,820 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:32:34,974 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:32:39,975 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:32:42,574 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:32:45,575 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:32:47,617 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:32:47,820 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:32:47,820 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:32:51,028 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:32:56,028 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:33:01,029 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:33:02,617 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:33:02,820 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:33:02,820 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:33:06,083 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:33:11,083 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:33:12,574 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:33:16,575 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:33:17,617 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:33:17,820 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:33:17,820 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:33:22,028 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:33:27,029 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:33:32,029 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:33:32,617 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:33:32,820 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:33:32,820 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:33:37,074 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:33:42,074 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:33:42,575 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:33:47,576 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:33:47,617 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:33:47,820 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:33:47,821 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:33:53,064 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:33:58,065 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:34:02,618 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:34:02,820 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:34:02,821 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:34:04,058 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:34:09,060 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:34:12,576 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:34:14,576 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:34:17,617 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:34:17,820 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:34:17,821 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:34:20,043 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:34:25,044 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:34:30,044 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:34:32,617 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:34:32,820 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:34:32,821 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:34:36,000 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:34:41,010 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:34:42,576 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:34:46,577 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:34:47,617 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:34:47,820 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:34:47,821 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:34:51,984 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:34:56,985 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:35:01,985 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:35:02,618 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:35:02,821 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:35:02,821 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:35:07,007 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:35:12,010 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:35:12,576 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:35:17,577 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:35:17,618 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:35:17,821 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:35:17,821 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:35:23,056 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:35:28,057 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:35:32,618 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:35:32,821 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:35:32,821 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:35:34,017 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:35:39,017 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:35:42,577 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:35:44,578 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:35:47,618 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:35:47,821 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:35:47,821 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:35:50,057 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:35:55,057 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:36:00,058 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:36:02,618 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:36:02,821 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:36:02,821 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:36:06,048 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:36:11,052 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:36:12,577 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:36:16,578 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:36:17,618 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:36:17,821 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:36:17,821 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:36:22,033 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:36:27,034 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:36:32,034 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:36:32,618 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:36:32,821 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:36:32,822 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:36:37,061 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:36:42,061 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:36:42,578 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:36:47,579 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:36:47,619 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:36:47,822 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:36:47,823 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:36:53,034 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:36:58,035 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:37:02,619 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:37:02,821 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:37:02,822 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:37:03,999 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:37:08,999 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:37:12,578 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:37:14,581 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:37:17,619 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:37:17,821 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:37:17,822 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:37:20,018 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:37:25,020 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:37:30,028 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:37:32,619 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:37:32,821 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:37:32,822 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:37:35,040 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:37:40,041 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:37:42,579 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:37:45,590 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:37:47,619 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:37:47,821 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:37:47,822 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:37:51,000 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:37:56,000 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:38:01,001 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:38:02,619 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:38:02,822 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:38:02,822 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:38:06,070 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:38:11,071 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:38:12,579 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:38:16,580 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:38:17,619 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:38:17,822 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:38:17,822 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:38:21,982 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:38:26,983 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:38:31,984 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:38:32,619 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:38:32,822 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:38:32,822 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:38:37,077 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:38:42,078 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:38:42,580 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:38:47,581 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:38:47,619 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:38:47,822 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:38:47,830 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:38:53,046 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:38:58,046 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:39:02,620 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:39:02,822 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:39:02,823 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:39:04,045 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:39:09,050 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:39:12,580 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:39:14,581 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:39:17,620 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:39:17,822 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:39:17,823 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:39:19,996 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:39:24,997 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:39:29,998 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:39:32,620 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:39:32,822 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:39:32,823 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:39:35,003 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:39:40,003 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:39:42,581 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:39:45,590 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:39:47,620 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:39:47,822 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:39:47,823 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:39:50,991 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:39:56,000 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:40:01,008 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:40:02,620 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:40:02,822 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:40:02,823 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:40:06,077 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:40:11,078 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:40:12,581 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:40:16,582 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:40:17,620 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:40:17,822 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:40:17,823 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:40:22,038 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:40:27,039 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:40:32,039 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:40:32,620 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:40:32,823 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:40:32,823 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:40:37,058 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:40:41,990 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 04:40:42,161 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:40:42,582 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:40:47,583 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:40:47,620 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:40:47,823 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:40:47,823 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:40:53,061 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:40:58,061 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:41:02,621 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:41:02,823 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:41:02,823 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:41:03,073 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:41:08,074 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:41:12,609 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:41:13,609 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:41:17,621 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:41:17,823 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:41:17,824 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:41:19,030 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:41:24,040 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:41:29,040 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:41:32,621 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:41:32,823 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:41:32,824 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:41:35,029 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:41:40,029 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:41:42,583 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:41:45,584 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:41:47,621 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:41:47,823 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:41:47,823 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:41:51,090 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:41:56,091 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:42:01,092 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:42:02,621 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:42:02,823 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:42:02,824 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:42:07,080 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:42:12,080 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:42:12,583 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:42:17,585 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:42:17,621 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:42:17,823 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:42:17,824 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:42:23,041 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:42:28,050 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:42:32,621 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:42:32,823 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:42:32,824 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:42:33,085 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:42:38,086 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:42:42,584 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:42:43,584 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:42:47,621 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:42:47,824 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:42:47,824 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:42:49,027 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:42:54,028 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:42:59,029 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:43:02,621 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:43:02,824 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:43:02,824 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:43:04,984 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:43:09,984 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:43:12,584 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:43:15,585 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:43:17,622 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:43:17,824 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:43:17,824 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:43:21,137 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:43:26,137 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:43:31,138 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:43:32,622 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:43:32,824 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:43:32,825 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:43:37,037 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:43:42,038 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:43:42,585 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:43:47,586 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:43:47,622 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:43:47,824 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:43:47,824 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:43:53,085 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:43:58,086 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:44:02,622 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:44:02,834 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:44:02,835 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:44:04,079 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:44:09,080 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:44:12,585 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:44:14,586 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:44:17,622 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:44:17,834 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:44:17,835 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:44:20,046 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:44:25,052 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:44:30,052 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:44:32,622 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:44:32,838 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:44:32,839 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:44:35,057 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:44:40,058 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:44:42,586 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:44:45,586 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:44:47,622 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:44:47,838 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:44:47,839 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:44:51,053 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:44:56,054 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:45:01,055 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:45:02,622 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:45:02,838 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:45:02,839 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:45:06,096 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:45:11,101 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:45:12,586 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:45:16,587 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:45:17,622 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:45:17,838 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:45:17,839 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:45:22,075 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:45:27,076 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:45:32,076 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:45:32,622 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:45:32,839 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:45:32,839 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:45:37,997 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:45:42,586 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:45:43,587 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:45:47,622 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:45:47,839 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:45:47,839 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:45:49,040 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:45:54,041 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:45:59,042 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:46:02,623 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:46:02,839 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:46:02,840 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:46:04,065 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:46:09,066 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:46:12,587 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:46:14,588 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:46:17,623 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:46:17,839 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:46:17,840 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:46:20,057 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:46:25,057 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:46:30,058 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:46:32,623 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:46:32,839 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:46:32,840 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:46:36,046 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:46:41,046 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:46:42,587 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:46:46,588 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:46:47,623 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:46:47,839 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:46:47,840 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:46:52,001 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:46:57,009 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:47:02,010 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:47:02,623 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:47:02,839 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:47:02,840 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:47:07,063 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:47:12,063 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:47:12,588 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:47:17,589 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:47:17,624 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:47:17,839 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:47:17,840 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:47:23,104 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:47:28,105 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:47:32,624 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:47:32,840 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:47:32,840 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:47:34,017 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:47:39,028 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:47:42,589 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:47:44,589 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:47:47,624 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:47:47,840 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:47:47,841 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:47:50,099 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:47:55,100 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:48:00,102 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:48:02,624 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:48:02,840 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:48:02,841 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:48:06,049 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:48:11,050 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:48:12,589 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:48:16,590 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:48:17,624 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:48:17,840 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:48:17,841 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:48:22,046 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:48:27,047 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:48:32,047 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:48:32,624 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:48:32,840 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:48:32,841 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:48:37,075 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:48:42,076 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:48:42,590 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:48:47,591 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:48:47,624 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:48:47,840 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:48:47,841 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:48:53,026 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:48:58,655 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:48:59,814 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 04:49:02,624 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:49:02,841 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:49:02,844 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:49:04,003 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:49:09,004 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:49:12,590 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:49:14,591 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:49:17,624 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:49:17,841 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:49:17,841 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:49:20,065 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:49:25,066 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:49:30,066 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:49:32,624 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:49:32,849 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:49:32,849 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:49:36,053 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:49:41,053 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:49:42,590 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:49:46,592 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:49:47,624 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:49:47,849 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:49:47,849 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:49:52,115 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:49:57,116 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:50:02,137 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:50:02,625 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:50:02,849 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:50:02,849 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:50:08,071 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:50:12,591 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:50:13,592 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:50:17,625 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:50:17,849 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:50:17,850 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:50:19,018 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:50:24,019 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:50:29,020 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:50:32,625 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:50:32,849 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:50:32,849 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:50:34,055 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:50:39,056 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:50:42,592 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:50:44,593 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:50:47,625 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:50:47,849 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:50:47,850 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:50:50,042 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:50:55,042 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:51:00,043 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:51:02,633 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:51:02,849 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:51:02,849 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:51:06,041 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:51:11,041 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:51:12,599 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:51:16,600 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:51:17,633 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:51:17,849 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:51:17,850 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:51:22,010 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:51:27,020 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:51:32,020 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:51:32,633 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:51:32,849 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:51:32,850 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:51:37,036 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:51:42,036 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:51:42,592 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:51:47,594 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:51:47,633 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:51:47,849 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:51:47,850 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:51:53,123 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:51:58,123 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:52:02,633 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:52:02,849 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:52:02,850 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:52:04,034 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:52:09,034 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:52:12,593 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:52:14,593 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:52:17,633 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:52:17,850 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:52:17,850 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:52:20,105 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:52:25,106 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:52:30,107 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:52:32,633 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:52:32,850 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:52:32,850 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:52:36,031 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:52:41,031 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:52:42,593 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:52:46,594 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:52:47,633 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:52:47,850 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:52:47,850 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:52:52,038 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:52:57,038 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:53:02,039 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:53:02,633 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:53:02,850 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:53:02,850 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:53:07,041 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:53:12,042 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:53:12,594 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:53:17,595 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:53:17,633 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:53:17,850 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:53:17,850 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:53:23,090 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:53:28,091 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:53:32,634 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:53:32,850 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:53:32,850 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:53:34,045 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:53:39,048 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:53:42,595 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:53:44,595 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:53:47,634 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:53:47,850 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:53:47,851 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:53:50,108 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:53:54,904 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 04:53:55,547 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:54:00,550 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:54:02,634 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:54:02,850 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:54:02,851 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:54:06,035 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:54:11,036 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:54:12,595 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:54:16,596 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:54:17,634 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:54:17,850 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:54:17,851 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:54:22,038 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:54:27,042 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:54:32,052 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:54:32,634 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:54:32,850 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:54:32,851 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:54:37,077 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:54:42,078 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:54:42,595 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:54:47,596 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:54:47,634 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:54:47,850 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:54:47,851 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:54:53,034 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:54:58,040 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:55:02,634 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:55:02,851 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:55:02,851 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:55:04,020 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:55:09,020 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:55:12,597 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:55:14,598 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:55:17,634 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:55:17,851 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:55:17,851 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:55:20,121 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:55:25,122 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:55:30,122 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:55:32,634 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:55:32,851 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:55:32,851 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:55:36,028 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:55:41,029 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:55:42,597 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:55:46,598 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:55:47,635 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:55:47,851 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:55:47,852 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:55:52,101 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:55:57,101 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:56:02,102 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:56:02,635 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:56:02,851 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:56:02,852 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:56:08,060 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:56:12,609 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:56:13,610 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:56:17,635 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:56:17,852 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:56:17,852 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:56:19,068 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:56:24,069 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:56:29,069 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:56:32,635 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:56:32,852 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:56:32,852 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:56:34,122 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:56:39,122 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:56:42,598 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:56:44,599 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:56:47,635 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:56:47,852 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:56:47,852 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:56:50,070 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:56:55,070 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:57:00,071 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:57:02,635 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:57:02,852 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:57:02,852 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:57:05,090 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:57:10,091 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:57:12,599 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:57:15,599 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:57:17,635 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:57:17,852 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:57:17,852 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:57:21,104 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:57:26,105 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:57:31,106 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:57:32,635 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:57:32,852 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:57:32,853 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:57:36,110 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:57:41,110 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:57:42,599 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:57:46,600 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:57:47,635 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:57:47,852 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:57:47,853 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:57:52,093 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:57:57,094 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:58:02,095 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:58:02,636 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:58:02,852 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:58:02,853 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:58:08,042 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:58:12,600 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:58:13,610 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:58:17,636 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:58:17,852 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:58:17,853 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:58:19,112 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:58:24,113 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:58:29,113 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:58:32,636 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:58:32,852 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:58:32,853 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:58:35,060 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:58:40,061 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:58:42,600 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:58:45,601 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:58:47,636 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:58:47,853 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:58:47,853 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:58:51,022 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:58:56,023 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:59:01,023 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:59:02,636 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:59:02,853 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:59:02,853 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:59:06,097 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:59:11,097 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:59:12,610 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:59:16,611 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:59:17,636 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:59:17,853 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:59:17,853 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:59:22,075 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:59:27,076 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:59:32,077 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:59:32,636 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:59:32,853 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:59:32,853 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:59:38,067 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:59:42,611 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 04:59:43,611 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:59:47,636 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 04:59:47,853 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 04:59:47,853 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 04:59:49,032 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:59:54,033 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 04:59:59,033 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:00:02,636 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:00:02,853 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:00:02,854 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:00:04,121 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:00:09,121 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:00:12,611 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:00:14,612 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:00:17,636 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:00:17,854 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:00:17,855 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:00:20,317 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:00:25,330 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:00:30,330 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:00:32,638 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:00:32,853 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:00:32,854 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:00:36,104 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:00:41,109 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:00:42,612 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:00:46,613 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:00:47,638 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:00:47,860 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:00:47,860 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:00:52,122 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:00:57,122 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:01:02,123 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:01:02,638 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:01:02,860 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:01:02,861 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:01:08,012 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:01:12,612 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:01:13,613 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:01:17,638 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:01:17,860 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:01:17,861 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:01:19,017 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:01:24,018 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:01:29,019 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:01:32,638 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:01:32,862 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:01:32,862 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:01:34,047 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:01:39,047 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:01:42,612 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:01:44,613 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:01:47,638 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:01:47,862 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:01:47,862 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:01:50,031 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:01:55,031 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:02:00,032 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:02:02,639 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:02:02,862 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:02:02,862 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:02:05,111 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:02:10,115 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:02:12,614 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:02:15,615 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:02:17,639 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:02:17,862 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:02:17,863 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:02:21,033 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:02:26,033 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:02:31,034 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:02:32,639 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:02:32,862 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:02:32,863 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:02:36,055 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:02:41,056 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:02:42,614 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:02:46,615 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:02:47,639 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:02:47,862 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:02:47,863 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:02:52,147 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:02:57,147 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:03:02,148 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:03:02,639 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:03:02,863 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:03:02,863 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:03:08,055 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:03:12,614 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:03:13,615 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:03:17,639 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:03:17,863 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:03:17,863 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:03:19,039 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:03:24,040 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:03:29,041 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:03:32,639 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:03:32,863 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:03:32,864 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:03:34,065 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:03:39,065 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:03:42,615 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:03:44,615 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:03:47,639 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:03:47,863 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:03:47,863 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:03:50,051 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:03:55,052 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:04:00,052 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:04:02,640 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:04:02,863 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:04:02,863 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:04:05,070 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:04:10,071 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:04:12,615 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:04:15,616 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:04:17,640 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:04:17,870 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:04:17,870 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:04:21,107 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:04:26,108 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:04:31,116 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:04:32,640 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:04:32,870 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:04:32,870 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:04:37,058 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:04:42,058 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:04:42,616 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:04:47,617 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:04:47,640 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:04:47,870 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:04:47,870 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:04:53,120 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:04:58,120 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:05:02,640 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:05:02,870 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:05:02,871 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:05:03,136 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:05:08,136 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:05:12,616 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:05:13,617 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:05:17,640 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:05:17,870 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:05:17,871 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:05:19,055 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:05:24,069 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:05:29,751 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:05:31,402 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 05:05:32,640 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:05:32,870 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:05:32,871 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:05:35,280 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:05:40,280 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:05:42,617 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:05:45,618 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:05:47,640 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:05:47,870 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:05:47,871 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:05:51,070 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:05:56,070 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:06:01,071 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:06:02,640 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:06:02,871 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:06:02,871 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:06:07,053 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:06:12,053 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:06:12,617 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:06:17,618 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:06:17,640 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:06:17,871 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:06:17,871 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:06:23,066 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:06:28,067 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:06:32,640 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:06:32,871 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:06:32,872 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:06:33,621 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 05:06:34,046 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:06:39,047 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:06:42,618 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:06:44,618 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:06:47,640 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:06:47,871 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:06:47,871 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:06:50,042 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:06:55,043 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:07:00,044 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:07:02,641 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:07:02,871 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:07:02,871 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:07:05,113 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:07:10,114 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:07:12,619 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:07:15,619 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:07:17,641 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:07:17,871 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:07:17,871 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:07:21,057 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:07:26,058 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:07:31,058 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:07:32,641 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:07:32,871 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:07:32,872 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:07:36,102 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:07:41,102 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:07:42,619 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:07:46,620 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:07:47,649 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:07:47,871 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:07:47,872 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:07:52,110 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:07:57,111 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:08:02,120 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:08:02,649 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:08:02,871 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:08:02,872 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:08:08,030 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:08:12,619 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:08:13,620 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:08:17,649 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:08:17,871 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:08:17,872 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:08:19,120 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:08:24,120 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:08:29,130 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:08:32,650 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:08:32,871 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:08:32,872 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:08:35,070 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:08:40,071 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:08:42,620 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:08:45,621 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:08:47,650 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:08:47,872 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:08:47,872 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:08:51,116 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:08:56,117 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:09:01,118 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:09:02,650 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:09:02,872 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:09:02,872 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:09:07,058 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:09:12,063 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:09:12,620 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:09:17,622 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:09:17,650 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:09:17,872 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:09:17,872 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:09:23,065 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:09:28,066 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:09:32,650 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:09:32,872 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:09:32,873 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:09:33,105 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:09:38,106 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:09:42,621 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:09:43,621 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:09:47,650 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:09:47,872 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:09:47,873 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:09:49,067 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:09:54,070 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:09:59,070 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:10:02,650 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:10:02,872 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:10:02,873 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:10:05,048 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:10:10,049 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:10:12,621 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:10:15,622 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:10:17,650 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:10:17,872 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:10:17,873 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:10:21,120 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:10:26,121 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:10:31,122 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:10:32,651 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:10:32,872 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:10:32,873 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:10:37,094 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:10:42,094 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:10:42,621 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:10:47,631 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:10:47,651 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:10:47,873 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:10:47,873 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:10:53,129 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:10:58,130 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:11:02,651 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:11:02,875 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:11:02,876 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:11:04,075 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:11:09,076 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:11:12,633 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:11:14,633 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:11:17,660 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:11:17,875 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:11:17,876 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:11:20,052 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:11:25,056 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:11:30,056 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:11:32,660 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:11:32,875 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:11:32,876 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:11:35,109 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:11:40,110 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:11:42,622 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:11:45,623 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:11:47,660 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:11:47,875 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:11:47,876 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:11:51,026 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:11:56,027 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:12:01,030 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:12:02,660 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:12:02,876 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:12:02,876 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:12:06,069 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:12:11,070 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:12:12,623 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:12:16,623 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:12:17,660 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:12:17,876 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:12:17,876 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:12:22,060 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:12:27,060 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:12:32,070 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:12:32,660 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:12:32,876 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:12:32,876 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:12:37,115 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:12:42,116 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:12:42,623 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:12:47,625 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:12:47,660 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:12:47,876 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:12:47,876 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:12:53,073 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:12:58,080 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:13:02,660 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:13:02,876 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:13:02,876 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:13:04,070 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:13:09,071 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:13:12,624 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:13:14,625 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:13:17,660 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:13:17,892 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:13:17,893 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:13:20,103 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:13:25,113 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:13:30,114 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:13:32,660 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:13:32,893 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:13:32,893 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:13:36,061 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:13:41,061 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:13:42,624 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:13:46,625 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:13:47,661 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:13:47,893 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:13:47,893 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:13:52,165 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:13:57,165 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:14:02,166 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:14:02,661 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:14:02,893 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:14:02,893 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:14:08,100 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:14:12,625 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:14:13,625 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:14:17,661 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:14:17,893 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:14:17,893 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:14:19,093 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:14:24,094 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:14:29,094 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:14:32,661 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:14:32,893 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:14:32,894 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:14:35,065 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:14:40,066 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:14:42,625 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:14:45,626 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:14:47,661 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:14:47,893 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:14:47,894 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:14:51,095 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:14:56,096 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:15:01,097 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:15:02,661 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:15:02,893 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:15:02,894 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:15:06,103 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:15:11,104 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:15:12,626 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:15:16,627 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:15:17,661 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:15:17,893 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:15:17,894 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:15:22,058 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:15:27,058 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:15:32,059 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:15:32,661 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:15:32,893 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:15:32,894 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:15:37,061 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:15:42,062 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:15:42,626 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:15:47,627 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:15:47,661 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:15:47,893 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:15:47,894 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:15:53,138 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:15:58,139 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:16:02,662 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:16:02,894 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:16:02,894 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:16:03,147 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:16:08,148 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:16:12,627 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:16:13,628 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:16:17,662 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:16:17,894 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:16:17,894 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:16:19,110 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:16:24,110 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:16:29,120 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:16:32,662 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:16:32,900 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:16:32,900 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:16:35,119 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:16:40,128 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:16:42,627 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:16:45,636 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:16:47,662 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:16:47,900 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:16:47,900 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:16:51,119 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:16:56,120 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:17:01,121 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:17:02,662 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:17:02,900 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:17:02,900 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:17:07,072 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:17:12,073 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:17:12,627 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:17:17,628 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:17:17,662 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:17:17,900 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:17:17,901 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:17:23,137 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:17:28,138 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:17:32,662 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:17:32,900 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:17:32,901 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:17:33,149 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:17:38,150 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:17:42,628 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:17:43,629 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:17:47,662 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:17:47,900 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:17:47,901 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:17:49,126 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:17:54,127 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:17:59,127 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:18:02,662 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:18:02,900 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:18:02,901 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:18:05,113 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:18:10,114 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:18:12,628 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:18:15,629 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:18:17,662 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:18:17,900 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:18:17,901 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:18:21,160 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:18:26,161 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:18:31,161 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:18:32,662 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:18:32,901 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:18:32,901 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:18:37,058 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:18:42,059 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:18:42,629 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:18:47,630 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:18:47,663 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:18:47,901 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:18:47,901 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:18:53,103 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:18:58,120 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:19:02,663 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:19:02,901 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:19:02,901 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:19:03,156 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:19:08,157 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:19:12,630 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:19:13,630 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:19:17,663 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:19:17,901 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:19:17,902 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:19:19,156 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:19:24,156 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:19:29,157 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:19:32,663 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:19:32,901 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:19:32,902 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:19:35,123 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:19:40,132 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:19:42,630 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:19:45,559 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:19:47,663 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:19:47,718 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 05:19:47,901 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:19:47,902 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:19:51,086 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:19:56,087 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:20:01,088 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:20:02,663 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:20:02,901 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:20:02,902 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:20:06,109 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:20:11,117 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:20:12,630 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:20:16,631 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:20:17,663 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:20:17,901 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:20:17,902 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:20:22,112 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:20:27,113 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:20:32,116 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:20:32,663 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:20:32,901 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:20:32,902 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:20:37,160 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:20:42,161 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:20:42,631 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:20:47,632 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:20:47,663 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:20:47,901 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:20:47,902 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:20:53,060 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:20:58,061 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:21:02,664 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:21:02,902 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:21:02,902 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:21:03,156 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:21:08,157 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:21:12,631 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:21:13,632 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:21:17,664 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:21:17,909 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:21:17,910 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:21:19,104 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:21:24,105 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:21:29,105 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:21:32,664 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:21:32,909 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:21:32,910 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:21:35,089 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:21:40,090 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:21:42,632 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:21:45,632 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:21:47,664 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:21:47,909 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:21:47,910 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:21:51,100 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:21:56,101 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:22:01,102 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:22:02,664 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:22:02,910 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:22:02,910 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:22:06,140 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:22:11,150 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:22:12,632 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:22:16,640 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:22:17,664 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:22:17,910 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:22:17,910 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:22:22,175 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:22:27,175 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:22:32,176 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:22:32,664 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:22:32,910 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:22:32,910 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:22:36,297 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 05:22:37,629 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:22:42,630 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:22:42,632 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:22:47,633 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:22:47,672 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:22:47,910 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:22:47,910 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:22:53,105 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:22:58,106 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:23:02,673 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:23:02,910 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:23:02,910 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:23:03,136 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:23:08,137 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:23:12,633 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:23:13,634 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:23:17,673 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:23:17,910 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:23:17,910 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:23:19,153 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:23:24,153 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:23:29,157 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:23:32,673 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:23:32,910 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:23:32,910 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:23:34,167 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:23:39,167 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:23:42,633 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:23:44,634 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:23:47,673 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:23:47,910 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:23:47,911 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:23:50,130 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:23:55,130 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:24:00,131 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:24:02,673 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:24:02,910 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:24:02,911 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:24:05,165 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:24:10,166 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:24:12,634 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:24:15,635 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:24:17,673 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:24:17,910 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:24:17,911 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:24:21,127 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:24:26,128 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:24:31,128 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:24:32,674 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:24:32,911 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:24:32,911 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:24:37,077 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:24:42,078 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:24:42,634 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:24:47,635 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:24:47,673 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:24:47,911 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:24:47,911 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:24:53,142 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:24:58,143 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:25:02,674 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:25:02,911 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:25:02,911 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:25:04,126 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:25:09,126 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:25:12,635 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:25:14,636 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:25:17,674 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:25:17,911 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:25:17,911 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:25:20,132 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:25:25,140 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:25:30,140 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:25:32,674 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:25:32,911 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:25:32,912 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:25:35,152 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:25:40,153 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:25:42,635 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:25:45,636 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:25:47,674 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:25:47,911 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:25:47,912 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:25:51,150 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:25:56,150 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:26:01,151 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:26:02,674 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:26:02,911 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:26:02,912 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:26:07,140 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:26:12,141 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:26:12,636 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:26:17,637 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:26:17,674 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:26:17,911 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:26:17,912 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:26:23,086 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:26:28,087 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:26:32,675 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:26:32,912 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:26:32,912 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:26:33,128 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:26:38,129 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:26:42,647 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:26:43,647 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:26:47,675 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:26:47,912 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:26:47,912 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:26:49,126 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:26:54,127 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:26:59,128 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:27:02,675 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:27:02,912 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:27:02,912 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:27:04,177 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:27:09,177 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:27:12,637 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:27:14,638 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:27:17,675 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:27:17,912 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:27:17,912 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:27:20,143 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:27:25,144 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:27:30,144 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:27:32,675 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:27:32,912 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:27:32,913 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:27:36,104 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:27:41,105 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:27:42,637 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:27:46,638 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:27:47,675 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:27:47,912 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:27:47,913 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:27:52,139 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:27:57,140 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:28:02,140 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:28:02,675 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:28:02,912 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:28:02,913 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:28:08,118 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:28:12,638 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:28:13,638 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:28:17,682 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:28:17,912 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:28:17,913 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:28:19,120 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:28:24,120 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:28:29,130 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:28:32,682 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:28:32,912 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:28:32,913 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:28:35,095 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:28:40,096 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:28:42,638 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:28:45,639 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:28:47,682 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:28:47,913 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:28:47,913 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:28:51,139 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:28:56,139 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:29:01,140 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:29:02,682 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:29:02,913 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:29:02,913 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:29:06,155 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:29:11,155 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:29:12,638 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:29:16,639 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:29:17,682 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:29:17,913 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:29:17,913 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:29:22,148 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:29:27,149 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:29:32,150 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:29:32,682 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:29:32,913 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:29:32,913 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:29:37,168 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:29:42,169 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:29:42,639 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:29:47,640 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:29:47,682 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:29:47,913 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:29:47,915 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:29:53,202 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:29:58,203 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:30:02,682 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:30:02,913 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:30:02,914 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:30:04,100 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:30:09,100 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:30:12,639 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:30:14,640 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:30:17,682 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:30:17,913 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:30:17,914 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:30:20,151 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:30:25,152 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:30:30,152 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:30:32,684 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:30:32,913 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:30:32,914 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:30:36,072 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:30:41,073 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:30:42,640 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:30:46,641 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:30:47,683 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:30:47,913 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:30:47,914 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:30:52,115 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:30:57,115 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:31:02,116 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:31:02,683 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:31:02,914 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:31:02,914 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:31:07,130 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:31:12,131 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:31:12,641 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:31:17,642 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:31:17,684 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:31:17,914 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:31:17,914 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:31:23,145 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:31:28,146 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:31:32,683 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:31:32,914 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:31:32,914 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:31:34,131 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:31:39,132 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:31:42,641 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:31:44,642 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:31:47,683 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:31:47,914 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:31:47,914 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:31:50,133 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:31:55,134 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:32:00,135 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:32:02,683 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:32:02,914 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:32:02,915 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:32:05,159 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:32:10,160 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:32:12,641 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:32:15,980 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:32:17,285 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 05:32:17,684 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:32:17,914 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:32:17,915 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:32:21,069 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:32:26,070 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:32:31,070 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:32:32,684 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:32:32,914 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:32:32,915 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:32:36,168 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:32:41,169 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:32:42,642 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:32:46,643 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:32:47,684 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:32:47,914 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:32:47,915 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:32:52,077 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:32:57,078 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:33:02,089 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:33:02,693 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:33:02,914 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:33:02,915 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:33:07,120 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:33:12,121 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:33:12,643 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:33:17,644 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:33:17,693 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:33:17,915 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:33:17,915 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:33:23,104 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:33:28,110 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:33:32,694 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:33:32,915 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:33:32,915 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:33:34,094 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:33:39,094 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:33:42,643 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:33:44,644 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:33:47,694 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:33:47,915 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:33:47,915 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:33:50,092 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:33:55,093 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:34:00,100 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:34:02,695 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:34:02,915 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:34:02,915 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:34:05,143 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:34:10,143 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:34:12,643 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:34:15,644 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:34:17,694 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:34:17,915 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:34:17,915 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:34:21,152 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:34:26,154 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:34:31,155 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:34:32,694 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:34:32,915 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:34:32,916 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:34:36,168 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:34:41,169 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:34:42,644 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:34:46,650 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:34:47,694 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:34:47,915 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:34:47,916 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:34:52,110 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:34:57,111 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:35:02,112 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:35:02,694 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:35:02,915 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:35:02,916 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:35:08,087 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:35:12,644 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:35:13,645 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:35:17,694 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:35:17,916 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:35:17,917 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:35:19,167 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:35:24,167 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:35:29,168 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:35:32,695 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:35:32,916 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:35:32,916 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:35:35,168 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:35:40,169 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:35:42,645 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:35:45,646 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:35:47,695 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:35:47,916 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:35:47,917 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:35:51,088 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:35:56,089 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:36:01,090 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:36:02,695 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:36:02,916 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:36:02,917 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:36:06,172 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:36:11,173 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:36:12,654 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:36:16,655 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:36:17,695 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:36:17,916 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:36:17,916 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:36:22,143 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:36:27,144 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:36:32,145 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:36:32,695 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:36:32,916 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:36:32,916 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:36:38,110 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:36:42,654 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:36:43,655 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:36:47,695 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:36:47,916 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:36:47,917 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:36:49,100 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:36:54,101 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:36:59,101 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:37:02,695 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:37:02,916 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:37:02,917 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:37:05,083 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:37:10,097 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:37:12,655 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:37:15,655 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:37:17,695 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:37:17,916 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:37:17,917 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:37:21,124 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:37:26,125 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:37:31,134 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:37:32,695 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:37:32,916 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:37:32,917 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:37:37,125 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:37:42,125 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:37:42,656 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:37:47,656 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:37:47,696 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:37:47,916 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:37:47,917 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:37:53,072 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:37:58,080 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:38:02,696 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:38:02,917 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:38:02,917 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:38:03,131 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:38:08,132 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:38:12,656 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:38:13,656 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:38:17,696 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:38:17,917 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:38:17,917 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:38:19,120 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:38:24,120 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:38:29,122 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:38:32,696 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:38:32,917 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:38:32,917 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:38:34,162 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:38:39,170 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:38:42,656 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:38:44,657 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:38:47,696 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:38:47,917 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:38:47,918 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:38:50,120 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:38:55,121 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:39:00,121 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:39:02,696 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:39:02,917 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:39:02,918 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:39:05,157 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:39:10,159 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:39:12,657 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:39:15,658 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:39:17,696 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:39:17,917 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:39:17,918 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:39:21,141 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:39:26,142 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:39:31,143 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:39:32,696 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:39:32,917 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:39:32,918 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:39:35,888 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 05:39:36,342 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:39:41,343 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:39:42,657 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:39:46,658 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:39:47,696 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:39:47,918 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:39:47,918 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:39:52,082 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:39:57,082 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:40:02,083 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:40:02,697 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:40:02,918 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:40:02,918 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:40:07,119 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:40:12,129 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:40:12,657 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:40:17,658 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:40:17,697 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:40:17,918 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:40:17,918 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:40:23,148 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:40:28,148 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:40:32,697 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:40:32,918 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:40:32,918 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:40:34,139 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:40:39,140 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:40:42,660 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:40:44,661 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:40:47,697 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:40:47,918 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:40:47,919 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:40:50,114 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:40:55,114 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:41:00,115 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:41:02,697 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:41:02,942 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:41:02,943 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:41:06,102 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:41:11,102 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:41:12,661 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:41:16,662 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:41:17,697 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:41:17,918 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:41:17,919 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:41:22,170 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:41:27,171 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:41:32,171 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:41:32,697 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:41:32,918 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:41:32,919 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:41:38,098 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:41:42,672 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:41:43,673 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:41:47,697 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:41:47,918 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:41:47,920 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:41:49,181 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:41:54,182 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:41:59,182 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:42:02,697 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:42:02,919 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:42:02,919 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:42:05,093 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:42:10,110 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:42:12,662 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:42:15,670 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:42:17,697 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:42:17,926 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:42:17,927 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:42:21,140 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:42:26,141 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:42:31,146 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:42:32,698 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:42:32,926 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:42:32,927 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:42:37,080 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:42:42,081 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:42:42,662 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:42:47,664 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:42:47,698 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:42:47,926 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:42:47,927 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:42:53,124 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:42:58,124 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:43:02,698 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:43:02,927 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:43:02,927 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:43:03,146 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:43:08,147 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:43:12,663 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:43:13,663 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:43:17,698 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:43:17,927 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:43:17,927 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:43:19,166 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:43:24,167 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:43:29,168 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:43:32,698 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:43:32,927 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:43:32,927 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:43:34,250 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:43:39,260 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:43:42,663 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:43:44,664 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:43:47,698 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:43:47,927 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:43:47,927 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:43:50,127 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:43:55,130 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:44:00,131 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:44:02,698 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:44:02,927 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:44:02,928 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:44:06,122 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:44:11,123 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:44:12,664 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:44:16,664 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:44:17,699 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:44:17,927 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:44:17,928 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:44:22,105 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:44:27,105 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:44:32,106 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:44:32,699 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:44:32,927 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:44:32,928 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:44:38,093 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:44:42,664 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:44:43,665 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:44:47,699 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:44:47,927 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:44:47,928 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:44:49,093 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:44:54,093 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:44:59,094 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:45:02,002 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 05:45:02,699 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:45:02,928 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:45:02,928 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:45:04,109 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:45:09,110 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:45:12,665 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:45:14,666 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:45:17,699 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:45:17,928 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:45:17,928 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:45:20,125 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:45:25,126 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:45:30,126 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:45:32,699 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:45:32,928 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:45:32,929 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:45:35,159 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:45:40,163 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:45:42,665 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:45:45,670 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:45:47,699 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:45:47,928 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:45:47,928 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:45:51,109 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:45:56,119 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:46:01,120 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:46:02,699 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:46:02,928 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:46:02,929 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:46:07,129 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:46:12,130 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:46:12,665 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:46:17,666 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:46:17,700 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:46:17,928 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:46:17,929 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:46:23,184 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:46:28,185 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:46:32,700 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:46:32,928 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:46:32,929 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:46:34,139 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:46:39,159 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:46:42,666 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:46:44,667 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:46:47,700 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:46:47,928 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:46:47,929 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:46:50,103 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:46:55,104 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:47:00,105 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:47:02,700 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:47:02,928 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:47:02,929 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:47:05,142 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:47:10,142 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:47:12,666 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:47:15,667 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:47:17,710 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:47:17,929 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:47:17,929 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:47:21,098 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:47:26,099 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:47:31,100 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:47:32,710 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:47:32,929 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:47:32,929 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:47:36,171 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:47:41,172 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:47:42,667 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:47:46,668 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:47:47,710 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:47:47,929 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:47:47,929 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:47:52,146 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:47:57,147 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:48:02,148 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:48:02,711 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:48:02,929 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:48:02,930 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:48:08,118 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:48:12,667 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:48:13,668 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:48:17,710 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:48:17,929 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:48:17,929 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:48:19,156 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:48:24,157 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:48:29,158 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:48:32,710 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:48:32,929 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:48:32,930 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:48:35,115 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:48:40,115 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:48:42,667 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:48:45,668 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:48:47,710 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:48:47,929 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:48:47,930 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:48:51,140 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:48:56,141 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:49:01,856 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 05:49:01,930 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:49:02,711 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:49:02,929 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:49:02,930 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:49:07,092 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:49:12,093 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:49:12,668 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:49:17,669 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:49:17,711 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:49:22,712 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:49:27,713 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:49:32,714 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:49:37,715 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:49:42,672 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:49:43,673 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:49:48,673 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:49:53,674 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:49:58,675 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:50:03,676 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:50:06,978 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:50:06,978 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:50:06,979 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:50:09,180 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:50:12,671 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:50:14,672 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:50:20,278 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 05:50:20,376 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:50:21,953 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:50:21,964 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:50:21,965 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:50:26,159 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:50:31,160 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:50:36,161 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:50:37,376 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:50:37,377 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:50:37,378 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:50:41,602 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:50:42,671 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:50:44,484 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 05:50:46,673 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:50:51,674 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:50:52,371 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:50:52,372 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:50:52,372 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:50:54,540 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 05:50:57,560 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:51:03,199 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:51:04,594 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 05:51:07,372 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:51:07,372 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:51:07,373 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:51:08,624 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:51:12,672 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:51:13,673 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:51:14,658 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 05:51:18,673 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:51:22,375 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:51:22,375 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:51:22,376 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:51:23,707 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 05:51:24,588 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:51:29,595 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:51:33,761 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 05:51:35,276 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:51:37,372 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:51:37,374 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:51:37,374 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:51:40,580 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:51:42,672 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:51:43,839 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 05:51:45,673 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:51:50,675 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:51:52,372 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:51:52,374 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:51:52,374 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:51:53,898 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 05:51:56,624 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:52:01,625 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:52:03,956 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 05:52:06,924 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:52:07,372 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:52:07,374 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:52:07,375 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:52:12,004 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 05:52:12,378 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:52:12,673 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:52:17,674 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:52:22,061 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 05:52:22,372 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:52:22,374 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:52:22,375 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:52:23,634 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:52:28,635 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:52:32,124 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 05:52:34,002 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:52:37,372 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:52:37,374 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:52:37,375 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:52:39,593 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:52:42,185 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 05:52:42,673 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:52:44,674 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:52:49,675 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:52:52,238 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 05:52:52,373 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:52:52,375 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:52:52,376 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:52:55,571 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:53:00,797 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:53:02,303 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 05:53:05,797 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:53:08,652 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:53:08,653 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:53:08,654 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:53:10,869 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:53:12,359 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 05:53:12,673 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:53:16,675 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:53:22,423 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 05:53:22,657 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:53:23,650 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:53:23,651 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:53:23,651 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:53:27,918 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:53:32,488 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 05:53:33,658 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:53:38,650 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:53:38,651 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:53:38,651 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:53:38,803 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:53:42,676 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:53:44,677 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:53:45,175 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 05:53:45,183 DEBUG SenderThread:3236 [sender.py:send():380] send: metric 2023-11-23 05:53:45,184 DEBUG SenderThread:3236 [sender.py:send():380] send: metric 2023-11-23 05:53:45,185 DEBUG SenderThread:3236 [sender.py:send():380] send: metric 2023-11-23 05:53:45,185 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 05:53:45,186 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 05:53:45,206 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 05:53:45,556 INFO Thread-12 :3236 [dir_watcher.py:_on_file_created():271] file/dir created: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 05:53:46,560 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 05:53:47,565 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 05:53:50,309 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:53:53,923 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:53:53,924 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:53:53,925 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:53:55,610 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:53:56,637 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/config.yaml 2023-11-23 05:53:58,647 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 05:54:00,882 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:54:05,883 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:54:08,714 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 05:54:08,920 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:54:08,936 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:54:08,955 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:54:11,170 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:54:12,675 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:54:17,162 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:54:18,774 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 05:54:22,163 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:54:24,698 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:54:24,699 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:54:24,699 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:54:27,933 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:54:28,852 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 05:54:32,934 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:54:36,903 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 05:54:38,323 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:54:39,697 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:54:39,698 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:54:39,698 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:54:42,692 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:54:43,694 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:54:46,966 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 05:54:48,848 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:54:53,849 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:54:54,697 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:54:54,697 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:54:54,698 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:54:56,087 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 05:54:58,915 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:55:03,917 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:55:06,165 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 05:55:09,697 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:55:09,698 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:55:09,699 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:55:09,699 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:55:12,678 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:55:15,664 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:55:16,228 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 05:55:20,664 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:55:24,697 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:55:24,698 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:55:24,698 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:55:25,854 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:55:26,294 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 05:55:30,854 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:55:36,086 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:55:36,356 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 05:55:39,698 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:55:39,698 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:55:39,699 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:55:41,934 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:55:42,679 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:55:46,426 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 05:55:47,128 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:55:52,130 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:55:54,698 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:55:54,699 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:55:54,700 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:55:56,492 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 05:55:57,890 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:56:02,891 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:56:04,548 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 05:56:08,216 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:56:09,699 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:56:09,701 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:56:09,702 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:56:12,679 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:56:13,680 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:56:14,629 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 05:56:18,681 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:56:24,007 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:56:24,697 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 05:56:24,698 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:56:24,700 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:56:24,700 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:56:29,960 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:56:32,749 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 05:56:35,721 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:56:39,977 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:56:39,979 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:56:39,980 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:56:41,151 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:56:42,680 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:56:42,811 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 05:56:46,681 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:56:52,605 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:56:52,888 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 05:56:54,975 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:56:54,976 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:56:54,976 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:56:58,195 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:57:00,030 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 05:57:00,031 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 05:57:00,032 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 05:57:00,033 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 05:57:00,939 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 05:57:01,944 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 05:57:02,950 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 05:57:04,035 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:57:09,037 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:57:09,975 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:57:09,976 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:57:09,976 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:57:10,998 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 05:57:12,681 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:57:14,681 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:57:20,367 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:57:21,059 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 05:57:24,975 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:57:24,976 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:57:24,977 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:57:26,166 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:57:31,125 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 05:57:32,088 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:57:37,090 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:57:39,976 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:57:39,976 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:57:39,977 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:57:41,194 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 05:57:42,163 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:57:42,681 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:57:47,682 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:57:50,254 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 05:57:52,912 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:57:56,133 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:57:56,134 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:57:56,135 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:57:58,320 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 05:57:58,400 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:58:03,401 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:58:08,385 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 05:58:08,824 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:58:11,133 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:58:11,133 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:58:11,134 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:58:12,682 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:58:14,683 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:58:18,460 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 05:58:19,712 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:58:24,713 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:58:26,133 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:58:26,133 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:58:26,134 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:58:28,519 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 05:58:30,325 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:58:35,326 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:58:36,576 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 05:58:40,513 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:58:41,133 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:58:41,133 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:58:41,134 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:58:42,683 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:58:45,801 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:58:46,640 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 05:58:50,801 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:58:56,133 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:58:56,133 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:58:56,134 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:58:56,362 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:58:56,702 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 05:59:01,363 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:59:04,755 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 05:59:06,708 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:59:11,710 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:59:12,467 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:59:12,468 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:59:12,470 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:59:12,685 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:59:14,856 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 05:59:16,733 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:59:21,734 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:59:24,917 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 05:59:27,201 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:59:27,467 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:59:27,467 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:59:27,468 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:59:32,597 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:59:34,988 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 05:59:37,598 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:59:42,468 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:59:42,469 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:59:42,470 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:59:42,680 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:59:42,684 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 05:59:45,050 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 05:59:48,138 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:59:53,139 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 05:59:55,108 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 05:59:57,467 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 05:59:57,468 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 05:59:57,469 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 05:59:58,630 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:00:04,067 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:00:05,177 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:00:09,068 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:00:12,471 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:00:12,473 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:00:12,473 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:00:12,723 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:00:13,640 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 06:00:13,641 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 06:00:13,641 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 06:00:13,642 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 06:00:14,228 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:00:14,228 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 06:00:14,643 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:00:15,232 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:00:19,644 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:00:24,905 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:00:25,299 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:00:27,467 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:00:27,469 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:00:27,469 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:00:30,630 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:00:35,362 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:00:36,215 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:00:41,217 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:00:42,468 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:00:42,469 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:00:42,470 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:00:42,685 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:00:43,414 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:00:46,687 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:00:51,688 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:00:54,485 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:00:57,340 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:00:57,468 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:00:57,469 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:00:57,470 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:01:02,446 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:01:04,545 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:01:07,447 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:01:12,449 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:01:12,469 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:01:12,476 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:01:12,477 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:01:12,723 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:01:14,624 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:01:17,831 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:01:23,518 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:01:24,698 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:01:27,468 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:01:27,470 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:01:27,470 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:01:28,687 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:01:33,825 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:01:34,766 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:01:38,826 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:01:42,471 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:01:42,472 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:01:42,472 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:01:42,687 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:01:44,415 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:01:44,826 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:01:49,415 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:01:54,824 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:01:54,950 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:01:57,469 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:01:57,471 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:01:57,471 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:02:00,688 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:02:06,241 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:02:07,033 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:02:11,242 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:02:12,688 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:02:13,207 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:02:13,209 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:02:13,209 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:02:17,016 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:02:17,095 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:02:22,017 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:02:27,094 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:02:27,170 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:02:28,205 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:02:28,206 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:02:28,206 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:02:32,455 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:02:37,234 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:02:37,498 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:02:42,500 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:02:42,689 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:02:43,208 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:02:43,210 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:02:43,210 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:02:47,300 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:02:47,769 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:02:52,771 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:02:57,397 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:02:58,206 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:02:58,207 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:02:58,207 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:02:58,415 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:03:03,417 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:03:05,471 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:03:09,088 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:03:12,691 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:03:13,207 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:03:13,208 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:03:13,209 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:03:14,696 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:03:15,548 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:03:19,697 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:03:24,862 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:03:25,609 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:03:28,207 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:03:28,207 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:03:28,208 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:03:30,478 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:03:34,210 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 06:03:34,211 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 06:03:34,211 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 06:03:34,214 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 06:03:34,672 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 06:03:35,677 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:03:36,215 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:03:36,681 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:03:41,217 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:03:42,692 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:03:43,208 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:03:43,210 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:03:43,212 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:03:44,735 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:03:46,302 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:03:51,303 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:03:54,803 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:03:56,928 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:03:58,207 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:03:58,208 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:03:58,209 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:04:02,373 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:04:04,870 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:04:08,068 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:04:12,707 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:04:13,208 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:04:13,209 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:04:13,210 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:04:13,441 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:04:14,944 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:04:19,351 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:04:24,352 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:04:27,032 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:04:28,207 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:04:28,210 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:04:28,210 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:04:29,405 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:04:34,427 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:04:37,098 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:04:39,486 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:04:42,695 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:04:43,871 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:04:43,872 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:04:43,873 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:04:45,117 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:04:47,161 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:04:50,297 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:04:55,299 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:04:57,232 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:04:58,870 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:04:58,871 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:04:58,871 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:05:01,043 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:05:06,578 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:05:07,299 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:05:11,579 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:05:12,697 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:05:14,638 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:05:14,639 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:05:14,640 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:05:16,886 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:05:17,368 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:05:21,887 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:05:27,069 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:05:27,501 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:05:29,637 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:05:29,638 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:05:29,639 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:05:32,838 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:05:37,623 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:05:38,350 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:05:42,697 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:05:43,699 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:05:45,467 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:05:45,468 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:05:45,469 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:05:48,781 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:05:49,705 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:05:53,782 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:05:58,955 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:05:59,888 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:06:00,467 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:06:00,467 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:06:00,468 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:06:04,724 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:06:06,935 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:06:10,507 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:06:12,698 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:06:15,471 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:06:15,471 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:06:15,472 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:06:15,649 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:06:17,012 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:06:21,163 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:06:26,796 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:06:27,081 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:06:30,467 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:06:30,468 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:06:30,468 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:06:32,694 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:06:37,149 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:06:37,805 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:06:42,700 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:06:43,702 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:06:45,468 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:06:45,468 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:06:45,469 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:06:47,227 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:06:48,726 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:06:53,727 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:06:58,728 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:07:00,467 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:07:00,468 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:07:00,469 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:07:04,660 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:07:09,661 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:07:12,700 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:07:14,702 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:07:15,468 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:07:15,468 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:07:15,469 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:07:15,996 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 06:07:15,998 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 06:07:15,998 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 06:07:16,001 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 06:07:16,388 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 06:07:17,394 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:07:18,400 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:07:20,003 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:07:25,005 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:07:27,463 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:07:30,260 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:07:30,468 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:07:30,468 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:07:30,469 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:07:35,713 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:07:37,532 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:07:41,512 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:07:42,700 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:07:45,469 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:07:45,470 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:07:45,470 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:07:46,526 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:07:47,599 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:07:51,527 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:07:57,041 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:07:57,751 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:08:00,468 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:08:00,469 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:08:00,469 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:08:02,716 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:08:07,792 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:08:07,821 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:08:12,701 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:08:13,703 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:08:15,469 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:08:15,469 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:08:15,470 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:08:17,997 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:08:19,686 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:08:24,687 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:08:29,688 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:08:30,469 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:08:30,469 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:08:30,470 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:08:32,106 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:08:34,708 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:08:39,710 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:08:41,169 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:08:42,702 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:08:45,469 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:08:45,470 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:08:45,471 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:08:45,727 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:08:51,225 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:08:51,593 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:08:56,594 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:09:00,469 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:09:00,470 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:09:00,470 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:09:01,286 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:09:01,672 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:09:06,673 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:09:11,474 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:09:12,572 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:09:12,703 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:09:15,469 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:09:15,470 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:09:15,470 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:09:17,681 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:09:21,569 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:09:23,188 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:09:28,190 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:09:30,469 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:09:30,470 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:09:30,470 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:09:31,633 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:09:33,646 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:09:38,648 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:09:39,743 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:09:42,703 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:09:43,704 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:09:45,469 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:09:45,470 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:09:45,470 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:09:49,271 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:09:51,815 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:09:54,272 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:10:00,189 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:10:00,470 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:10:00,470 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:10:00,471 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:10:01,949 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:10:05,736 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:10:10,009 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:10:10,926 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:10:12,703 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:10:15,470 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:10:15,470 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:10:15,471 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:10:16,691 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:10:20,065 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:10:21,712 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:10:26,713 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:10:30,138 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:10:30,470 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:10:30,470 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:10:30,471 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:10:31,728 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:10:36,730 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:10:38,001 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 06:10:38,002 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 06:10:38,003 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 06:10:38,006 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 06:10:38,203 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 06:10:39,209 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:10:40,213 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:10:42,018 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:10:42,704 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:10:46,278 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:10:46,279 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:10:46,280 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:10:47,491 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:10:50,276 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:10:52,507 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:10:58,095 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:11:00,352 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:11:01,277 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:11:01,278 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:11:01,279 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:11:03,489 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:11:09,411 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:11:09,433 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:11:12,704 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:11:14,705 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:11:16,281 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:11:16,282 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:11:16,282 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:11:17,485 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:11:19,951 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:11:24,958 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:11:27,552 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:11:30,814 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:11:31,278 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:11:31,279 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:11:31,280 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:11:36,166 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:11:37,619 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:11:41,167 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:11:42,708 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:11:46,278 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:11:46,279 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:11:46,280 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:11:46,280 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:11:47,695 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:11:51,526 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:11:57,255 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:11:57,758 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:12:01,279 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:12:01,280 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:12:01,289 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:12:02,544 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:12:05,814 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:12:08,092 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:12:12,709 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:12:13,711 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:12:15,877 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:12:16,278 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:12:16,279 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:12:16,280 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:12:19,497 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:12:24,499 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:12:25,945 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:12:29,764 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:12:31,279 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:12:31,280 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:12:31,280 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:12:35,547 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:12:36,010 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:12:40,548 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:12:42,710 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:12:45,950 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:12:46,077 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:12:46,279 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:12:46,280 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:12:46,280 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:12:51,467 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:12:56,144 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:12:56,548 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:13:01,550 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:13:01,830 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:13:01,831 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:13:01,832 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:13:04,203 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:13:07,051 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:13:12,053 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:13:12,710 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:13:14,272 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:13:16,830 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:13:16,830 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:13:16,831 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:13:18,027 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:13:24,007 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:13:24,338 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:13:29,008 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:13:31,833 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:13:31,834 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:13:31,836 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:13:34,086 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:13:34,397 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:13:39,087 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:13:42,711 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:13:44,477 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:13:44,752 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:13:46,831 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:13:46,832 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:13:46,833 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:13:50,093 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:13:52,813 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 06:13:52,814 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 06:13:52,815 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 06:13:52,817 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 06:13:53,542 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 06:13:54,547 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:13:55,552 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:13:55,826 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:14:00,827 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:14:01,831 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:14:01,833 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:14:01,833 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:14:04,611 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:14:06,474 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:14:11,475 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:14:12,711 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:14:13,670 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:14:16,712 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:14:16,831 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:14:16,833 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:14:16,833 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:14:22,442 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:14:23,726 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:14:27,443 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:14:31,789 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:14:31,831 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:14:31,833 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:14:31,833 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:14:33,010 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:14:38,011 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:14:41,853 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:14:42,712 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:14:43,712 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:14:46,831 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:14:46,833 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:14:46,833 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:14:49,071 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:14:51,922 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:14:54,508 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:14:59,510 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:15:01,831 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:15:01,833 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:15:01,833 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:15:02,111 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:15:05,086 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:15:10,772 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:15:12,168 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:15:12,712 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:15:16,714 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:15:18,086 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:15:18,088 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:15:18,089 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:15:20,262 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:15:22,260 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:15:27,279 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:15:30,321 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:15:32,982 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:15:33,086 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:15:33,087 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:15:33,087 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:15:38,258 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:15:40,397 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:15:42,713 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:15:43,714 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:15:48,088 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:15:48,089 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:15:48,091 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:15:48,892 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:15:50,459 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:15:53,892 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:15:58,928 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:16:00,548 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:16:03,087 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:16:03,088 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:16:03,089 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:16:04,300 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:16:09,467 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:16:10,626 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:16:12,714 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:16:14,715 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:16:18,087 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:16:18,088 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:16:18,089 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:16:20,372 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:16:20,684 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:16:25,374 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:16:30,751 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:16:31,080 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:16:33,087 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:16:33,088 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:16:33,089 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:16:36,259 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:16:40,817 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:16:41,596 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:16:42,714 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:16:46,715 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:16:47,870 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:16:48,087 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:16:48,088 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:16:48,089 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:16:52,328 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:16:57,943 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:16:58,134 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:17:03,087 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:17:03,088 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:17:03,089 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:17:03,335 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:17:06,810 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 06:17:06,811 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 06:17:06,811 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 06:17:06,814 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 06:17:07,007 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 06:17:08,014 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:17:08,816 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:17:09,020 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:17:12,714 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:17:14,716 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:17:18,085 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:17:18,087 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:17:18,088 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:17:18,089 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:17:20,322 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:17:25,323 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:17:28,146 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:17:30,593 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:17:33,088 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:17:33,091 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:17:33,092 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:17:35,988 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:17:38,212 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:17:40,989 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:17:42,717 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:17:46,677 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:17:48,088 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:17:48,089 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:17:48,089 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:17:48,273 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:17:52,281 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:17:56,335 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:17:58,053 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:18:03,054 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:18:03,326 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:18:03,329 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:18:03,330 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:18:06,411 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:18:08,603 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:18:12,719 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:18:13,721 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:18:16,474 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:18:18,326 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:18:18,327 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:18:18,328 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:18:19,525 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:18:25,496 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:18:26,543 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:18:30,497 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:18:33,326 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:18:33,327 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:18:33,328 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:18:36,356 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:18:36,610 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:18:41,357 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:18:42,719 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:18:46,669 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:18:47,107 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:18:48,326 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:18:48,327 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:18:48,327 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:18:52,522 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:18:54,731 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:18:58,437 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:19:03,326 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:19:03,327 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:19:03,328 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:19:03,543 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:19:04,802 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:19:08,544 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:19:12,720 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:19:13,809 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:19:14,876 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:19:18,326 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:19:18,327 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:19:18,328 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:19:19,532 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:19:24,941 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:19:25,077 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:19:30,079 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:19:32,994 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:19:33,326 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:19:33,327 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:19:33,328 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:19:35,512 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:19:40,514 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:19:42,720 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:19:43,051 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:19:45,721 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:19:48,327 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:19:48,328 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:19:48,329 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:19:51,533 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:19:53,119 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:19:56,534 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:20:01,535 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:20:02,167 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:20:03,330 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:20:03,330 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:20:03,331 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:20:06,578 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:20:11,579 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:20:12,251 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:20:12,721 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:20:16,722 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:20:18,328 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:20:18,330 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:20:18,340 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:20:21,380 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 06:20:21,382 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 06:20:21,382 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 06:20:21,384 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 06:20:22,322 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:20:22,323 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 06:20:22,386 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:20:23,327 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:20:27,387 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:20:32,379 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:20:33,238 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:20:33,327 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:20:33,330 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:20:33,330 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:20:38,575 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:20:42,438 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:20:42,721 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:20:43,722 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:20:48,724 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:20:49,543 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:20:49,544 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:20:49,545 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:20:52,496 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:20:53,787 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:20:58,789 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:21:02,574 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:21:03,856 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:21:04,541 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:21:04,541 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:21:04,542 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:21:09,797 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:21:12,649 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:21:12,722 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:21:15,723 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:21:19,544 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:21:19,544 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:21:19,544 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:21:21,050 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:21:22,714 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:21:26,051 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:21:31,080 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:21:32,775 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:21:34,541 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:21:34,542 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:21:34,542 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:21:36,729 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:21:42,040 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:21:42,722 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:21:42,846 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:21:47,724 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:21:49,541 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:21:49,542 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:21:49,542 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:21:50,900 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:21:52,750 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:21:57,752 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:22:00,964 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:22:03,590 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:22:04,541 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:22:04,542 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:22:04,542 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:22:08,759 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:22:11,038 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:22:12,723 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:22:14,727 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:22:19,541 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:22:19,542 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:22:19,542 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:22:19,799 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:22:21,106 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:22:24,800 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:22:29,802 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:22:31,189 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:22:34,542 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:22:34,543 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:22:34,544 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:22:35,760 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:22:41,267 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:22:41,512 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:22:42,723 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:22:46,742 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:22:49,542 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:22:49,542 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:22:49,543 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:22:51,785 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:22:55,358 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:22:57,689 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:23:02,690 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:23:04,419 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:23:04,542 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:23:04,543 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:23:04,543 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:23:07,755 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:23:12,724 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:23:12,981 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:23:14,485 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:23:17,982 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:23:19,544 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:23:19,553 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:23:19,555 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:23:23,693 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:23:24,575 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:23:28,694 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:23:34,543 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:23:34,545 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:23:34,552 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:23:34,553 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:23:34,638 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:23:39,715 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:23:42,004 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 06:23:42,006 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 06:23:42,006 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 06:23:42,009 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 06:23:42,691 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:23:42,692 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 06:23:42,724 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:23:43,698 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:23:44,725 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:23:49,726 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:23:50,262 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:23:50,263 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:23:50,263 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:23:52,761 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:23:55,690 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:24:00,691 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:24:02,827 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:24:05,263 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:24:05,263 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:24:05,264 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:24:06,479 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:24:11,992 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:24:12,725 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:24:12,901 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:24:17,726 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:24:20,264 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:24:20,265 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:24:20,266 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:24:22,964 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:24:23,619 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:24:28,621 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:24:33,026 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:24:34,605 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:24:35,263 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:24:35,263 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:24:35,264 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:24:40,182 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:24:41,083 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:24:42,726 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:24:45,727 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:24:50,263 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:24:50,264 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:24:50,264 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:24:51,145 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:24:51,453 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:24:56,454 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:25:01,210 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:25:02,227 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:25:05,263 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:25:05,264 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:25:05,264 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:25:07,499 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:25:11,275 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:25:12,726 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:25:12,727 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:25:17,728 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:25:20,263 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:25:20,264 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:25:20,264 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:25:21,337 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:25:23,508 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:25:29,357 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:25:31,399 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:25:34,358 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:25:36,603 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:25:36,605 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:25:36,606 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:25:39,464 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:25:39,904 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:25:42,726 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:25:45,728 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:25:49,527 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:25:51,449 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:25:51,601 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:25:51,602 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:25:51,603 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:25:56,825 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:25:59,586 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:26:01,871 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:26:06,607 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:26:06,608 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:26:06,614 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:26:07,861 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:26:08,644 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:26:12,729 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:26:13,731 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:26:16,710 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:26:18,935 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:26:21,608 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:26:21,608 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:26:21,614 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:26:24,827 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:26:26,771 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:26:30,509 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:26:35,510 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:26:36,608 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:26:36,608 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:26:36,614 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:26:36,836 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:26:40,771 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:26:42,730 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:26:46,299 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:26:46,921 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:26:51,300 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:26:51,609 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:26:51,612 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:26:51,616 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:26:54,810 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 06:26:54,811 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 06:26:54,811 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 06:26:54,814 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 06:26:54,981 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 06:26:55,988 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:26:56,816 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:26:56,994 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:27:01,817 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:27:05,053 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:27:06,608 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:27:06,609 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:27:06,614 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:27:06,876 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:27:11,878 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:27:12,731 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:27:15,126 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:27:17,251 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:27:22,253 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:27:23,054 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:27:23,054 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:27:23,055 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:27:25,200 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:27:27,318 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:27:32,319 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:27:35,268 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:27:37,523 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:27:38,053 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:27:38,054 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:27:38,054 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:27:42,731 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:27:42,732 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:27:45,330 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:27:47,962 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:27:53,053 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:27:53,054 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:27:53,054 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:27:53,258 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:27:55,397 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:27:58,259 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:28:03,454 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:28:03,486 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:28:08,053 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:28:08,054 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:28:08,054 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:28:09,241 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:28:12,733 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:28:14,734 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:28:15,524 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:28:19,735 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:28:23,053 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:28:23,054 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:28:23,054 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:28:25,265 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:28:25,587 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:28:30,266 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:28:33,641 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:28:35,858 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:28:38,054 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:28:38,068 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:28:38,069 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:28:41,345 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:28:42,732 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:28:43,713 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:28:46,734 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:28:51,735 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:28:53,054 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:28:53,054 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:28:53,055 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:28:53,812 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:28:57,264 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:29:02,265 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:29:05,884 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:29:07,822 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:29:08,054 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:29:08,057 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:29:08,057 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:29:12,733 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:29:12,837 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:29:14,945 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:29:17,838 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:29:22,998 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:29:23,054 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:29:23,055 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:29:23,055 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:29:23,284 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:29:28,285 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:29:33,063 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:29:33,543 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:29:38,055 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:29:38,056 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:29:38,057 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:29:39,273 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:29:42,733 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:29:43,136 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:29:44,734 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:29:49,736 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:29:53,054 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:29:53,055 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:29:53,055 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:29:53,199 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:29:55,227 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:30:01,204 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:30:01,249 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:30:06,205 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:30:09,378 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:30:09,379 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:30:09,379 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:30:10,690 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 06:30:10,691 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 06:30:10,691 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 06:30:10,692 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 06:30:11,315 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:30:11,316 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 06:30:11,693 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:30:12,321 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:30:12,735 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:30:16,736 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:30:21,401 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:30:22,047 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:30:24,376 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:30:24,377 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:30:24,378 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:30:27,587 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:30:31,467 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:30:33,252 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:30:38,253 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:30:39,376 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:30:39,377 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:30:39,377 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:30:41,549 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:30:42,735 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:30:43,735 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:30:49,605 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:30:49,694 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:30:54,377 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:30:54,378 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:30:54,378 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:30:55,569 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:30:59,667 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:31:00,640 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:31:05,641 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:31:09,376 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:31:09,377 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:31:09,377 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:31:11,561 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:31:11,743 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:31:12,735 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:31:16,737 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:31:21,812 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:31:21,957 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:31:24,377 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:31:24,377 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:31:24,378 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:31:27,564 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:31:29,871 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:31:32,734 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:31:37,735 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:31:39,376 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:31:39,377 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:31:39,378 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:31:39,936 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:31:42,736 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:31:42,737 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:31:47,755 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:31:50,003 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:31:53,336 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:31:54,376 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:31:54,379 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:31:54,380 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:31:59,077 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:32:00,076 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:32:04,078 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:32:09,377 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:32:09,378 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:32:09,378 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:32:09,378 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:32:10,142 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:32:12,737 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:32:14,738 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:32:19,199 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:32:19,944 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:32:24,946 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:32:25,314 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:32:25,316 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:32:25,316 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:32:27,251 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:32:30,579 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:32:35,580 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:32:37,320 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:32:40,313 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:32:40,313 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:32:40,314 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:32:41,522 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:32:42,738 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:32:47,230 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:32:47,393 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:32:52,232 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:32:55,313 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:32:55,314 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:32:55,314 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:32:55,453 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:32:57,536 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:33:02,538 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:33:05,712 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:33:07,628 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:33:10,313 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:33:10,314 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:33:10,314 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:33:12,740 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:33:12,743 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:33:15,781 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:33:18,226 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:33:23,227 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:33:23,657 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 06:33:23,668 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 06:33:23,669 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 06:33:23,671 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 06:33:23,838 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 06:33:25,313 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:33:25,315 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:33:25,315 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:33:25,848 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:33:26,853 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:33:28,520 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:33:33,797 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:33:33,906 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:33:38,798 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:33:41,014 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:33:41,015 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:33:41,015 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:33:42,739 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:33:43,973 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:33:44,740 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:33:49,742 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:33:54,041 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:33:54,830 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:33:56,013 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:33:56,014 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:33:56,015 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:34:00,284 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:34:04,112 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:34:06,025 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:34:11,015 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:34:11,016 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:34:11,016 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:34:11,194 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:34:12,740 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:34:14,180 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:34:16,741 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:34:22,200 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:34:24,264 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:34:26,013 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:34:26,014 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:34:26,014 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:34:27,350 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:34:33,106 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:34:34,327 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:34:38,107 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:34:41,013 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:34:41,014 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:34:41,014 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:34:42,383 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:34:42,740 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:34:43,741 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:34:48,742 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:34:53,455 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:34:54,335 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:34:56,013 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:34:56,014 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:34:56,015 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:35:00,171 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:35:01,508 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:35:05,907 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:35:11,014 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:35:11,014 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:35:11,015 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:35:11,176 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:35:11,579 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:35:12,741 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:35:16,742 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:35:21,654 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:35:22,063 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:35:26,014 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:35:26,014 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:35:26,015 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:35:27,217 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:35:31,709 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:35:32,607 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:35:37,609 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:35:41,014 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:35:41,014 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:35:41,015 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:35:41,774 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:35:42,742 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:35:42,742 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:35:47,744 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:35:51,833 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:35:53,274 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:35:56,017 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:35:56,018 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:35:56,019 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:35:58,281 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:35:59,894 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:36:03,964 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:36:08,965 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:36:12,041 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:36:12,042 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:36:12,042 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:36:12,753 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:36:14,754 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:36:19,756 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:36:24,032 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:36:25,368 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:36:27,041 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:36:27,041 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:36:27,042 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:36:31,204 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:36:34,092 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:36:36,995 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:36:42,041 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:36:42,041 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:36:42,042 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:36:42,042 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:36:42,754 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:36:44,149 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:36:47,755 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:36:51,850 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 06:36:51,851 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 06:36:51,851 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 06:36:51,854 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 06:36:52,202 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 06:36:52,855 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:36:54,212 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:36:57,041 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:36:57,042 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:36:57,042 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:36:58,245 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:37:03,447 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:37:04,284 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:37:08,449 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:37:12,042 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:37:12,043 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:37:12,043 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:37:12,754 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:37:13,755 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:37:14,353 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:37:18,757 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:37:22,405 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:37:23,968 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:37:27,042 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:37:27,042 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:37:27,043 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:37:29,228 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:37:34,229 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:37:38,499 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:37:39,374 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:37:42,042 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:37:42,042 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:37:42,043 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:37:42,757 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:37:44,759 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:37:48,565 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:37:50,630 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:37:56,377 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:37:57,042 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:37:57,042 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:37:57,043 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:37:58,627 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:38:02,287 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:38:05,682 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:38:07,904 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:38:12,757 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:38:13,511 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:38:13,512 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:38:13,513 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:38:13,513 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:38:15,751 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:38:18,700 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:38:23,701 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:38:25,823 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:38:28,510 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:38:28,510 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:38:28,511 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:38:28,771 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:38:33,772 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:38:35,890 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:38:39,034 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:38:42,758 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:38:43,509 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:38:43,510 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:38:43,510 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:38:43,936 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:38:44,756 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:38:49,757 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:38:55,520 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:38:56,014 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:38:58,509 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:38:58,510 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:38:58,510 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:39:00,712 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:39:04,069 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:39:06,319 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:39:11,321 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:39:12,758 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:39:13,510 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:39:13,510 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:39:13,510 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:39:14,135 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:39:16,764 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:39:21,766 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:39:24,207 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:39:27,277 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:39:28,510 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:39:28,511 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:39:28,521 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:39:32,554 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:39:34,273 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:39:37,555 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:39:42,556 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:39:42,758 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:39:43,510 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:39:43,511 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:39:43,521 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:39:44,341 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:39:47,763 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:39:52,764 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:39:54,414 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:39:58,034 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:39:58,510 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:39:58,511 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:39:58,521 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:40:03,362 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:40:04,473 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:40:08,364 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:40:11,972 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 06:40:11,974 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 06:40:11,975 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 06:40:11,978 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 06:40:12,527 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:40:12,528 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 06:40:12,759 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:40:13,511 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:40:13,512 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:40:13,522 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:40:13,721 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:40:14,543 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:40:18,722 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:40:24,386 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:40:24,620 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:40:28,510 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:40:28,511 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:40:28,522 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:40:29,779 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:40:34,682 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:40:35,015 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:40:40,017 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:40:42,760 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:40:43,510 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:40:43,511 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:40:43,522 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:40:44,751 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:40:45,736 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:40:50,737 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:40:54,816 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:40:56,629 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:40:59,767 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:40:59,768 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:40:59,769 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:41:01,974 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:41:02,880 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:41:06,974 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:41:11,935 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:41:12,644 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:41:12,761 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:41:14,767 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:41:14,767 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:41:14,768 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:41:17,976 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:41:22,012 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:41:23,440 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:41:28,442 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:41:29,768 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:41:29,769 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:41:29,769 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:41:32,077 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:41:33,968 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:41:39,867 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:41:40,133 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:41:42,761 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:41:44,767 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:41:44,767 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:41:44,768 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:41:44,981 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:41:50,197 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:41:50,612 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:41:55,613 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:41:59,767 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:41:59,767 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:41:59,768 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:42:00,262 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:42:00,945 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:42:05,947 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:42:10,338 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:42:11,417 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:42:12,762 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:42:14,769 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:42:14,771 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:42:14,772 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:42:17,036 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:42:20,402 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:42:23,004 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:42:28,410 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:42:28,530 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:42:29,768 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:42:29,769 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:42:29,769 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:42:33,976 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:42:38,601 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:42:39,734 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:42:42,762 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:42:44,763 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:42:44,832 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:42:44,833 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:42:44,834 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:42:48,770 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:42:50,014 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:42:55,015 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:42:56,836 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:42:59,832 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:42:59,833 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:42:59,833 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:43:01,003 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:43:06,448 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:43:06,894 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:43:11,449 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:43:12,777 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:43:14,834 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:43:14,835 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:43:14,835 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:43:16,957 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:43:17,100 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:43:22,103 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:43:24,770 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 06:43:24,771 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 06:43:24,771 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 06:43:24,774 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 06:43:25,009 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 06:43:27,019 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:43:27,775 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:43:29,833 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:43:29,834 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:43:29,834 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:43:33,116 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:43:37,095 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:43:38,307 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:43:42,763 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:43:43,769 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:43:44,833 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:43:44,833 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:43:44,834 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:43:46,162 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:43:49,125 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:43:54,219 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:43:54,710 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:43:59,711 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:43:59,833 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:43:59,834 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:43:59,834 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:44:04,288 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:44:05,207 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:44:10,208 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:44:12,764 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:44:14,354 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:44:14,833 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:44:14,834 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:44:14,834 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:44:16,074 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:44:21,076 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:44:24,422 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:44:26,199 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:44:30,682 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:44:30,684 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:44:30,686 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:44:31,899 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:44:34,491 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:44:37,009 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:44:42,011 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:44:42,764 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:44:44,551 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:44:45,682 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:44:45,683 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:44:45,683 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:44:47,934 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:44:52,953 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:44:54,619 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:44:57,954 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:45:00,683 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:45:00,685 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:45:00,686 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:45:02,675 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:45:03,651 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:45:08,652 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:45:12,741 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:45:12,765 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:45:13,766 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:45:15,682 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:45:15,683 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:45:15,684 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:45:18,910 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:45:22,805 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:45:24,562 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:45:29,563 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:45:30,683 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:45:30,684 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:45:30,685 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:45:30,862 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:45:34,867 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:45:40,659 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:45:40,927 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:45:42,765 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:45:45,684 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:45:45,685 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:45:45,685 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:45:45,686 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:45:50,938 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:45:50,993 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:45:55,940 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:46:00,683 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:46:00,684 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:46:00,685 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:46:01,051 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:46:01,846 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:46:06,847 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:46:11,129 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:46:12,249 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:46:12,766 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:46:16,437 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:46:16,438 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:46:16,440 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:46:17,633 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:46:19,184 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:46:22,656 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:46:28,350 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:46:29,255 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:46:31,437 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:46:31,437 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:46:31,438 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:46:33,690 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:46:36,870 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 06:46:36,871 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 06:46:36,871 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 06:46:36,874 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 06:46:37,306 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 06:46:38,875 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:46:39,314 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:46:42,766 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:46:44,768 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:46:46,437 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:46:46,437 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:46:46,437 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:46:49,385 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:46:50,638 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:46:55,639 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:46:56,432 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:47:00,968 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:47:01,437 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:47:01,437 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:47:01,438 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:47:06,504 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:47:06,623 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:47:11,624 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:47:12,768 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:47:16,437 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:47:16,437 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:47:16,438 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:47:16,565 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:47:16,639 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:47:21,641 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:47:26,634 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:47:26,869 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:47:31,439 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:47:31,440 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:47:31,441 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:47:32,627 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:47:36,696 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:47:38,521 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:47:42,770 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:47:43,773 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:47:46,437 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:47:46,438 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:47:46,438 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:47:46,783 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:47:49,664 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:47:55,298 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:47:56,854 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:48:00,299 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:48:02,791 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:48:02,792 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:48:02,793 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:48:06,126 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:48:06,917 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:48:11,127 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:48:12,771 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:48:14,972 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:48:16,926 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:48:17,790 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:48:17,790 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:48:17,791 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:48:22,065 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:48:25,134 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:48:27,359 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:48:32,361 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:48:32,790 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:48:32,790 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:48:32,791 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:48:35,198 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:48:38,028 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:48:42,771 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:48:43,772 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:48:45,298 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:48:47,790 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:48:47,791 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:48:47,792 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:48:48,986 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:48:53,355 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:48:54,108 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:48:59,110 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:49:02,790 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:49:02,790 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:49:02,791 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:49:03,421 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:49:04,992 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:49:09,994 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:49:12,772 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:49:13,493 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:49:15,773 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:49:17,790 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:49:17,791 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:49:17,792 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:49:20,975 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:49:23,562 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:49:26,162 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:49:31,898 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:49:32,631 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:49:32,790 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:49:32,791 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:49:32,791 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:49:37,041 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:49:42,625 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:49:42,697 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:49:42,772 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:49:47,773 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:49:48,982 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:49:48,983 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:49:48,984 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:49:50,220 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 06:49:50,221 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 06:49:50,221 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 06:49:50,223 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 06:49:50,757 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:49:50,758 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 06:49:51,763 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:49:53,224 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:49:58,225 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:50:00,846 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:50:03,925 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:50:03,981 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:50:03,982 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:50:03,982 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:50:09,215 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:50:10,908 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:50:12,773 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:50:14,774 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:50:18,981 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:50:18,982 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:50:18,983 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:50:20,166 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:50:20,980 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:50:25,167 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:50:31,045 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:50:31,064 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:50:33,982 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:50:33,982 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:50:33,983 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:50:36,199 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:50:41,115 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:50:42,037 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:50:42,773 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:50:47,774 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:50:48,982 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:50:48,982 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:50:48,983 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:50:51,171 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:50:53,165 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:50:59,035 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:50:59,227 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:51:03,982 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:51:03,982 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:51:03,983 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:51:04,256 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:51:09,303 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:51:09,477 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:51:12,774 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:51:14,775 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:51:18,982 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:51:18,983 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:51:18,983 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:51:19,360 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:51:20,163 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:51:25,165 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:51:29,435 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:51:30,460 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:51:33,984 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:51:33,986 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:51:33,987 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:51:36,261 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:51:39,502 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:51:42,117 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:51:42,774 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:51:47,197 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:51:48,982 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:51:48,983 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:51:48,984 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:51:49,566 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:51:52,236 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:51:57,237 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:52:02,830 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:52:03,665 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:52:03,983 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:52:03,984 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:52:03,984 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:52:08,239 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:52:12,775 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:52:13,725 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:52:13,776 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:52:18,777 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:52:18,983 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:52:18,985 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:52:18,986 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:52:24,650 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:52:25,792 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:52:29,651 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:52:33,983 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:52:33,984 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:52:33,985 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:52:35,162 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:52:35,853 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:52:40,163 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:52:42,786 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:52:42,901 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:52:45,787 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:52:48,983 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:52:48,984 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:52:48,985 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:52:51,200 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:52:52,967 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:52:57,187 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:53:02,914 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:53:03,029 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:53:03,983 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:53:03,984 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:53:03,985 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:53:08,148 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:53:11,420 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 06:53:11,421 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 06:53:11,421 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 06:53:11,423 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 06:53:12,082 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 06:53:12,776 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:53:13,085 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:53:13,777 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:53:14,088 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:53:18,778 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:53:19,631 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:53:19,632 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:53:19,633 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:53:23,146 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:53:24,746 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:53:29,748 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:53:33,213 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:53:34,630 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:53:34,631 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:53:34,632 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:53:34,825 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:53:39,826 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:53:42,776 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:53:43,272 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:53:45,777 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:53:49,633 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:53:49,633 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:53:49,634 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:53:51,222 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:53:51,326 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:53:56,223 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:54:01,392 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:54:01,646 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:54:04,630 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:54:04,632 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:54:04,632 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:54:06,894 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:54:11,471 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:54:12,510 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:54:12,777 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:54:17,779 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:54:19,631 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:54:19,632 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:54:19,633 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:54:21,561 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:54:22,878 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:54:27,880 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:54:31,656 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:54:32,941 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:54:34,631 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:54:34,632 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:54:34,633 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:54:38,844 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:54:41,714 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:54:42,777 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:54:44,780 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:54:49,631 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:54:49,632 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:54:49,633 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:54:49,883 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:54:51,789 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:54:54,884 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:55:00,712 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:55:01,895 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:55:04,631 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:55:04,633 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:55:04,633 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:55:05,845 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:55:09,944 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:55:11,574 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:55:12,778 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:55:16,814 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:55:19,632 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:55:19,633 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:55:19,633 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:55:20,022 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:55:21,858 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:55:26,859 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:55:30,094 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:55:32,204 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:55:34,631 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:55:34,633 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:55:34,634 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:55:37,875 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:55:39,154 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:55:42,778 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:55:43,779 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:55:49,210 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:55:49,632 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:55:49,633 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:55:49,633 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:55:49,634 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:55:54,896 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:55:59,289 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:56:00,398 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:56:05,400 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:56:05,843 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:56:05,845 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:56:05,846 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:56:09,355 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:56:11,309 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:56:12,779 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:56:16,780 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:56:19,428 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:56:20,842 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:56:20,842 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:56:20,843 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:56:22,096 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:56:26,480 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 06:56:26,481 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 06:56:26,481 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 06:56:26,484 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 06:56:26,489 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 06:56:27,486 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:56:27,495 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:56:32,486 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:56:35,844 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:56:35,845 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:56:35,846 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:56:37,565 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:56:38,101 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:56:42,780 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:56:43,782 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:56:47,645 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:56:49,667 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:56:50,843 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:56:50,843 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:56:50,844 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:56:54,988 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:56:57,709 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:56:59,989 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:57:05,707 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:57:05,767 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:57:05,842 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:57:05,843 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:57:05,843 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:57:11,055 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:57:12,782 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:57:15,838 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:57:16,556 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:57:20,843 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:57:20,843 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:57:20,843 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:57:22,059 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:57:25,904 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:57:27,129 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:57:32,130 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:57:35,843 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:57:35,843 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:57:35,844 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:57:35,973 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:57:38,090 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:57:42,782 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:57:43,423 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:57:46,044 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:57:48,424 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:57:51,576 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:57:51,577 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:57:51,578 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:57:53,663 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:57:54,113 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:57:58,664 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:58:04,190 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:58:04,192 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:58:06,575 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:58:06,576 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:58:06,577 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:58:09,792 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:58:12,782 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:58:14,273 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:58:15,783 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:58:20,785 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:58:21,576 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:58:21,577 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:58:21,578 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:58:24,340 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:58:25,786 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:58:30,787 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:58:33,400 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:58:36,237 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:58:36,575 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:58:36,575 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:58:36,576 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:58:41,455 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:58:41,716 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:58:42,790 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:58:46,800 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:58:51,516 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:58:51,576 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:58:51,577 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:58:51,577 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:58:52,743 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:58:57,744 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:59:01,592 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:59:03,307 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:59:06,575 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:59:06,575 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:59:06,576 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:59:08,794 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:59:11,656 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:59:12,783 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:59:14,784 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:59:19,785 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:59:21,575 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:59:21,576 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:59:21,577 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:59:21,727 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:59:24,794 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:59:30,349 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:59:31,794 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:59:35,350 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:59:37,796 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:59:37,841 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:59:37,894 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:59:39,147 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 06:59:39,149 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 06:59:39,150 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 06:59:39,152 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 06:59:39,961 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:59:40,023 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 06:59:41,154 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:59:41,971 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:59:42,784 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 06:59:46,785 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:59:50,028 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 06:59:52,764 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 06:59:52,795 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 06:59:52,796 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 06:59:52,796 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 06:59:58,048 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:00:00,113 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:00:03,789 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:00:07,799 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:00:07,800 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:00:07,800 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:00:09,581 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:00:10,180 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:00:12,784 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:00:14,785 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:00:20,252 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:00:20,338 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:00:22,796 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:00:22,796 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:00:22,797 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:00:26,047 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:00:30,320 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:00:32,013 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:00:37,015 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:00:37,798 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:00:37,799 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:00:37,840 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:00:40,391 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:00:42,467 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:00:42,785 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:00:47,787 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:00:52,469 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:00:52,796 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:00:52,797 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:00:52,797 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:00:52,840 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:00:57,962 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:01:00,521 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:01:03,617 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:01:07,797 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:01:07,798 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:01:07,798 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:01:09,380 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:01:10,585 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:01:12,785 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:01:14,786 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:01:19,651 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:01:20,613 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:01:22,797 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:01:22,797 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:01:22,798 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:01:25,989 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:01:27,705 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:01:31,011 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:01:36,013 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:01:37,797 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:01:37,813 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:01:37,813 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:01:39,789 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:01:41,047 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:01:42,786 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:01:46,788 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:01:47,857 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:01:52,052 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:01:52,797 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:01:52,813 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:01:52,814 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:01:57,578 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:01:57,915 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:02:02,579 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:02:07,797 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:02:07,798 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:02:07,813 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:02:07,814 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:02:07,996 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:02:12,788 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:02:13,791 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:02:18,071 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:02:19,037 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:02:22,799 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:02:22,815 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:02:22,818 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:02:24,051 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:02:28,137 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:02:29,711 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:02:34,713 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:02:37,798 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:02:37,814 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:02:37,814 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:02:38,203 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:02:40,060 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:02:42,788 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:02:45,737 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:02:46,252 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:02:50,738 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:02:53,093 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:02:53,093 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:02:53,094 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:02:54,660 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 07:02:54,661 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 07:02:54,661 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 07:02:54,662 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 07:02:55,322 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 07:02:56,329 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:02:56,664 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:03:01,666 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:03:06,392 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:03:07,191 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:03:08,092 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:03:08,092 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:03:08,093 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:03:12,338 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:03:12,790 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:03:16,537 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:03:17,966 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:03:22,967 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:03:23,094 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:03:23,095 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:03:23,096 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:03:26,607 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:03:28,550 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:03:34,102 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:03:34,655 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:03:38,092 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:03:38,093 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:03:38,093 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:03:39,346 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:03:42,790 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:03:44,722 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:03:44,791 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:03:49,793 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:03:53,092 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:03:53,093 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:03:53,093 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:03:54,789 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:03:55,321 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:04:00,323 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:04:03,865 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:04:06,169 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:04:08,092 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:04:08,092 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:04:08,093 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:04:11,325 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:04:12,791 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:04:13,943 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:04:16,792 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:04:21,794 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:04:23,092 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:04:23,093 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:04:23,094 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:04:24,015 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:04:27,360 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:04:32,545 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:04:34,136 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:04:37,546 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:04:39,700 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:04:39,701 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:04:39,703 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:04:42,193 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:04:42,791 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:04:42,792 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:04:47,793 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:04:52,269 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:04:53,305 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:04:54,700 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:04:54,701 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:04:54,701 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:04:58,908 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:05:02,338 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:05:04,670 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:05:09,672 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:05:09,700 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:05:09,702 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:05:09,703 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:05:12,408 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:05:12,792 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:05:14,793 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:05:19,794 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:05:22,469 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:05:24,700 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:05:24,700 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:05:24,700 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:05:24,926 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:05:30,225 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:05:30,526 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:05:35,226 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:05:39,700 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:05:39,700 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:05:39,701 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:05:40,601 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:05:40,929 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:05:42,792 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:05:46,794 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:05:50,671 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:05:52,731 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:05:54,700 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:05:54,701 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:05:54,701 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:05:57,951 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:06:00,740 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:06:03,583 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:06:08,584 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:06:09,701 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:06:09,702 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:06:09,702 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:06:12,793 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:06:13,010 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 07:06:13,011 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 07:06:13,011 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 07:06:13,013 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 07:06:13,863 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 07:06:14,014 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:06:14,867 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:06:19,015 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:06:24,565 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:06:24,700 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:06:24,701 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:06:24,701 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:06:24,959 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:06:29,947 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:06:35,027 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:06:35,606 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:06:40,609 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:06:41,259 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:06:41,260 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:06:41,261 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:06:42,793 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:06:44,137 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:06:45,794 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:06:50,796 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:06:54,201 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:06:56,147 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:06:56,256 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:06:56,257 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:06:56,258 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:07:01,455 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:07:02,249 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:07:06,626 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:07:11,263 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:07:11,264 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:07:11,264 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:07:12,315 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:07:12,532 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:07:12,794 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:07:17,795 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:07:22,383 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:07:23,435 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:07:26,263 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:07:26,264 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:07:26,264 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:07:29,428 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:07:32,450 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:07:34,465 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:07:39,466 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:07:41,263 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:07:41,264 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:07:41,264 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:07:42,509 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:07:42,794 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:07:44,795 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:07:50,736 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:07:52,576 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:07:55,737 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:07:56,265 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:07:56,267 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:07:56,268 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:08:00,631 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:08:01,204 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:08:06,205 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:08:10,700 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:08:11,264 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:08:11,264 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:08:11,265 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:08:11,265 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:08:12,795 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:08:16,801 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:08:20,765 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:08:22,236 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:08:27,237 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:08:27,460 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:08:27,461 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:08:27,462 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:08:30,842 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:08:32,862 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:08:37,863 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:08:40,915 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:08:42,459 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:08:42,459 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:08:42,460 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:08:42,795 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:08:43,796 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:08:49,726 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:08:50,986 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:08:54,727 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:08:57,463 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:08:57,465 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:08:57,466 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:08:59,788 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:09:01,053 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:09:04,789 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:09:10,696 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:09:11,117 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:09:12,459 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:09:12,461 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:09:12,461 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:09:12,796 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:09:15,797 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:09:21,127 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:09:21,187 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:09:26,130 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:09:27,460 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:09:27,461 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:09:27,461 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:09:27,673 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 07:09:27,674 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 07:09:27,675 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 07:09:27,677 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 07:09:28,239 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 07:09:29,246 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:09:31,679 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:09:36,680 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:09:38,304 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:09:42,229 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:09:42,460 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:09:42,460 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:09:42,462 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:09:42,797 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:09:48,047 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:09:48,360 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:09:53,047 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:09:57,460 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:09:57,461 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:09:57,461 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:09:58,539 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:09:58,672 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:10:03,674 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:10:08,606 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:10:09,053 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:10:12,461 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:10:12,462 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:10:12,463 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:10:12,798 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:10:14,800 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:10:16,663 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:10:19,908 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:10:24,947 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:10:26,723 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:10:27,460 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:10:27,461 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:10:27,463 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:10:30,638 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:10:35,831 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:10:36,788 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:10:40,832 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:10:42,800 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:10:43,496 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:10:43,497 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:10:43,498 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:10:46,753 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:10:46,860 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:10:51,754 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:10:54,911 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:10:57,376 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:10:58,495 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:10:58,495 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:10:58,496 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:11:02,756 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:11:04,980 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:11:07,897 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:11:12,800 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:11:13,495 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:11:13,496 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:11:13,496 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:11:13,703 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:11:15,037 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:11:18,704 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:11:24,122 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:11:25,106 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:11:28,495 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:11:28,495 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:11:28,495 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:11:29,714 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:11:34,719 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:11:35,168 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:11:39,721 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:11:42,801 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:11:43,495 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:11:43,495 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:11:43,496 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:11:44,737 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:11:45,240 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:11:49,738 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:11:53,285 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:11:54,804 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:11:59,806 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:12:00,038 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:12:00,039 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:12:00,040 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:12:03,352 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:12:05,114 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:12:10,116 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:12:12,801 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:12:13,413 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:12:15,036 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:12:15,037 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:12:15,038 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:12:15,300 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:12:20,302 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:12:23,483 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:12:25,661 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:12:30,038 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:12:30,039 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:12:30,040 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:12:30,808 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:12:33,553 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:12:35,809 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:12:40,653 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 07:12:40,654 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 07:12:40,655 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 07:12:40,657 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 07:12:41,615 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 07:12:41,660 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:12:42,618 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:12:42,802 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:12:45,036 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:12:45,036 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:12:45,037 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:12:47,236 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:12:52,237 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:12:57,237 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:13:00,036 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:13:00,037 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:13:00,038 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:13:02,307 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:13:07,308 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:13:10,772 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:13:12,802 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:13:12,803 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:13:15,036 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:13:15,037 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:13:15,037 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:13:18,298 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:13:20,840 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:13:23,720 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:13:28,721 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:13:30,036 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:13:30,037 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:13:30,037 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:13:30,899 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:13:34,240 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:13:39,241 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:13:40,956 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:13:42,803 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:13:44,804 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:13:45,037 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:13:45,037 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:13:45,038 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:13:50,255 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:13:51,036 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:13:55,256 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:13:59,086 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:14:00,037 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:14:00,037 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:14:00,038 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:14:00,277 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:14:05,279 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:14:09,146 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:14:10,859 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:14:12,803 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:14:16,251 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:14:16,251 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:14:16,252 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:14:16,472 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:14:19,203 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:14:21,589 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:14:26,590 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:14:29,280 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:14:31,250 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:14:31,251 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:14:31,252 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:14:32,447 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:14:37,515 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:14:37,679 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:14:42,680 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:14:42,804 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:14:46,250 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:14:46,250 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:14:46,251 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:14:48,504 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:14:53,506 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:14:58,507 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:15:01,250 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:15:01,251 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:15:01,251 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:15:04,459 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:15:09,460 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:15:12,804 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:15:14,805 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:15:16,250 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:15:16,251 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:15:16,251 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:15:20,452 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:15:24,896 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:15:25,957 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:15:30,959 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:15:31,256 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:15:31,258 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:15:31,260 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:15:34,963 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:15:35,965 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:15:40,967 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:15:42,804 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:15:43,017 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:15:46,252 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:15:46,256 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:15:46,256 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:15:46,427 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:15:51,429 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:15:53,084 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:15:56,913 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:16:01,252 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:16:01,256 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:16:01,257 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:16:02,447 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:16:03,158 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:16:07,448 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:16:12,805 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:16:12,806 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:16:13,223 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:16:16,252 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:16:16,256 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:16:16,257 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:16:18,514 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:16:23,294 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:16:23,605 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:16:28,607 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:16:31,253 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:16:31,256 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:16:31,257 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:16:33,357 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:16:34,440 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:16:39,460 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:16:41,407 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:16:42,806 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:16:44,807 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:16:46,253 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:16:46,256 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:16:46,259 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:16:50,503 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:16:50,504 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 07:16:50,506 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 07:16:50,507 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 07:16:50,510 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 07:16:51,494 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:16:51,495 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 07:16:55,513 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:17:01,136 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:17:01,253 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:17:01,256 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:17:01,257 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:17:01,560 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:17:06,510 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:17:11,639 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:17:11,845 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:17:12,806 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:17:16,253 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:17:16,256 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:17:16,257 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:17:17,500 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:17:21,707 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:17:22,815 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:17:27,817 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:17:31,253 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:17:31,257 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:17:31,257 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:17:31,767 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:17:33,436 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:17:38,438 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:17:41,836 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:17:42,808 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:17:43,809 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:17:46,253 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:17:46,257 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:17:46,257 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:17:49,455 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:17:51,901 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:17:55,194 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:18:00,872 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:18:01,253 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:18:01,257 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:18:01,257 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:18:01,957 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:18:06,452 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:18:11,056 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:18:11,824 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:18:12,808 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:18:16,253 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:18:16,257 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:18:16,258 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:18:17,513 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:18:22,513 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:18:27,515 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:18:31,181 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:18:31,253 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:18:31,257 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:18:31,258 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:18:33,480 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:18:38,482 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:18:41,248 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:18:42,808 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:18:43,809 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:18:46,254 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:18:46,257 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:18:46,258 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:18:49,481 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:18:51,318 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:18:55,111 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:19:00,492 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:19:01,254 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:19:01,257 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:19:01,258 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:19:01,399 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:19:06,425 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:19:09,460 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:19:12,092 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:19:12,809 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:19:16,255 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:19:16,258 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:19:16,259 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:19:17,446 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:19:21,528 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:19:23,174 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:19:28,175 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:19:29,590 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:19:31,254 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:19:31,258 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:19:31,258 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:19:33,519 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:19:38,947 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:19:39,655 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:19:42,809 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:19:44,811 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:19:46,256 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:19:46,258 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:19:46,259 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:19:49,709 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:19:50,549 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:19:55,550 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:19:57,911 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:20:00,719 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:20:01,255 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:20:01,258 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:20:01,259 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:20:06,119 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:20:08,115 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:20:11,119 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:20:12,810 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:20:16,257 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:20:16,260 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:20:16,260 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:20:16,505 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 07:20:16,518 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 07:20:16,519 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:20:16,519 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 07:20:16,522 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 07:20:17,170 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:20:17,171 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 07:20:18,176 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:20:21,524 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:20:26,968 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:20:28,245 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:20:31,256 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:20:31,258 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:20:31,259 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:20:32,459 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:20:37,330 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:20:37,617 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:20:42,618 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:20:42,811 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:20:46,256 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:20:46,258 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:20:46,259 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:20:47,404 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:20:48,516 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:20:53,517 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:20:55,460 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:20:58,881 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:21:01,257 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:21:01,259 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:21:01,260 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:21:04,474 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:21:05,532 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:21:09,475 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:21:12,813 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:21:15,161 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:21:15,599 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:21:16,257 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:21:16,259 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:21:16,259 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:21:20,505 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:21:25,594 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:21:25,661 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:21:30,596 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:21:31,896 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:21:31,897 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:21:31,897 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:21:33,715 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:21:36,119 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:21:41,632 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:21:41,751 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:21:42,823 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:21:46,824 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:21:46,897 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:21:46,898 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:21:46,900 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:21:49,801 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:21:51,937 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:21:56,939 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:21:59,864 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:22:01,894 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:22:01,895 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:22:01,896 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:22:02,144 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:22:07,146 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:22:10,015 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:22:12,710 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:22:12,813 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:22:16,895 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:22:16,896 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:22:16,896 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:22:18,261 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:22:18,317 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:22:23,318 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:22:28,332 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:22:28,491 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:22:31,895 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:22:31,895 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:22:31,896 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:22:34,115 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:22:38,395 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:22:39,809 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:22:42,814 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:22:44,816 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:22:45,445 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:22:46,895 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:22:46,895 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:22:46,896 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:22:50,147 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:22:55,517 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:22:55,658 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:23:00,659 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:23:03,350 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:23:03,351 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:23:03,352 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:23:05,584 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:23:05,926 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:23:10,927 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:23:12,816 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:23:15,655 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:23:16,672 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:23:18,349 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:23:18,350 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:23:18,350 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:23:22,605 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:23:25,717 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:23:28,414 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:23:33,349 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:23:33,365 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:23:33,365 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:23:33,622 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:23:34,060 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 07:23:34,061 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 07:23:34,061 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 07:23:34,063 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 07:23:34,768 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 07:23:35,772 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:23:39,065 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:23:42,815 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:23:44,500 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:23:45,833 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:23:48,349 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:23:48,350 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:23:48,352 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:23:49,645 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:23:53,891 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:23:54,900 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:23:59,902 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:24:03,349 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:24:03,350 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:24:03,352 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:24:03,965 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:24:05,599 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:24:10,601 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:24:12,816 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:24:14,040 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:24:15,817 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:24:18,349 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:24:18,350 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:24:18,353 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:24:21,532 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:24:24,107 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:24:26,925 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:24:32,321 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:24:32,358 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:24:33,349 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:24:33,350 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:24:33,353 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:24:37,609 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:24:42,386 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:24:42,694 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:24:42,816 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:24:47,818 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:24:49,473 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:24:49,474 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:24:49,475 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:24:52,463 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:24:53,743 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:24:58,745 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:25:02,532 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:25:04,070 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:25:04,472 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:25:04,481 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:25:04,481 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:25:09,713 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:25:11,600 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:25:12,817 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:25:14,818 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:25:19,472 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:25:19,472 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:25:19,473 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:25:20,682 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:25:21,662 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:25:25,683 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:25:29,721 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:25:31,452 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:25:34,472 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:25:34,473 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:25:34,474 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:25:36,684 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:25:39,794 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:25:41,879 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:25:42,817 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:25:47,759 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:25:49,473 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:25:49,473 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:25:49,474 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:25:49,860 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:25:53,725 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:25:57,921 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:25:58,933 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:26:03,934 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:26:05,273 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:26:05,273 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:26:05,274 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:26:07,986 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:26:09,543 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:26:12,819 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:26:14,821 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:26:18,054 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:26:20,273 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:26:20,273 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:26:20,274 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:26:20,495 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:26:26,082 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:26:26,116 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:26:31,083 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:26:35,273 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:26:35,274 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:26:35,275 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:26:36,177 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:26:36,525 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:26:41,526 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:26:42,820 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:26:45,050 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 07:26:45,051 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 07:26:45,051 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 07:26:45,053 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 07:26:45,229 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 07:26:46,235 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:26:47,055 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:26:50,273 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:26:50,274 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:26:50,274 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:26:52,483 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:26:56,307 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:26:58,462 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:27:03,463 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:27:05,274 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:27:05,274 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:27:05,275 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:27:06,375 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:27:09,451 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:27:12,820 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:27:14,441 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:27:15,166 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:27:20,168 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:27:20,276 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:27:20,277 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:27:20,278 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:27:25,260 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:27:26,512 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:27:30,261 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:27:34,566 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:27:35,274 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:27:35,274 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:27:35,276 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:27:35,502 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:27:40,504 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:27:42,821 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:27:44,632 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:27:45,822 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:27:50,501 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:27:50,502 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:27:50,502 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:27:51,752 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:27:52,697 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:27:56,791 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:28:02,098 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:28:02,762 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:28:05,500 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:28:05,500 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:28:05,501 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:28:07,732 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:28:12,816 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:28:12,821 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:28:12,822 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:28:17,823 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:28:20,499 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:28:20,500 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:28:20,500 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:28:21,873 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:28:23,752 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:28:28,754 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:28:29,927 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:28:34,515 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:28:35,499 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:28:35,500 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:28:35,500 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:28:40,008 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:28:40,032 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:28:42,822 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:28:45,823 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:28:50,063 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:28:50,499 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:28:50,500 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:28:50,501 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:28:51,753 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:28:56,756 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:29:00,137 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:29:02,108 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:29:06,285 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:29:06,287 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:29:06,287 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:29:07,513 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:29:10,208 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:29:12,675 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:29:12,822 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:29:18,552 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:29:20,277 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:29:21,284 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:29:21,284 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:29:21,285 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:29:24,467 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:29:28,331 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:29:30,239 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:29:35,241 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:29:36,285 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:29:36,285 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:29:36,286 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:29:38,397 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:29:40,823 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:29:42,823 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:29:45,824 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:29:48,483 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:29:51,190 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:29:51,284 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:29:51,285 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:29:51,285 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:29:56,120 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 07:29:56,121 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 07:29:56,121 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 07:29:56,123 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 07:29:56,545 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 07:29:57,124 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:29:58,563 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:30:02,125 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:30:06,285 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:30:06,285 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:30:06,285 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:30:07,497 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:30:08,631 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:30:12,498 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:30:12,823 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:30:18,488 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:30:18,695 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:30:21,285 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:30:21,285 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:30:21,286 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:30:24,484 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:30:28,751 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:30:29,653 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:30:34,655 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:30:36,288 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:30:36,289 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:30:36,289 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:30:38,815 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:30:40,518 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:30:42,824 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:30:45,825 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:30:48,879 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:30:51,275 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:30:51,285 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:30:51,286 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:30:51,286 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:30:57,042 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:30:58,943 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:31:02,043 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:31:06,285 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:31:06,286 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:31:06,286 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:31:07,515 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:31:08,030 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:31:12,516 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:31:12,837 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:31:16,081 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:31:18,432 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:31:21,285 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:31:21,286 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:31:21,286 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:31:23,498 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:31:26,150 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:31:29,082 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:31:34,082 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:31:36,286 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:31:36,286 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:31:36,286 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:31:39,505 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:31:42,825 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:31:44,827 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:31:48,273 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:31:50,741 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:31:51,286 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:31:51,287 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:31:51,287 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:31:56,458 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:31:58,336 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:32:01,546 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:32:06,286 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:32:06,286 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:32:06,289 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:32:07,529 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:32:08,407 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:32:12,530 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:32:12,827 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:32:17,683 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:32:18,471 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:32:21,286 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:32:21,287 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:32:21,287 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:32:23,520 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:32:26,518 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:32:29,207 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:32:34,208 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:32:36,286 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:32:36,287 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:32:36,287 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:32:36,573 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:32:39,534 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:32:42,829 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:32:45,266 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:32:46,633 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:32:50,267 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:32:52,401 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:32:52,402 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:32:52,403 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:32:54,696 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:32:55,679 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:33:00,680 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:33:04,762 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:33:06,203 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:33:07,400 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:33:07,400 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:33:07,401 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:33:11,595 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:33:12,828 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:33:14,832 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:33:16,829 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:33:21,830 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:33:22,402 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:33:22,402 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:33:22,403 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:33:22,840 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 07:33:22,841 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 07:33:22,841 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 07:33:22,843 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 07:33:22,888 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 07:33:24,899 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:33:26,844 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:33:31,846 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:33:34,970 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:33:37,214 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:33:37,400 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:33:37,401 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:33:37,402 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:33:42,608 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:33:42,829 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:33:45,073 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:33:47,831 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:33:52,401 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:33:52,402 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:33:52,403 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:33:53,586 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:33:55,184 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:33:58,590 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:34:04,020 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:34:05,246 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:34:07,400 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:34:07,401 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:34:07,401 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:34:09,643 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:34:12,295 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:34:12,829 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:34:14,830 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:34:19,832 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:34:22,368 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:34:22,401 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:34:22,401 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:34:22,401 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:34:25,652 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:34:31,640 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:34:32,447 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:34:36,641 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:34:38,888 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:34:38,889 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:34:38,890 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:34:42,161 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:34:42,512 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:34:42,830 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:34:47,831 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:34:50,579 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:34:53,308 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:34:53,887 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:34:53,888 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:34:53,888 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:34:58,758 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:35:00,650 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:35:03,759 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:35:08,889 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:35:08,889 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:35:08,890 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:35:08,890 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:35:10,721 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:35:12,830 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:35:14,831 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:35:19,833 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:35:20,792 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:35:23,888 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:35:23,888 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:35:23,889 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:35:25,119 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:35:30,554 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:35:30,860 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:35:35,555 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:35:38,891 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:35:38,892 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:35:38,892 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:35:40,932 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:35:41,132 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:35:42,831 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:35:46,832 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:35:50,998 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:35:52,406 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:35:53,888 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:35:53,889 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:35:53,890 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:35:57,026 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:35:57,620 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:36:02,621 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:36:07,075 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:36:07,718 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:36:08,888 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:36:08,889 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:36:08,889 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:36:09,088 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:36:12,832 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:36:12,834 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:36:17,155 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:36:18,429 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:36:23,429 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:36:23,889 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:36:23,889 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:36:23,890 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:36:29,112 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:36:34,112 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:36:38,889 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:36:38,889 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:36:38,930 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:36:40,062 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:36:42,833 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:36:45,834 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:36:50,875 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:36:51,363 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:36:53,889 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:36:53,889 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:36:53,890 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:36:56,130 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:36:58,940 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 07:36:58,941 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 07:36:58,941 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 07:36:58,945 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 07:36:59,411 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 07:37:00,419 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:37:01,422 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:37:01,963 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:37:06,965 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:37:08,889 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:37:08,890 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:37:08,890 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:37:10,485 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:37:12,092 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:37:12,833 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:37:17,834 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:37:18,544 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:37:22,941 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:37:23,890 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:37:23,891 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:37:23,891 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:37:28,620 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:37:28,848 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:37:33,849 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:37:38,698 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:37:38,890 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:37:38,890 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:37:38,890 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:37:39,087 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:37:42,834 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:37:44,859 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:37:48,764 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:37:50,424 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:37:53,891 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:37:53,892 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:37:53,893 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:37:56,096 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:37:58,833 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:38:01,154 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:38:06,156 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:38:08,890 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:38:08,890 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:38:08,891 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:38:08,899 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:38:12,093 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:38:12,834 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:38:17,629 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:38:18,966 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:38:22,629 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:38:24,934 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:38:24,934 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:38:24,940 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:38:28,237 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:38:29,045 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:38:33,238 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:38:37,099 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:38:38,598 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:38:39,934 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:38:39,934 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:38:39,935 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:38:42,836 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:38:43,837 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:38:47,166 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:38:49,279 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:38:54,280 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:38:54,934 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:38:54,934 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:38:54,935 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:38:57,227 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:39:00,165 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:39:05,306 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:39:07,300 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:39:09,934 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:39:09,935 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:39:09,935 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:39:11,156 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:39:12,836 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:39:16,295 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:39:17,374 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:39:21,296 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:39:24,934 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:39:24,935 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:39:24,935 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:39:25,428 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:39:27,136 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:39:32,137 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:39:35,492 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:39:37,524 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:39:39,936 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:39:39,938 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:39:39,939 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:39:42,837 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:39:42,839 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:39:45,559 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:39:48,147 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:39:53,149 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:39:54,626 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:39:54,935 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:39:54,938 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:39:54,938 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:39:59,139 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:40:04,514 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:40:04,684 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:40:09,515 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:40:09,937 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:40:09,939 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:40:09,941 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:40:12,839 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:40:13,016 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 07:40:13,017 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 07:40:13,018 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 07:40:13,020 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 07:40:13,740 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 07:40:14,745 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:40:15,022 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:40:20,024 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:40:22,794 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:40:24,935 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:40:24,936 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:40:24,937 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:40:25,115 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:40:30,116 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:40:32,878 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:40:35,785 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:40:40,122 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:40:40,123 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:40:40,124 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:40:41,321 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:40:42,840 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:40:42,947 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:40:46,841 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:40:52,510 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:40:53,014 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:40:55,121 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:40:55,122 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:40:55,122 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:40:58,314 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:41:03,090 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:41:04,039 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:41:09,040 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:41:10,325 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:41:10,326 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:41:10,327 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:41:12,838 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:41:13,153 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:41:14,839 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:41:19,841 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:41:23,210 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:41:25,325 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:41:25,326 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:41:25,326 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:41:25,326 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:41:31,230 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:41:31,274 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:41:36,231 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:41:40,325 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:41:40,326 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:41:40,326 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:41:41,334 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:41:41,594 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:41:42,839 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:41:46,840 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:41:51,393 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:41:51,886 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:41:55,326 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:41:55,326 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:41:55,326 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:41:57,530 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:42:01,482 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:42:02,759 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:42:07,760 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:42:10,326 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:42:10,326 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:42:10,326 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:42:11,560 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:42:12,839 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:42:12,840 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:42:17,841 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:42:19,616 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:42:23,255 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:42:26,224 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:42:26,225 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:42:26,226 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:42:28,365 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:42:29,680 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:42:33,366 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:42:38,893 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:42:39,744 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:42:41,224 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:42:41,225 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:42:41,225 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:42:42,840 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:42:44,841 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:42:47,798 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:42:50,410 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:42:55,411 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:42:56,225 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:42:56,226 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:42:56,227 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:42:56,855 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:43:00,425 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:43:05,427 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:43:06,921 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:43:10,816 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:43:11,224 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:43:11,225 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:43:11,225 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:43:12,842 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:43:16,440 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:43:16,998 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:43:21,441 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:43:25,164 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 07:43:25,165 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 07:43:25,165 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 07:43:25,167 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 07:43:26,069 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 07:43:26,224 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:43:26,226 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:43:26,226 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:43:27,074 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:43:27,387 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:43:32,388 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:43:37,139 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:43:37,848 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:43:39,149 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:43:41,224 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:43:41,225 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:43:41,225 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:43:42,844 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:43:43,846 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:43:47,210 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:43:49,834 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:43:54,836 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:43:56,224 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:43:56,225 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:43:56,225 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:43:57,280 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:44:00,471 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:44:05,472 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:44:07,340 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:44:10,628 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:44:11,225 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:44:11,225 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:44:11,226 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:44:12,845 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:44:15,815 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:44:17,406 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:44:20,816 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:44:25,469 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:44:26,225 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:44:26,226 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:44:26,226 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:44:26,531 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:44:31,533 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:44:35,539 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:44:36,545 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:44:41,547 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:44:42,590 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:44:42,591 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:44:42,591 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:44:42,846 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:44:45,599 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:44:46,789 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:44:51,791 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:44:55,672 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:44:57,322 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:44:57,589 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:44:57,590 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:44:57,590 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:45:02,765 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:45:05,742 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:45:07,842 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:45:12,590 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:45:12,590 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:45:12,650 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:45:12,845 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:45:12,846 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:45:13,796 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:45:17,847 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:45:23,771 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:45:23,867 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:45:27,589 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:45:27,590 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:45:27,590 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:45:28,839 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:45:33,933 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:45:34,127 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:45:39,129 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:45:42,590 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:45:42,590 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:45:42,591 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:45:42,846 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:45:44,013 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:45:44,847 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:45:49,848 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:45:51,062 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:45:55,474 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:45:58,561 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:45:58,571 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:45:58,572 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:46:00,724 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:46:01,124 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:46:05,725 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:46:11,362 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:46:11,411 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:46:12,847 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:46:13,560 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:46:13,570 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:46:13,570 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:46:16,789 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:46:21,422 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:46:22,154 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:46:27,155 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:46:28,561 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:46:28,571 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:46:28,571 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:46:29,468 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:46:32,808 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:46:37,809 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:46:38,046 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 07:46:38,047 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 07:46:38,047 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 07:46:38,050 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 07:46:38,534 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 07:46:39,539 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:46:42,847 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:46:42,848 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:46:43,560 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:46:43,570 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:46:43,571 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:46:48,838 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:46:49,606 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:46:53,838 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:46:58,561 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:46:58,570 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:46:58,571 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:46:59,676 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:46:59,737 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:47:04,739 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:47:09,751 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:47:10,020 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:47:12,848 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:47:13,562 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:47:13,572 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:47:13,574 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:47:15,736 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:47:17,808 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:47:20,842 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:47:25,843 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:47:27,875 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:47:28,561 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:47:28,571 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:47:28,571 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:47:31,805 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:47:37,773 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:47:37,940 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:47:42,774 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:47:42,850 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:47:44,016 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:47:44,017 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:47:44,019 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:47:48,001 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:47:48,229 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:47:53,230 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:47:56,053 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:47:58,591 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:47:59,014 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:47:59,015 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:47:59,015 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:48:04,127 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:48:06,122 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:48:09,128 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:48:12,850 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:48:14,015 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:48:14,015 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:48:14,015 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:48:14,270 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:48:16,180 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:48:19,271 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:48:24,667 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:48:25,236 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:48:29,015 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:48:29,015 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:48:29,015 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:48:30,167 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:48:35,297 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:48:36,098 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:48:41,099 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:48:42,851 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:48:43,354 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:48:44,015 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:48:44,015 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:48:44,016 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:48:46,246 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:48:51,248 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:48:53,422 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:48:56,496 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:48:59,017 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:48:59,018 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:48:59,019 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:49:01,795 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:49:03,480 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:49:06,796 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:49:12,401 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:49:12,851 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:49:13,546 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:49:14,015 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:49:14,016 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:49:14,016 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:49:18,206 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:49:21,602 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:49:23,992 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:49:28,994 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:49:29,136 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:49:29,137 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:49:29,138 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:49:31,670 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:49:34,338 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:49:39,339 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:49:41,743 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:49:42,852 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:49:44,135 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:49:44,135 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:49:44,135 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:49:45,298 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:49:49,670 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 07:49:49,671 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 07:49:49,671 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 07:49:49,673 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 07:49:49,792 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 07:49:50,675 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:49:51,801 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:49:55,676 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:49:59,137 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:49:59,138 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:49:59,138 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:50:01,394 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:50:01,872 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:50:06,395 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:50:11,825 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:50:11,938 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:50:12,865 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:50:14,135 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:50:14,136 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:50:14,136 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:50:17,391 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:50:22,001 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:50:23,219 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:50:28,221 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:50:29,135 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:50:29,136 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:50:29,136 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:50:30,059 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:50:33,294 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:50:38,295 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:50:42,853 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:50:44,136 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:50:44,136 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:50:44,137 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:50:44,137 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:50:46,157 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:50:49,342 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:50:54,208 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:50:54,886 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:50:59,137 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:50:59,139 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:50:59,140 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:51:00,305 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:51:04,284 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:51:05,746 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:51:10,747 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:51:12,853 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:51:14,136 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:51:14,137 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:51:14,137 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:51:14,350 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:51:16,356 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:51:21,358 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:51:23,415 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:51:26,577 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:51:29,137 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:51:29,139 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:51:29,140 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:51:32,052 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:51:33,480 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:51:37,053 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:51:42,511 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:51:42,854 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:51:43,553 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:51:44,136 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:51:44,137 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:51:44,137 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:51:48,392 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:51:51,600 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:51:53,792 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:51:58,793 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:51:59,137 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:51:59,138 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:51:59,140 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:52:01,662 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:52:04,166 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:52:09,168 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:52:11,726 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:52:12,854 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:52:14,137 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:52:14,138 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:52:14,138 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:52:14,386 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:52:19,387 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:52:21,800 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:52:24,824 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:52:29,137 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:52:29,138 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:52:29,138 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:52:30,372 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:52:31,864 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:52:35,373 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:52:39,924 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:52:40,788 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:52:42,855 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:52:44,137 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:52:44,138 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:52:44,141 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:52:46,354 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:52:50,088 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:52:52,318 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:52:57,319 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:52:59,137 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:52:59,138 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:52:59,139 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:53:00,148 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:53:03,310 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:53:07,610 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 07:53:07,611 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 07:53:07,611 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 07:53:07,613 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 07:53:08,195 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 07:53:08,615 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:53:10,203 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:53:12,855 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:53:13,856 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:53:14,139 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:53:14,140 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:53:14,141 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:53:18,255 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:53:19,059 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:53:24,060 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:53:28,437 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:53:29,137 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:53:29,138 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:53:29,139 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:53:29,139 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:53:34,313 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:53:38,493 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:53:39,685 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:53:42,857 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:53:44,138 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:53:44,139 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:53:44,140 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:53:44,907 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:53:46,547 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:53:49,908 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:53:55,101 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:53:55,611 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:53:59,138 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:53:59,139 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:53:59,139 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:54:00,318 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:54:05,673 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:54:06,102 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:54:11,104 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:54:12,857 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:54:14,138 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:54:14,139 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:54:14,139 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:54:15,742 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:54:16,372 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:54:21,374 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:54:23,794 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:54:27,296 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:54:29,138 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:54:29,139 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:54:29,140 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:54:33,045 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:54:33,866 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:54:38,046 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:54:42,858 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:54:43,859 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:54:43,929 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:54:44,138 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:54:44,139 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:54:44,140 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:54:49,352 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:54:54,018 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:54:54,582 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:54:59,141 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:54:59,144 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:54:59,145 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:55:00,341 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:55:04,094 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:55:06,090 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:55:11,091 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:55:12,149 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:55:12,858 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:55:14,139 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:55:14,141 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:55:14,141 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:55:16,387 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:55:21,896 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:55:22,213 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:55:26,897 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:55:29,141 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:55:29,142 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:55:29,144 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:55:32,171 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:55:32,362 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:55:37,172 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:55:42,433 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:55:42,588 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:55:42,859 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:55:44,139 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:55:44,141 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:55:44,142 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:55:48,348 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:55:50,494 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:55:53,507 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:55:58,509 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:55:59,139 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:55:59,141 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:55:59,142 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:56:00,564 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:56:04,361 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:56:09,686 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:56:10,638 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:56:12,859 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:56:14,139 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:56:14,141 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:56:14,142 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:56:15,338 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:56:18,131 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 07:56:18,133 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 07:56:18,133 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 07:56:18,135 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 07:56:18,697 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 07:56:19,704 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:56:20,709 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:56:21,138 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:56:26,141 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:56:29,139 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:56:29,142 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:56:29,142 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:56:29,778 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:56:31,377 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:56:36,379 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:56:37,829 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:56:42,329 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:56:42,860 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:56:44,141 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:56:44,143 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:56:44,145 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:56:47,349 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:56:47,892 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:56:52,350 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:56:57,954 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:56:57,963 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:56:59,140 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:56:59,142 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:56:59,142 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:57:03,337 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:57:08,016 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:57:08,530 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:57:12,861 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:57:13,864 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:57:14,825 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:57:14,843 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:57:14,843 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:57:18,096 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:57:19,150 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:57:24,152 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:57:26,188 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:57:29,643 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:57:29,825 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:57:29,825 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:57:29,826 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:57:35,054 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:57:36,251 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:57:40,553 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:57:42,863 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:57:44,825 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:57:44,826 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:57:44,826 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:57:45,997 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:57:46,312 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:57:50,998 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:57:56,376 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:57:56,799 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:57:59,825 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:57:59,826 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:57:59,826 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:58:01,995 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:58:06,437 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:58:07,155 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:58:12,157 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:58:12,863 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:58:14,825 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:58:14,826 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:58:14,826 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:58:16,502 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:58:18,029 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:58:23,030 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:58:26,562 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:58:28,094 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:58:29,825 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:58:29,826 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:58:29,826 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:58:33,483 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:58:34,614 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:58:38,483 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:58:42,864 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:58:44,292 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:58:44,675 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:58:44,825 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:58:44,826 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:58:44,826 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:58:50,009 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:58:54,749 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:58:55,713 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:59:00,714 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:59:01,041 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:59:01,042 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:59:01,043 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:59:04,813 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:59:06,267 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:59:11,268 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:59:12,864 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:59:12,873 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:59:16,040 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:59:16,040 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:59:16,041 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:59:17,212 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:59:21,938 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:59:22,229 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:59:27,230 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:59:31,042 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:59:31,043 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:59:31,044 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:59:31,360 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 07:59:31,361 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 07:59:31,361 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 07:59:31,364 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 07:59:32,010 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:59:32,011 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 07:59:32,366 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:59:37,367 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:59:42,076 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:59:42,742 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:59:42,874 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 07:59:46,040 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 07:59:46,041 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 07:59:46,041 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 07:59:48,301 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:59:52,147 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 07:59:54,213 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 07:59:59,214 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:00:01,041 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:00:01,042 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:00:01,042 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:00:02,208 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:00:04,225 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:00:09,227 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:00:12,269 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:00:12,865 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:00:14,866 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:00:16,041 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:00:16,041 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:00:16,042 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:00:20,122 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:00:22,338 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:00:25,123 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:00:30,398 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:00:30,491 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:00:31,041 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:00:31,042 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:00:31,042 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:00:36,287 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:00:40,474 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:00:42,135 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:00:42,866 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:00:47,469 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:00:47,470 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:00:47,471 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:00:47,700 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:00:50,550 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:00:53,698 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:00:59,383 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:01:00,601 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:01:02,469 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:01:02,469 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:01:02,470 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:01:04,718 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:01:10,239 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:01:10,665 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:01:12,866 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:01:15,867 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:01:17,470 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:01:17,471 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:01:17,471 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:01:18,717 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:01:21,697 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:01:26,698 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:01:28,788 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:01:31,780 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:01:32,470 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:01:32,471 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:01:32,471 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:01:37,021 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:01:38,852 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:01:42,022 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:01:42,868 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:01:46,906 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:01:47,470 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:01:47,471 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:01:47,471 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:01:47,471 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:01:52,689 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:01:56,962 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:01:58,601 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:02:03,603 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:02:03,618 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:02:03,619 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:02:03,619 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:02:07,021 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:02:08,735 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:02:12,870 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:02:13,871 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:02:16,125 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:02:18,618 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:02:18,619 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:02:18,619 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:02:19,814 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:02:25,358 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:02:26,197 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:02:30,359 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:02:33,618 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:02:33,619 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:02:33,629 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:02:34,250 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:02:35,876 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:02:40,877 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:02:42,871 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:02:44,320 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 08:02:44,321 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 08:02:44,321 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 08:02:44,322 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 08:02:45,325 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 08:02:46,324 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:02:46,330 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:02:48,619 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:02:48,619 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:02:48,629 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:02:51,828 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:02:54,384 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:02:57,710 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:03:02,711 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:03:03,619 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:03:03,620 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:03:03,629 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:03:04,452 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:03:07,869 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:03:12,871 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:03:12,871 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:03:14,512 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:03:17,872 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:03:18,619 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:03:18,620 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:03:18,629 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:03:23,432 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:03:24,587 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:03:28,433 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:03:33,619 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:03:33,620 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:03:33,630 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:03:33,837 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:03:34,655 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:03:38,838 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:03:42,871 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:03:43,877 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:03:44,824 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:03:48,619 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:03:48,620 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:03:48,630 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:03:48,907 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:03:54,660 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:03:54,891 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:03:59,661 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:04:02,947 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:04:03,619 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:04:03,620 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:04:03,630 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:04:04,905 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:04:09,907 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:04:12,872 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:04:13,011 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:04:15,873 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:04:18,620 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:04:18,621 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:04:18,630 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:04:21,851 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:04:23,071 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:04:26,979 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:04:32,524 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:04:33,140 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:04:33,620 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:04:33,620 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:04:33,630 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:04:37,907 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:04:42,872 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:04:43,201 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:04:43,873 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:04:48,620 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:04:48,621 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:04:48,630 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:04:48,906 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:04:53,908 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:04:57,291 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:04:59,257 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:05:03,962 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:05:03,963 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:05:03,964 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:05:05,122 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:05:06,347 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:05:10,123 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:05:12,873 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:05:15,568 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:05:16,427 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:05:18,962 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:05:18,962 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:05:18,963 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:05:21,194 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:05:26,496 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:05:26,573 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:05:31,574 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:05:33,964 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:05:33,964 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:05:33,965 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:05:36,569 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:05:37,334 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:05:42,336 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:05:42,875 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:05:46,630 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:05:47,403 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:05:48,962 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:05:48,963 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:05:48,964 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:05:53,269 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:05:56,691 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:05:58,320 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:06:03,321 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:06:03,962 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:06:03,963 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:06:03,963 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:06:04,030 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 08:06:04,237 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 08:06:04,237 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 08:06:04,239 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 08:06:04,749 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 08:06:06,758 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:06:09,241 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:06:12,876 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:06:14,820 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:06:15,215 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:06:18,962 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:06:18,963 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:06:18,963 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:06:21,127 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:06:24,884 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:06:26,957 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:06:31,959 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:06:33,962 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:06:33,963 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:06:33,964 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:06:34,943 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:06:37,205 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:06:42,207 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:06:42,876 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:06:45,010 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:06:47,878 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:06:48,965 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:06:48,967 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:06:48,968 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:06:53,075 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:06:53,228 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:06:58,229 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:07:03,138 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:07:03,828 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:07:03,963 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:07:03,963 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:07:03,964 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:07:09,144 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:07:12,877 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:07:13,207 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:07:14,878 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:07:19,807 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:07:19,808 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:07:19,810 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:07:19,964 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:07:23,272 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:07:24,973 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:07:29,974 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:07:31,332 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:07:34,805 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:07:34,806 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:07:34,806 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:07:35,037 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:07:40,632 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:07:41,397 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:07:42,878 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:07:45,879 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:07:49,805 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:07:49,806 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:07:49,806 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:07:51,040 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:07:51,469 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:07:56,041 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:08:01,459 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:08:01,535 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:08:04,805 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:08:04,806 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:08:04,806 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:08:07,073 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:08:10,592 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:08:12,666 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:08:12,878 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:08:17,879 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:08:18,640 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:08:19,805 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:08:19,806 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:08:19,806 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:08:22,962 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:08:28,358 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:08:28,710 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:08:33,359 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:08:35,693 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:08:35,694 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:08:35,696 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:08:38,801 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:08:39,068 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:08:42,879 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:08:44,880 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:08:46,851 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:08:50,551 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:08:50,692 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:08:50,693 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:08:50,693 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:08:55,891 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:08:58,926 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:09:01,720 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:09:05,694 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:09:05,695 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:09:05,695 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:09:06,987 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:09:07,077 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:09:12,078 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:09:12,881 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:09:15,713 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 08:09:15,715 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 08:09:15,715 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 08:09:15,718 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 08:09:16,040 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 08:09:17,046 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:09:17,726 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:09:20,692 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:09:20,694 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:09:20,694 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:09:22,886 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:09:27,114 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:09:28,052 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:09:33,054 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:09:35,693 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:09:35,693 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:09:35,694 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:09:37,184 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:09:38,909 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:09:42,881 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:09:44,077 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:09:45,237 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:09:49,078 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:09:50,695 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:09:50,697 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:09:50,697 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:09:54,615 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:09:55,301 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:09:59,616 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:10:05,176 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:10:05,374 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:10:05,693 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:10:05,694 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:10:05,694 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:10:10,930 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:10:12,881 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:10:15,452 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:10:16,882 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:10:20,925 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:10:20,926 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:10:20,928 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:10:22,105 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:10:23,506 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:10:27,169 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:10:32,958 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:10:33,569 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:10:35,924 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:10:35,925 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:10:35,925 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:10:38,184 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:10:42,882 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:10:43,639 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:10:43,883 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:10:48,884 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:10:50,924 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:10:50,925 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:10:50,925 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:10:53,724 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:10:54,169 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:10:59,171 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:11:02,788 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:11:04,561 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:11:05,924 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:11:05,925 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:11:05,925 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:11:09,852 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:11:10,873 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:11:12,883 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:11:14,884 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:11:20,075 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:11:20,925 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:11:20,925 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:11:20,925 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:11:20,950 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:11:25,119 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:11:30,767 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:11:31,002 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:11:35,769 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:11:36,961 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:11:36,962 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:11:36,964 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:11:39,059 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:11:41,173 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:11:42,883 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:11:46,884 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:11:49,119 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:11:51,960 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:11:51,961 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:11:51,961 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:11:52,195 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:11:57,737 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:11:59,180 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:12:02,738 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:12:06,960 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:12:06,961 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:12:06,961 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:12:07,237 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:12:08,205 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:12:12,884 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:12:13,885 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:12:17,302 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:12:19,686 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:12:21,962 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:12:21,980 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:12:21,982 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:12:24,815 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:12:24,816 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 08:12:24,819 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 08:12:24,819 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 08:12:24,822 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 08:12:25,354 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 08:12:27,364 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:12:29,825 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:12:35,405 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:12:36,965 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:12:36,980 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:12:36,981 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:12:37,433 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:12:41,228 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:12:42,887 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:12:45,490 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:12:46,879 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:12:51,880 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:12:52,077 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:12:52,086 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:12:52,088 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:12:55,553 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:12:57,465 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:13:02,466 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:13:05,613 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:13:07,075 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:13:07,076 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:13:07,076 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:13:08,258 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:13:12,886 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:13:13,887 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:13:15,675 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:13:18,888 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:13:22,079 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:13:22,080 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:13:22,081 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:13:23,729 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:13:24,253 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:13:29,254 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:13:33,802 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:13:34,780 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:13:37,076 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:13:37,076 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:13:37,077 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:13:40,332 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:13:42,887 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:13:43,863 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:13:45,888 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:13:51,662 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:13:52,076 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:13:52,077 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:13:52,077 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:13:53,919 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:13:57,232 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:14:00,961 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:14:02,446 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:14:07,076 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:14:07,077 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:14:07,078 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:14:08,339 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:14:11,030 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:14:12,887 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:14:13,888 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:14:18,890 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:14:21,089 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:14:22,076 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:14:22,077 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:14:22,077 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:14:24,228 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:14:29,951 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:14:31,143 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:14:34,952 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:14:37,608 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:14:37,609 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:14:37,610 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:14:40,784 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:14:41,211 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:14:42,888 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:14:45,889 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:14:49,270 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:14:51,509 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:14:52,607 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:14:52,608 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:14:52,608 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:14:56,805 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:14:59,333 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:15:02,774 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:15:07,607 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:15:07,607 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:15:07,608 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:15:07,789 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:15:09,394 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:15:12,790 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:15:12,888 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:15:18,240 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:15:19,455 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:15:22,607 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:15:22,608 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:15:22,609 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:15:23,826 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:15:29,112 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:15:29,521 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:15:34,113 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:15:37,009 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 08:15:37,010 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 08:15:37,011 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 08:15:37,014 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 08:15:37,572 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 08:15:37,607 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:15:37,608 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:15:37,608 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:15:39,580 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:15:39,826 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:15:42,888 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:15:44,890 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:15:47,639 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:15:50,260 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:15:52,607 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:15:52,608 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:15:52,608 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:15:55,774 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:15:57,704 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:16:00,786 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:16:06,285 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:16:07,625 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:16:07,626 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:16:07,626 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:16:07,762 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:16:11,828 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:16:12,890 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:16:15,822 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:16:17,661 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:16:22,663 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:16:22,668 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:16:22,669 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:16:22,671 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:16:25,910 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:16:27,838 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:16:32,839 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:16:35,969 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:16:37,667 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:16:37,668 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:16:37,669 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:16:37,853 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:16:42,890 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:16:42,891 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:16:46,032 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:16:47,891 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:16:52,667 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:16:52,668 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:16:52,668 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:16:52,920 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:16:54,096 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:16:57,921 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:17:03,659 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:17:04,156 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:17:07,667 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:17:07,668 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:17:07,669 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:17:08,899 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:17:12,890 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:17:13,213 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:17:14,891 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:17:19,893 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:17:22,668 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:17:22,668 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:17:22,669 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:17:23,279 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:17:25,872 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:17:31,346 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:17:31,439 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:17:36,440 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:17:38,524 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:17:38,525 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:17:38,526 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:17:41,412 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:17:41,672 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:17:42,892 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:17:46,893 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:17:51,475 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:17:52,019 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:17:53,523 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:17:53,524 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:17:53,524 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:17:57,785 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:17:59,545 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:18:03,599 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:18:08,523 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:18:08,523 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:18:08,523 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:18:08,766 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:18:09,608 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:18:12,892 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:18:13,893 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:18:19,676 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:18:19,854 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:18:23,523 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:18:23,524 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:18:23,524 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:18:25,709 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:18:29,743 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:18:31,101 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:18:36,102 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:18:38,523 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:18:38,524 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:18:38,524 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:18:39,809 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:18:41,699 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:18:42,893 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:18:46,410 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 08:18:46,411 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 08:18:46,411 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 08:18:46,413 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 08:18:46,848 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:18:46,849 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 08:18:47,421 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:18:47,854 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:18:52,422 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:18:53,523 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:18:53,524 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:18:53,524 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:18:57,741 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:19:01,948 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:19:03,730 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:19:08,731 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:19:09,508 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:19:09,509 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:19:09,509 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:19:12,012 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:19:12,893 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:19:13,894 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:19:18,895 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:19:22,079 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:19:24,506 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:19:24,506 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:19:24,507 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:19:24,507 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:19:29,668 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:19:32,138 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:19:34,872 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:19:39,509 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:19:39,510 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:19:39,511 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:19:40,771 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:19:42,200 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:19:42,893 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:19:45,894 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:19:51,608 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:19:52,264 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:19:54,507 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:19:54,508 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:19:54,508 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:19:56,802 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:20:02,080 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:20:02,327 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:20:07,082 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:20:09,381 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:20:09,507 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:20:09,507 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:20:09,508 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:20:12,743 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:20:12,894 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:20:17,895 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:20:19,435 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:20:23,380 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:20:24,507 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:20:24,508 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:20:24,508 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:20:28,915 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:20:29,501 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:20:33,915 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:20:39,180 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:20:39,507 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:20:39,508 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:20:39,508 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:20:39,563 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:20:42,894 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:20:44,895 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:20:49,633 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:20:50,590 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:20:55,591 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:20:55,824 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:20:55,825 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:20:55,826 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:20:59,691 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:21:01,129 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:21:06,130 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:21:07,749 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:21:10,823 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:21:10,824 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:21:10,824 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:21:12,078 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:21:12,895 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:21:17,718 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:21:17,811 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:21:22,719 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:21:25,826 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:21:25,826 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:21:25,827 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:21:27,878 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:21:28,211 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:21:33,212 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:21:37,939 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:21:38,542 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:21:40,823 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:21:40,824 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:21:40,824 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:21:42,896 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:21:43,898 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:21:46,000 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:21:49,627 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:21:55,127 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:21:55,823 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:21:55,824 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:21:55,824 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:21:56,066 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:22:01,070 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:22:04,120 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 08:22:04,121 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 08:22:04,121 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 08:22:04,123 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 08:22:05,124 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:22:05,125 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 08:22:06,124 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:22:06,133 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:22:11,125 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:22:12,224 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:22:12,225 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:22:12,225 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:22:12,896 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:22:16,210 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:22:16,722 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:22:21,723 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:22:26,279 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:22:27,223 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:22:27,223 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:22:27,224 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:22:27,225 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:22:32,444 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:22:36,356 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:22:37,454 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:22:42,225 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:22:42,227 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:22:42,228 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:22:42,897 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:22:42,898 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:22:44,414 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:22:48,158 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:22:53,808 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:22:54,485 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:22:57,223 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:22:57,225 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:22:57,226 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:22:59,413 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:23:04,513 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:23:04,551 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:23:09,514 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:23:12,223 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:23:12,225 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:23:12,226 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:23:12,898 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:23:13,610 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:23:14,898 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:23:19,900 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:23:23,674 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:23:25,291 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:23:27,224 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:23:27,225 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:23:27,226 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:23:30,436 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:23:33,741 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:23:35,596 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:23:40,597 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:23:41,938 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:23:42,224 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:23:42,226 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:23:42,226 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:23:42,898 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:23:45,899 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:23:50,900 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:23:52,008 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:23:55,973 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:23:57,225 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:23:57,227 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:23:57,228 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:24:01,250 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:24:02,077 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:24:06,251 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:24:11,835 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:24:12,136 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:24:12,224 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:24:12,226 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:24:12,226 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:24:12,899 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:24:16,901 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:24:20,196 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:24:21,993 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:24:26,995 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:24:27,253 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:24:27,254 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:24:27,254 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:24:30,252 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:24:32,490 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:24:37,491 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:24:40,318 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:24:42,253 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:24:42,253 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:24:42,253 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:24:42,899 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:24:42,899 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:24:48,332 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:24:48,376 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:24:53,333 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:24:57,253 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:24:57,254 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:24:57,254 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:24:58,440 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:24:58,446 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:25:03,442 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:25:08,515 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:25:08,518 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:25:12,253 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:25:12,253 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:25:12,254 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:25:12,902 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:25:13,904 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:25:15,810 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 08:25:15,811 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 08:25:15,811 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 08:25:15,813 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 08:25:16,571 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 08:25:17,578 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:25:18,582 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:25:19,815 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:25:24,816 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:25:26,638 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:25:27,253 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:25:27,263 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:25:27,264 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:25:30,499 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:25:35,512 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:25:36,699 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:25:40,513 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:25:42,253 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:25:42,254 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:25:42,254 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:25:42,901 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:25:45,902 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:25:50,903 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:25:55,908 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:25:56,827 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:25:57,253 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:25:57,254 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:25:57,254 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:26:02,489 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:26:05,973 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:26:07,550 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:26:12,254 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:26:12,255 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:26:12,256 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:26:12,902 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:26:12,903 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:26:14,026 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:26:18,041 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:26:23,117 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:26:24,093 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:26:27,254 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:26:27,255 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:26:27,255 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:26:28,539 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:26:33,776 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:26:34,164 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:26:38,778 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:26:42,225 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:26:42,254 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:26:42,255 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:26:42,255 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:26:42,902 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:26:43,903 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:26:48,904 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:26:52,286 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:26:54,880 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:26:57,256 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:26:57,257 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:26:57,259 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:27:00,307 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:27:02,353 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:27:05,308 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:27:10,758 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:27:12,254 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:27:12,254 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:27:12,300 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:27:12,416 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:27:12,903 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:27:15,916 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:27:20,474 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:27:21,902 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:27:26,903 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:27:27,256 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:27:27,257 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:27:27,300 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:27:30,533 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:27:32,185 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:27:37,186 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:27:40,609 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:27:42,255 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:27:42,255 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:27:42,256 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:27:42,256 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:27:42,903 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:27:47,275 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:27:48,667 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:27:52,276 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:27:57,255 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:27:57,255 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:27:57,257 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:27:57,419 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:27:58,740 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:28:02,420 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:28:07,975 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:28:08,811 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:28:12,256 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:28:12,257 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:28:12,258 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:28:12,905 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:28:13,907 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:28:18,878 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:28:19,659 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:28:24,660 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:28:25,926 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:28:27,255 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:28:27,256 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:28:27,256 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:28:30,468 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:28:35,020 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 08:28:35,021 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 08:28:35,022 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 08:28:35,025 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 08:28:35,990 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:28:35,991 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 08:28:36,032 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:28:41,033 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:28:42,906 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:28:43,111 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:28:43,112 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:28:43,113 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:28:46,153 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:28:46,312 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:28:51,313 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:28:56,210 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:28:56,905 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:28:58,109 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:28:58,110 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:28:58,110 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:29:02,349 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:29:04,271 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:29:08,342 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:29:12,906 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:29:13,110 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:29:13,110 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:29:13,111 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:29:13,357 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:29:14,339 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:29:18,357 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:29:24,088 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:29:24,404 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:29:28,110 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:29:28,110 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:29:28,111 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:29:29,267 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:29:34,475 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:29:35,223 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:29:40,225 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:29:42,907 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:29:43,110 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:29:43,110 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:29:43,111 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:29:44,539 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:29:45,360 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:29:50,361 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:29:52,585 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:29:56,161 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:29:59,355 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:29:59,355 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:29:59,356 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:30:01,574 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:30:02,653 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:30:06,574 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:30:12,205 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:30:12,720 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:30:12,907 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:30:14,354 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:30:14,355 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:30:14,355 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:30:17,634 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:30:20,773 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:30:23,313 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:30:28,315 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:30:29,355 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:30:29,355 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:30:29,356 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:30:30,841 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:30:33,566 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:30:38,567 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:30:40,918 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:30:42,908 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:30:43,908 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:30:44,355 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:30:44,356 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:30:44,356 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:30:49,541 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:30:50,983 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:30:54,542 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:30:59,355 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:30:59,356 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:30:59,356 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:30:59,603 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:31:01,050 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:31:04,605 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:31:09,893 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:31:10,114 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:31:12,908 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:31:14,355 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:31:14,356 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:31:14,356 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:31:15,541 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:31:18,165 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:31:20,660 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:31:25,662 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:31:28,228 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:31:29,355 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:31:29,356 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:31:29,365 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:31:31,585 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:31:36,586 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:31:38,299 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:31:41,641 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:31:42,909 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:31:45,014 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:31:45,015 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:31:45,016 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:31:46,330 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 08:31:46,331 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 08:31:46,331 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 08:31:46,332 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 08:31:46,352 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 08:31:47,333 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:31:48,365 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:31:52,334 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:31:56,426 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:31:57,842 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:32:00,013 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:32:00,013 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:32:00,014 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:32:03,276 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:32:06,496 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:32:08,490 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:32:12,910 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:32:13,912 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:32:15,013 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:32:15,014 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:32:15,015 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:32:18,583 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:32:19,102 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:32:24,104 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:32:26,637 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:32:30,013 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:32:30,013 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:32:30,014 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:32:30,297 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:32:35,553 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:32:36,701 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:32:40,554 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:32:42,912 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:32:44,754 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:32:45,013 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:32:45,014 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:32:45,014 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:32:46,233 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:32:51,235 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:32:54,817 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:32:56,443 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:33:01,444 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:33:01,573 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:33:01,574 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:33:01,575 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:33:04,920 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:33:06,743 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:33:11,744 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:33:12,911 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:33:14,982 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:33:16,572 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:33:16,572 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:33:16,572 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:33:16,775 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:33:21,776 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:33:27,175 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:33:29,074 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:33:31,571 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:33:31,572 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:33:31,572 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:33:32,775 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:33:37,135 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:33:37,936 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:33:42,913 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:33:43,916 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:33:46,571 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:33:46,572 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:33:46,572 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:33:47,203 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:33:49,835 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:33:55,826 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:33:57,268 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:34:00,827 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:34:01,573 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:34:01,574 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:34:01,576 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:34:06,390 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:34:06,505 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:34:11,507 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:34:12,915 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:34:16,466 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:34:16,572 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:34:16,572 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:34:16,573 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:34:16,843 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:34:21,845 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:34:24,516 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:34:27,697 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:34:31,824 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:34:31,825 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:34:31,826 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:34:33,043 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:34:34,581 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:34:38,044 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:34:42,915 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:34:43,916 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:34:44,736 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:34:46,823 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:34:46,824 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:34:46,824 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:34:49,109 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:34:53,023 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:34:54,939 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:34:59,940 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:35:01,670 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 08:35:01,672 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 08:35:01,673 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 08:35:01,675 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 08:35:01,823 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:35:01,823 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:35:01,824 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:35:02,100 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 08:35:03,105 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:35:04,109 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:35:05,113 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:35:05,637 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:35:10,638 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:35:12,915 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:35:15,916 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:35:16,823 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:35:16,824 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:35:16,824 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:35:21,008 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:35:26,009 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:35:31,010 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:35:31,823 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:35:31,824 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:35:31,824 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:35:36,039 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:35:41,040 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:35:42,916 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:35:46,823 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:35:46,824 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:35:46,824 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:35:46,825 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:35:52,056 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:35:57,057 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:36:01,824 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:36:01,824 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:36:01,825 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:36:02,058 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:36:07,059 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:36:12,060 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:36:12,916 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:36:16,824 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:36:16,824 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:36:16,825 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:36:17,074 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:36:22,075 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:36:27,075 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:36:31,824 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:36:31,824 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:36:31,825 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:36:32,990 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:36:37,991 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:36:42,917 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:36:43,917 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:36:46,824 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:36:46,824 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:36:46,825 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:36:49,057 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:36:54,057 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:36:59,058 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:37:01,824 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:37:01,825 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:37:01,825 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:37:04,068 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:37:09,069 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:37:12,918 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:37:14,919 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:37:16,824 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:37:16,825 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:37:16,826 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:37:20,035 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:37:25,036 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:37:30,037 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:37:31,825 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:37:31,825 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:37:31,825 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:37:36,011 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:37:41,012 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:37:42,918 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:37:46,824 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:37:46,825 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:37:46,825 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:37:46,825 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:37:52,069 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:37:57,069 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:38:01,825 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:38:01,825 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:38:01,826 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:38:03,058 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:38:08,058 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:38:12,918 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:38:13,919 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:38:16,825 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:38:16,826 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:38:16,826 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:38:19,047 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:38:24,048 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:38:29,049 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:38:31,825 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:38:31,825 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:38:31,826 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:38:35,000 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:38:40,001 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:38:42,919 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:38:45,920 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:38:46,825 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:38:46,825 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:38:46,826 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:38:51,094 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:38:56,095 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:39:01,095 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:39:01,825 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:39:01,826 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:39:01,826 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:39:07,042 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:39:12,043 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:39:12,929 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:39:16,825 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:39:16,826 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:39:16,826 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:39:17,092 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:39:22,092 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:39:27,093 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:39:31,826 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:39:31,826 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:39:31,827 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:39:33,043 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:39:38,044 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:39:42,920 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:39:43,921 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:39:46,826 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:39:46,826 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:39:46,826 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:39:49,044 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:39:54,045 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:39:59,046 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:40:01,825 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:40:01,826 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:40:01,827 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:40:04,056 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:40:09,057 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:40:12,920 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:40:14,921 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:40:16,826 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:40:16,826 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:40:16,827 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:40:20,013 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:40:25,013 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:40:30,014 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:40:31,826 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:40:31,826 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:40:31,827 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:40:35,041 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:40:40,042 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:40:42,925 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:40:45,926 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:40:46,826 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:40:46,826 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:40:46,827 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:40:51,077 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:40:56,078 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:41:01,079 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:41:01,826 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:41:01,827 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:41:01,827 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:41:06,107 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:41:11,107 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:41:12,921 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:41:16,826 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:41:16,827 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:41:16,827 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:41:16,827 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:41:22,117 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:41:27,118 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:41:31,827 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:41:31,827 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:41:31,828 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:41:33,087 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:41:38,088 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:41:42,921 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:41:43,922 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:41:46,826 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:41:46,827 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:41:46,827 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:41:49,050 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:41:54,051 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:41:59,052 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:42:01,827 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:42:01,827 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:42:01,827 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:42:04,986 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:42:09,987 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:42:12,922 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:42:15,923 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:42:16,827 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:42:16,827 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:42:16,828 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:42:21,087 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:42:26,088 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:42:31,089 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:42:31,827 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:42:31,827 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:42:31,828 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:42:37,008 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:42:42,008 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:42:42,923 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:42:46,827 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:42:46,828 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:42:46,828 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:42:47,046 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:42:52,047 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:42:57,048 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:43:01,827 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:43:01,828 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:43:01,828 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:43:02,088 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:43:07,089 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:43:12,090 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:43:12,923 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:43:16,827 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:43:16,828 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:43:16,828 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:43:18,060 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:43:23,061 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:43:28,062 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:43:31,827 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:43:31,828 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:43:31,828 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:43:34,016 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:43:39,017 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:43:42,923 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:43:44,924 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:43:46,828 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:43:46,828 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:43:46,828 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:43:50,034 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:43:55,034 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:44:00,036 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:44:01,827 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:44:01,828 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:44:01,828 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:44:05,075 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:44:10,076 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:44:12,924 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:44:15,925 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:44:16,828 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:44:16,829 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:44:16,829 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:44:21,027 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:44:26,028 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:44:31,028 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:44:31,828 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:44:31,829 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:44:31,829 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:44:36,081 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:44:41,082 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:44:42,924 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:44:46,828 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:44:46,829 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:44:46,829 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:44:46,829 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:44:52,136 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:44:57,137 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:45:01,828 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:45:01,829 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:45:01,829 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:45:03,059 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:45:08,060 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:45:09,252 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 08:45:09,256 DEBUG SenderThread:3236 [sender.py:send():380] send: metric 2023-11-23 08:45:09,257 DEBUG SenderThread:3236 [sender.py:send():380] send: metric 2023-11-23 08:45:09,257 DEBUG SenderThread:3236 [sender.py:send():380] send: metric 2023-11-23 08:45:09,258 DEBUG SenderThread:3236 [sender.py:send():380] send: metric 2023-11-23 08:45:09,258 DEBUG SenderThread:3236 [sender.py:send():380] send: metric 2023-11-23 08:45:09,258 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 08:45:09,259 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 08:45:09,261 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 08:45:09,852 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 08:45:10,857 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:45:11,863 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:45:12,925 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:45:13,992 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:45:15,886 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:45:16,987 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:45:16,988 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:45:17,122 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:45:19,203 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:45:25,107 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:45:30,113 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:45:31,174 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/config.yaml 2023-11-23 08:45:32,331 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:45:35,697 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:45:40,698 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:45:41,039 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: stop_status 2023-11-23 08:45:41,040 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: stop_status 2023-11-23 08:45:42,235 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:45:42,925 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:45:43,242 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:45:45,926 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:45:50,927 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:45:55,928 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:46:33,887 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:46:33,887 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:46:33,889 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:46:33,889 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: internal_messages 2023-11-23 08:46:34,350 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:46:34,352 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:46:34,353 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:46:34,363 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:46:36,368 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:46:38,379 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:46:39,718 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:46:44,719 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:46:47,867 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:46:49,720 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:46:54,721 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:46:59,721 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:47:03,516 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:47:05,521 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:47:06,382 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:47:10,518 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:47:15,518 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:47:20,519 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:47:25,520 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:47:30,521 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:47:33,518 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:47:36,519 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:47:37,749 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:47:41,797 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:47:41,828 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:47:46,801 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:47:46,829 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:47:51,801 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:47:51,830 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:47:56,802 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:47:56,830 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:48:01,803 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:48:01,831 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:48:03,518 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:48:06,804 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:48:07,518 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:48:11,804 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:48:12,519 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:48:16,805 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:48:17,520 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:48:21,806 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:48:22,521 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:48:26,807 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:48:27,521 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:48:31,808 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:48:32,523 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:48:33,518 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:48:34,967 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:48:36,810 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:48:38,520 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:48:41,811 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:48:43,520 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:48:46,962 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:48:49,516 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:48:50,059 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:48:51,964 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:48:54,517 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:48:57,700 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:48:57,701 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:49:00,080 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:49:00,134 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:49:02,760 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:49:03,519 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:49:05,520 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:49:08,194 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:49:08,196 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:49:10,198 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:49:10,895 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:49:13,196 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:49:15,896 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:49:18,459 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:49:20,248 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:49:21,772 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:49:23,460 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:49:26,773 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:49:28,765 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:49:32,259 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:49:32,342 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:49:33,519 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:49:33,798 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:49:37,521 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:49:39,066 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:49:39,067 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:49:42,411 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:49:43,470 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:49:44,068 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:49:48,472 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:49:49,319 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:49:52,478 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:49:53,548 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:49:54,320 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:49:58,549 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:49:59,329 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:50:03,520 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:50:04,330 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:50:04,520 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:50:09,331 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:50:09,522 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:50:14,331 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:50:14,522 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:50:19,500 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:50:19,535 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:50:22,705 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:50:24,502 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:50:24,904 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:50:29,510 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:50:29,905 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:50:33,520 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:50:34,950 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:50:35,211 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:50:35,521 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:50:40,217 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:50:40,523 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:50:45,018 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:50:45,256 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:50:45,646 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:50:50,312 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:50:50,648 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:50:55,081 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:50:55,537 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:50:55,764 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:51:00,553 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:51:00,765 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:51:03,521 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:51:05,145 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:51:06,240 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:51:06,522 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:51:11,524 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:51:11,760 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:51:14,208 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:51:16,761 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:51:17,105 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:51:22,107 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:51:22,288 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:51:24,277 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:51:27,326 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:51:27,618 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:51:32,512 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:51:32,620 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:51:33,522 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:51:36,350 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:51:37,513 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:51:37,829 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:51:42,832 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:51:43,491 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:51:46,419 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:51:48,057 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:51:48,492 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:51:53,058 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:51:54,067 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:51:54,468 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:51:58,678 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:51:59,477 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:52:03,534 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:52:04,323 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:52:04,910 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:52:06,553 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:52:09,324 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:52:09,911 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:52:15,075 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:52:15,864 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:52:16,701 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:52:20,076 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:52:22,392 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:52:23,957 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 08:52:23,965 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 08:52:23,967 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 08:52:23,971 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 08:52:24,763 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 08:52:25,980 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:52:26,774 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:52:27,393 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:52:30,981 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:52:32,976 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:52:33,523 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:52:36,198 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:52:36,864 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:52:37,977 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:52:41,199 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:52:43,502 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:52:43,503 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:52:44,915 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:52:46,762 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:52:48,503 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:52:51,764 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:52:53,592 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:52:56,849 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:52:56,991 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:52:58,593 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:53:01,851 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:53:03,524 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:53:04,152 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:53:05,046 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:53:07,525 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:53:09,153 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:53:12,527 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:53:14,590 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:53:15,111 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:53:17,776 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:53:20,972 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:53:23,169 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:53:25,180 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:53:25,973 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:53:28,170 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:53:31,552 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:53:33,236 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:53:33,524 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:53:33,525 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:53:36,553 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:53:38,526 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:53:42,155 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:53:43,307 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:53:44,486 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:53:47,156 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:53:49,488 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:53:52,376 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:53:53,058 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:53:55,478 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:53:58,059 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:54:00,480 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:54:02,466 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:54:03,191 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:54:03,525 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:54:05,526 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:54:08,192 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:54:10,528 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:54:12,531 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:54:14,120 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:54:16,350 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:54:19,931 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:54:22,237 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:54:22,595 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:54:24,938 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:54:27,257 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:54:29,986 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:54:29,986 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:54:32,367 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:54:32,658 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:54:33,525 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:54:34,987 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:54:37,526 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:54:40,157 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:54:42,722 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:54:43,361 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:54:45,758 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:54:48,362 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:54:50,891 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:54:53,537 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:54:54,793 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:54:55,892 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:54:58,538 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:55:01,092 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:55:03,526 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:55:04,527 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:55:04,859 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:55:06,097 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:55:09,527 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:55:11,588 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:55:11,589 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:55:11,589 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:55:14,925 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:55:14,944 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:55:16,590 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:55:19,945 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:55:21,777 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:55:25,000 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:55:25,068 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:55:26,817 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:55:30,070 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:55:31,914 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:55:33,526 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:55:35,065 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:55:35,528 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:55:36,916 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:55:40,529 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:55:42,213 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:55:42,214 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:55:42,214 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:55:43,572 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 08:55:43,574 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 08:55:43,575 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 08:55:43,577 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 08:55:44,125 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:55:44,126 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 08:55:45,131 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:55:45,578 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:55:47,216 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:55:50,580 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:55:52,594 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:55:56,091 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:55:57,200 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:55:57,595 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:56:01,093 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:56:02,798 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:56:03,527 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:56:07,074 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:56:07,261 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:56:07,817 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:56:12,076 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:56:13,656 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:56:13,656 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:56:15,322 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:56:18,010 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:56:18,670 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:56:23,012 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:56:24,310 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:56:25,398 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:56:28,912 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:56:29,371 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:56:33,528 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:56:34,007 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:56:34,623 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:56:35,472 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:56:39,016 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:56:39,624 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:56:44,017 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:56:45,155 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:56:45,155 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:56:46,578 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:56:49,402 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:56:50,156 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:56:54,403 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:56:56,049 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:56:56,647 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:56:59,476 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:57:01,053 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:57:03,529 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:57:04,704 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:57:05,075 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:57:06,806 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:57:10,076 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:57:12,549 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:57:14,772 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:57:15,811 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:57:17,550 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:57:20,812 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:57:22,569 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:57:24,841 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:57:25,857 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:57:27,570 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:57:30,858 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:57:33,485 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:57:33,529 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:57:34,906 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:57:36,648 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:57:38,486 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:57:41,650 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:57:43,891 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:57:44,980 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:57:47,118 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:57:48,892 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:57:52,120 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:57:54,665 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:57:55,152 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:57:58,099 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:57:59,667 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:58:03,529 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:58:03,531 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:58:05,211 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:58:05,504 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:58:08,531 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:58:11,588 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:58:13,975 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:58:15,277 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:58:16,599 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:58:18,976 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:58:22,103 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:58:24,369 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:58:25,344 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:58:27,104 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:58:29,370 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:58:32,796 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:58:33,530 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:58:34,531 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:58:35,411 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:58:37,797 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:58:39,532 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:58:42,898 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:58:43,463 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:58:45,115 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:58:47,899 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:58:50,116 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:58:53,469 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:58:53,532 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:58:55,825 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:58:58,470 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:59:00,827 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:59:01,684 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 08:59:01,685 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 08:59:01,686 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 08:59:01,688 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 08:59:02,596 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:59:02,597 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 08:59:03,530 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:59:03,602 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:59:04,397 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:59:06,532 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:59:10,515 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:59:11,533 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:59:13,675 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:59:15,516 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:59:16,947 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:59:21,211 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:59:22,540 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:59:22,769 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:59:26,212 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:59:27,550 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:59:31,947 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:59:32,827 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:59:33,501 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:59:33,538 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 08:59:36,948 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:59:38,540 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:59:41,982 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:59:42,884 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:59:44,344 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:59:46,983 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:59:49,345 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:59:52,940 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 08:59:52,954 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:59:54,369 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 08:59:57,955 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 08:59:59,371 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:00:03,004 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:00:03,161 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:00:03,539 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:00:04,540 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:00:08,161 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:00:09,541 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:00:13,068 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:00:13,724 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:00:15,091 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:00:18,725 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:00:20,093 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:00:23,134 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:00:24,336 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:00:25,604 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:00:30,052 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:00:30,053 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:00:30,606 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:00:33,203 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:00:33,539 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:00:35,054 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:00:36,540 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:00:40,730 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:00:41,961 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:00:43,270 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:00:45,731 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:00:46,962 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:00:50,761 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:00:52,146 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:00:53,324 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:00:55,762 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:00:57,146 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:01:01,068 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:01:01,068 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:01:02,607 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:01:03,405 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:01:03,540 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:01:06,069 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:01:08,541 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:01:11,957 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:01:13,472 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:01:14,159 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:01:16,958 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:01:19,161 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:01:22,705 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:01:23,543 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:01:24,919 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:01:27,707 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:01:29,921 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:01:33,551 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:01:33,604 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:01:33,614 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:01:35,552 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:01:39,713 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:01:40,553 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:01:43,667 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:01:44,714 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:01:45,911 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:01:50,418 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:01:50,912 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:01:53,729 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:01:55,418 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:01:56,257 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:02:00,419 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:02:01,258 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:02:03,542 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:02:05,936 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:02:06,413 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:02:07,807 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:02:10,937 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:02:11,414 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:02:16,277 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:02:16,278 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:02:16,415 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:02:17,873 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:02:21,278 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:02:21,447 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:02:26,358 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 09:02:26,360 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 09:02:26,361 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 09:02:26,363 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 09:02:26,937 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 09:02:26,971 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:02:27,365 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:02:27,943 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:02:28,946 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:02:31,972 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:02:32,366 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:02:33,545 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:02:37,010 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:02:37,624 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:02:37,858 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:02:42,625 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:02:42,859 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:02:47,167 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:02:48,301 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:02:48,649 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:02:53,650 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:02:54,477 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:02:57,230 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:02:58,857 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:02:59,478 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:03:03,546 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:03:04,548 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:03:05,377 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:03:07,303 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:03:09,768 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:03:10,378 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:03:14,769 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:03:16,276 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:03:17,389 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:03:20,469 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:03:21,277 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:03:26,129 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:03:26,949 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:03:27,451 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:03:31,129 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:03:33,484 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:03:33,484 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:03:33,546 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:03:36,847 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:03:37,508 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:03:38,485 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:03:41,850 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:03:43,495 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:03:47,577 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:03:47,712 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:03:48,496 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:03:52,713 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:03:53,978 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:03:55,635 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:03:58,372 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:03:58,979 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:04:03,373 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:04:03,547 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:04:04,546 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:04:05,710 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:04:08,765 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:04:09,547 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:04:14,320 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:04:15,058 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:04:15,773 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:04:19,338 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:04:20,060 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:04:24,911 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:04:25,656 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:04:25,845 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:04:29,912 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:04:31,246 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:04:31,246 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:04:31,247 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:04:33,547 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:04:33,901 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:04:35,548 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:04:36,247 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:04:40,549 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:04:41,987 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:04:43,966 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:04:46,126 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:04:46,989 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:04:51,128 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:04:52,570 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:04:54,042 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:04:56,867 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:04:57,571 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:05:02,380 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:05:03,121 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:05:03,548 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:05:04,096 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:05:07,549 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:05:08,123 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:05:12,887 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:05:13,175 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:05:13,752 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:05:17,889 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:05:19,416 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:05:19,417 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:05:21,227 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:05:23,697 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:05:24,417 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:05:28,698 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:05:30,289 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:05:31,496 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:05:33,549 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:05:34,559 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:05:35,290 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:05:39,560 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:05:40,349 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 09:05:40,352 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 09:05:40,352 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 09:05:40,355 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 09:05:40,558 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 09:05:40,958 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:05:41,564 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:05:45,357 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:05:45,969 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:05:51,012 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:05:51,259 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:05:51,623 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:05:56,017 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:05:56,260 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:06:01,261 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:06:01,826 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:06:01,838 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:06:01,838 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:06:03,549 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:06:03,711 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:06:06,557 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:06:06,838 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:06:11,559 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:06:12,417 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:06:13,775 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:06:16,635 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:06:17,418 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:06:22,178 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:06:22,910 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:06:23,836 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:06:27,178 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:06:27,911 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:06:32,930 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:06:33,479 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:06:33,549 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:06:33,991 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:06:38,481 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:06:38,551 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:06:44,033 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:06:44,045 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:06:44,196 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:06:49,197 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:06:49,629 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:06:49,630 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:06:52,123 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:06:54,631 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:06:54,833 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:06:59,834 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:07:00,012 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:07:02,204 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:07:03,550 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:07:05,076 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:07:05,551 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:07:11,007 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:07:11,209 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:07:12,269 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:07:16,008 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:07:16,209 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:07:21,330 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:07:21,526 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:07:21,773 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:07:26,775 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:07:28,023 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:07:31,395 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:07:32,256 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:07:33,024 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:07:33,551 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:07:37,552 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:07:38,686 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:07:39,446 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:07:42,834 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:07:43,687 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:07:47,836 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:07:49,449 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:07:49,450 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:07:49,520 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:07:53,656 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:07:54,450 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:07:59,514 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:07:59,589 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:08:00,327 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:08:03,552 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:08:04,553 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:08:06,697 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:08:09,651 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:08:09,949 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:08:11,698 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:08:14,950 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:08:17,196 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:08:19,774 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:08:20,412 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:08:22,197 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:08:25,414 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:08:27,684 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:08:27,832 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:08:30,960 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:08:32,685 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:08:33,553 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:08:36,479 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:08:37,904 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:08:38,216 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:08:41,480 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:08:44,732 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:08:46,976 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:08:47,980 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:08:49,733 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:08:51,977 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:08:55,625 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:08:55,861 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 09:08:55,863 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 09:08:55,864 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 09:08:55,868 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 09:08:56,033 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 09:08:57,039 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:08:57,870 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:08:58,045 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:09:00,626 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:09:02,871 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:09:03,554 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:09:06,259 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:09:08,111 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:09:08,437 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:09:11,260 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:09:13,438 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:09:16,158 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:09:16,737 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:09:19,011 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:09:23,305 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:09:24,012 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:09:26,219 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:09:28,306 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:09:29,575 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:09:33,554 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:09:33,905 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:09:33,906 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:09:35,171 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:09:36,284 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:09:38,906 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:09:40,172 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:09:44,734 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:09:45,873 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:09:46,347 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:09:49,735 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:09:50,875 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:09:55,508 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:09:56,410 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:09:56,867 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:10:00,509 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:10:01,869 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:10:03,555 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:10:05,566 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:10:05,912 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:10:07,561 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:10:11,606 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:10:12,562 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:10:13,618 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:10:16,607 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:10:17,761 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:10:21,988 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:10:21,989 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:10:23,245 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:10:23,677 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:10:26,989 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:10:28,245 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:10:32,678 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:10:33,555 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:10:33,556 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:10:33,760 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:10:37,679 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:10:38,557 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:10:43,241 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:10:43,826 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:10:44,468 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:10:48,243 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:10:49,470 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:10:51,878 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:10:53,957 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:10:55,113 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:10:59,447 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:11:00,114 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:11:01,947 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:11:03,566 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:11:04,449 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:11:05,567 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:11:10,439 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:11:10,568 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:11:12,017 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:11:15,440 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:11:15,854 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:11:21,228 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:11:21,368 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:11:22,079 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:11:26,229 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:11:26,369 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:11:31,550 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:11:31,738 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:11:32,138 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:11:33,556 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:11:36,552 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:11:37,558 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:11:42,197 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:11:42,285 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:11:42,616 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:11:47,617 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:11:48,176 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:11:52,266 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:11:53,177 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:11:53,348 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:11:58,349 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:11:58,480 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:12:00,312 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:12:03,481 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:12:03,557 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:12:03,557 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:12:08,404 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 09:12:08,405 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 09:12:08,406 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 09:12:08,408 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 09:12:09,206 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:12:09,206 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:12:09,379 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:12:09,380 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 09:12:09,410 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:12:10,386 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:12:14,207 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:12:14,411 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:12:19,228 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:12:19,659 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:12:20,447 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:12:24,661 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:12:26,012 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:12:28,502 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:12:30,238 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:12:31,013 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:12:33,558 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:12:35,559 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:12:36,691 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:12:36,692 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:12:38,569 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:12:40,905 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:12:41,755 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:12:45,907 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:12:47,576 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:12:48,637 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:12:50,939 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:12:52,577 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:12:55,940 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:12:58,290 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:12:58,711 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:13:01,675 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:13:03,292 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:13:03,558 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:13:07,454 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:13:07,767 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:13:08,737 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:13:12,455 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:13:14,691 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:13:17,832 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:13:17,880 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:13:19,692 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:13:22,881 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:13:24,710 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:13:27,895 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:13:28,068 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:13:29,765 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:13:33,070 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:13:33,561 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:13:35,692 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:13:37,967 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:13:38,935 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:13:40,703 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:13:43,937 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:13:46,024 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:13:46,199 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:13:49,420 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:13:51,200 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:13:54,422 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:13:56,096 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:13:56,985 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:14:00,161 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:14:03,491 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:14:03,561 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:14:05,791 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:14:06,168 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:14:08,492 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:14:10,792 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:14:13,673 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:14:16,029 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:14:16,236 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:14:18,674 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:14:21,030 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:14:23,759 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:14:23,760 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:14:26,300 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:14:26,993 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:14:28,761 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:14:31,994 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:14:33,564 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:14:34,242 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:14:36,373 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:14:37,487 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:14:39,243 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:14:42,489 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:14:44,635 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:14:46,434 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:14:47,877 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:14:49,636 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:14:52,878 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:14:54,490 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:14:55,077 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:14:58,416 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:15:01,686 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:15:03,563 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:15:03,564 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:15:04,546 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:15:06,687 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:15:08,565 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:15:12,331 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:15:13,589 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:15:14,614 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:15:17,332 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:15:18,590 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:15:22,356 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:15:22,608 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 09:15:22,610 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 09:15:22,611 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 09:15:22,616 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 09:15:22,671 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 09:15:23,618 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:15:23,676 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:15:24,683 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:15:27,357 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:15:28,619 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:15:32,943 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:15:33,564 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:15:34,564 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:15:34,741 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:15:37,944 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:15:39,566 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:15:42,801 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:15:43,302 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:15:45,481 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:15:49,761 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:15:50,482 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:15:52,869 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:15:54,762 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:15:55,931 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:16:00,160 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:16:01,438 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:16:02,925 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:16:03,564 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:16:05,161 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:16:06,566 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:16:11,124 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:16:11,125 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:16:12,005 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:16:12,371 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:16:16,125 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:16:17,371 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:16:21,153 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:16:22,373 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:16:26,094 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:16:26,897 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:16:26,897 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:16:28,000 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:16:32,293 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:16:33,001 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:16:33,565 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:16:34,142 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:16:37,294 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:16:38,568 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:16:42,821 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:16:42,821 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:16:44,088 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:16:44,207 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:16:47,822 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:16:49,106 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:16:52,896 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:16:54,267 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:16:54,627 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:16:58,502 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:16:59,628 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:17:03,565 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:17:04,126 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:17:04,343 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:17:05,566 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:17:10,567 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:17:10,718 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:17:10,718 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:17:14,410 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:17:15,719 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:17:15,886 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:17:20,887 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:17:21,311 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:17:22,475 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:17:26,312 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:17:26,475 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:17:31,824 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:17:32,008 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:17:32,539 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:17:33,566 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:17:36,826 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:17:37,567 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:17:42,372 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:17:42,569 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:17:44,616 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:17:47,373 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:17:47,803 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:17:52,805 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:17:53,127 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:17:53,128 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:17:53,128 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:17:56,690 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:17:58,129 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:17:58,698 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:18:03,567 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:18:04,202 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:18:04,569 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:18:06,748 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:18:09,266 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:18:10,426 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:18:15,260 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:18:15,428 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:18:18,846 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:18:20,328 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:18:20,524 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:18:25,526 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:18:25,531 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:18:25,531 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:18:28,963 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:18:30,532 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:18:30,658 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:18:33,567 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:18:35,783 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:18:36,568 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:18:39,034 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:18:40,784 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:18:42,010 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:18:46,699 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:18:46,845 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 09:18:46,846 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 09:18:46,847 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 09:18:46,849 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 09:18:47,084 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 09:18:47,851 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:18:48,089 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:18:49,093 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:18:51,700 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:18:52,852 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:18:56,807 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:18:56,807 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:18:58,086 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:18:58,151 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:19:01,808 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:19:03,087 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:19:03,568 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:19:07,102 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:19:08,225 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:19:08,661 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:19:12,104 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:19:13,662 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:19:18,011 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:19:19,486 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:19:20,306 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:19:23,115 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:19:24,487 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:19:28,954 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:19:30,125 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:19:30,378 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:19:33,568 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:19:33,980 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:19:35,569 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:19:39,104 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:19:40,466 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:19:41,449 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:19:44,105 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:19:46,451 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:19:49,163 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:19:50,528 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:19:52,445 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:19:54,164 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:19:57,447 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:19:59,241 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:20:00,597 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:20:02,567 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:20:03,569 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:20:04,243 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:20:07,570 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:20:10,127 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:20:10,664 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:20:13,320 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:20:15,128 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:20:18,321 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:20:20,610 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:20:20,741 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:20:23,906 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:20:25,612 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:20:28,908 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:20:31,018 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:20:32,809 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:20:33,569 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:20:34,570 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:20:36,019 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:20:39,571 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:20:41,854 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:20:42,924 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:20:45,129 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:20:46,856 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:20:50,130 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:20:51,903 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:20:52,991 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:20:55,326 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:20:56,904 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:21:01,086 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:21:02,850 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:21:03,056 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:21:03,581 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:21:06,582 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:21:08,775 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:21:12,189 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:21:13,120 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:21:13,776 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:21:17,190 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:21:19,250 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:21:22,613 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:21:23,180 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:21:24,251 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:21:27,614 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:21:29,325 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:21:32,786 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:21:33,245 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:21:33,570 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:21:34,328 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:21:38,572 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:21:39,527 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:21:39,528 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:21:39,528 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:21:43,307 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:21:43,865 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:21:44,529 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:21:48,867 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:21:50,450 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:21:52,368 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:21:54,836 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:21:55,451 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:21:59,837 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:22:00,764 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:22:02,437 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:22:03,572 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:22:05,573 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:22:05,765 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:22:10,574 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:22:11,127 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:22:11,559 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 09:22:11,560 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 09:22:11,560 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 09:22:11,562 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 09:22:12,506 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:22:12,507 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 09:22:14,515 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:22:16,128 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:22:16,564 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:22:21,310 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:22:21,565 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:22:22,575 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:22:26,311 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:22:26,600 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:22:31,453 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:22:31,602 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:22:32,649 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:22:33,572 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:22:36,454 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:22:37,573 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:22:42,358 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:22:42,358 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:22:42,736 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:22:42,778 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:22:47,359 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:22:47,779 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:22:52,672 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:22:52,811 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:22:52,854 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:22:57,673 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:22:57,855 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:23:02,879 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:23:02,907 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:23:03,573 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:23:03,618 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:23:08,574 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:23:10,195 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:23:12,935 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:23:14,540 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:23:15,197 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:23:19,542 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:23:21,067 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:23:23,000 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:23:25,310 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:23:26,076 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:23:30,312 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:23:32,022 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:23:33,075 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:23:33,573 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:23:35,574 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:23:37,023 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:23:40,575 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:23:42,359 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:23:43,141 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:23:46,264 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:23:47,996 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:23:51,266 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:23:53,206 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:23:53,229 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:23:56,579 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:23:58,231 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:24:02,060 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:24:03,270 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:24:03,573 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:24:03,785 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:24:07,575 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:24:09,431 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:24:12,611 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:24:13,335 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:24:14,432 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:24:17,613 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:24:20,021 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:24:21,386 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:24:23,242 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:24:25,022 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:24:28,243 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:24:30,962 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:24:33,473 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:24:33,476 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:24:33,574 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:24:35,963 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:24:38,576 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:24:41,359 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:24:42,537 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:24:43,714 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:24:46,603 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:24:48,716 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:24:52,364 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:24:52,598 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:24:53,741 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:24:57,366 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:24:58,743 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:25:00,655 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:25:03,105 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:25:03,574 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:25:04,575 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:25:08,755 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:25:09,608 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:25:10,724 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:25:13,756 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:25:14,930 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:25:19,367 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:25:20,651 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:25:20,793 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:25:24,368 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:25:25,652 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:25:29,462 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:25:29,463 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:25:29,733 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 09:25:29,734 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 09:25:29,734 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 09:25:29,737 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 09:25:29,851 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 09:25:30,738 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:25:30,855 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:25:32,864 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:25:33,575 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:25:34,463 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:25:36,576 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:25:40,105 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:25:40,918 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:25:42,421 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:25:45,106 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:25:47,422 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:25:50,969 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:25:51,015 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:25:53,181 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:25:57,449 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:25:58,182 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:26:01,075 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:26:02,450 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:26:03,576 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:26:03,577 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:26:08,302 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:26:08,578 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:26:11,138 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:26:13,304 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:26:13,697 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:26:18,675 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:26:18,699 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:26:21,197 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:26:23,676 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:26:23,949 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:26:28,950 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:26:28,955 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:26:28,955 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:26:31,270 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:26:33,576 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:26:33,956 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:26:34,577 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:26:39,229 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:26:40,553 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:26:41,335 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:26:44,277 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:26:45,554 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:26:49,332 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:26:50,776 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:26:51,405 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:26:54,333 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:26:55,777 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:27:00,048 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:27:00,049 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:27:01,310 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:27:01,471 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:27:03,577 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:27:05,049 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:27:06,578 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:27:10,361 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:27:10,361 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:27:11,527 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:27:11,979 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:27:15,698 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:27:16,981 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:27:20,836 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:27:21,588 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:27:22,221 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:27:25,845 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:27:27,223 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:27:31,649 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:27:31,753 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:27:32,942 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:27:33,578 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:27:37,688 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:27:38,579 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:27:41,713 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:27:42,689 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:27:44,037 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:27:48,469 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:27:49,744 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:27:50,772 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:27:53,470 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:27:54,745 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:27:58,828 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:27:59,044 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:28:00,273 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:28:03,578 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:28:04,045 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:28:05,580 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:28:08,900 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:28:09,810 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:28:11,134 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:28:14,812 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:28:16,136 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:28:18,962 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:28:20,542 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:28:21,738 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:28:26,480 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:28:26,740 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:28:29,024 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:28:31,483 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:28:32,689 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:28:33,579 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:28:37,146 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:28:38,325 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:28:39,099 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:28:42,147 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:28:43,326 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:28:47,050 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 09:28:47,051 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 09:28:47,052 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 09:28:47,055 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 09:28:47,157 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 09:28:47,841 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:28:49,056 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:28:49,170 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:28:52,842 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:28:54,058 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:28:57,228 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:28:58,404 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:28:59,594 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:29:03,406 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:29:03,580 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:29:05,582 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:29:07,288 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:29:09,241 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:29:10,593 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:29:15,308 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:29:15,595 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:29:17,386 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:29:20,309 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:29:21,592 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:29:25,688 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:29:26,593 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:29:29,464 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:29:30,689 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:29:31,843 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:29:33,582 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:29:36,234 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:29:37,422 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:29:37,524 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:29:41,235 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:29:42,423 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:29:46,889 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:29:47,588 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:29:48,096 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:29:51,890 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:29:53,097 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:29:57,296 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:29:57,666 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:29:58,820 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:30:02,297 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:30:03,583 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:30:04,585 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:30:07,736 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:30:08,075 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:30:10,303 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:30:13,966 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:30:13,967 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:30:16,195 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:30:17,797 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:30:18,967 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:30:21,195 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:30:24,786 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:30:27,125 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:30:27,858 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:30:29,787 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:30:32,126 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:30:33,596 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:30:35,688 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:30:36,923 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:30:37,915 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:30:40,689 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:30:42,917 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:30:45,761 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:30:45,762 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:30:46,984 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:30:48,165 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:30:50,770 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:30:53,166 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:30:56,482 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:30:57,050 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:30:59,051 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:31:01,708 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:31:03,584 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:31:04,585 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:31:08,023 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:31:10,372 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:31:11,129 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:31:13,024 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:31:15,372 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:31:18,155 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:31:20,437 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:31:21,277 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:31:23,156 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:31:25,438 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:31:28,885 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:31:28,886 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:31:30,439 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:31:33,584 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:31:33,675 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:31:33,887 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:31:35,585 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:31:38,887 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:31:40,587 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:31:43,743 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:31:44,056 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:31:46,159 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:31:49,058 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:31:51,160 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:31:53,816 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:31:54,744 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:31:56,944 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:32:00,153 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:32:02,242 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:32:03,585 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:32:03,882 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:32:05,154 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:32:07,586 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:32:10,684 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:32:10,957 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 09:32:10,958 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 09:32:10,959 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 09:32:10,961 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 09:32:11,941 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:32:11,942 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 09:32:12,945 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:32:12,963 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:32:15,686 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:32:17,964 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:32:21,192 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:32:23,478 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:32:24,016 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:32:26,194 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:32:28,480 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:32:31,895 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:32:33,082 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:32:33,585 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:32:33,586 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:32:36,896 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:32:38,588 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:32:42,850 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:32:43,156 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:32:44,207 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:32:47,852 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:32:49,209 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:32:51,205 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:32:53,669 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:32:54,914 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:32:59,163 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:32:59,163 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:32:59,915 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:33:01,277 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:33:03,586 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:33:04,164 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:33:05,587 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:33:10,106 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:33:11,338 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:33:11,502 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:33:15,107 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:33:16,503 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:33:21,088 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:33:21,401 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:33:22,342 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:33:26,089 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:33:27,343 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:33:31,430 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:33:31,473 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:33:32,593 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:33:33,586 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:33:36,461 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:33:38,588 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:33:41,537 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:33:42,260 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:33:44,503 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:33:47,261 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:33:49,504 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:33:51,598 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:33:52,826 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:33:55,226 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:33:58,652 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:34:00,828 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:34:01,674 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:34:03,590 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:34:03,653 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:34:06,591 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:34:09,139 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:34:11,739 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:34:12,441 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:34:14,140 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:34:17,443 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:34:20,117 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:34:21,808 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:34:23,376 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:34:25,118 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:34:28,378 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:34:30,431 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:34:31,869 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:34:33,587 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:34:33,588 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:34:35,876 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:34:38,590 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:34:41,727 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:34:41,939 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:34:43,875 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:34:46,728 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:34:48,876 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:34:52,418 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:34:53,884 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:34:57,419 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:34:58,885 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:35:03,235 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:35:03,588 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:35:04,055 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:35:04,589 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:35:08,236 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:35:09,590 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:35:13,908 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:35:14,158 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:35:15,459 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:35:18,909 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:35:20,461 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:35:24,229 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:35:24,768 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:35:26,023 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:35:30,642 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:35:30,643 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:35:30,643 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:35:31,025 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:35:33,283 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:35:33,589 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:35:35,644 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:35:36,590 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:35:40,644 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:35:41,591 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:35:45,646 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:35:46,591 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:35:51,510 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:35:51,593 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:35:54,018 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 09:35:54,020 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 09:35:54,021 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 09:35:54,024 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 09:35:54,427 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 09:35:55,432 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:35:56,577 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:35:57,026 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:36:02,027 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:36:02,527 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:36:03,590 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:36:05,496 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:36:07,528 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:36:07,984 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:36:12,967 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:36:12,985 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:36:15,584 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:36:17,968 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:36:18,292 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:36:23,294 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:36:23,965 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:36:25,648 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:36:28,966 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:36:29,063 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:36:33,590 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:36:34,283 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:36:34,284 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:36:34,591 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:36:35,714 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:36:39,284 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:36:39,592 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:36:44,922 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:36:45,088 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:36:45,776 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:36:50,090 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:36:51,271 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:36:53,833 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:36:55,577 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:36:56,272 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:37:00,579 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:37:02,123 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:37:02,124 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:37:03,591 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:37:03,907 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:37:05,592 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:37:07,125 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:37:10,593 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:37:12,493 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:37:13,968 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:37:15,665 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:37:17,494 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:37:20,667 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:37:22,845 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:37:24,046 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:37:26,120 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:37:27,845 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:37:31,122 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:37:33,124 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:37:33,591 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:37:34,110 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:37:36,592 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:37:40,044 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:37:42,280 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:37:44,178 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:37:45,045 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:37:47,281 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:37:50,621 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:37:50,622 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:37:52,234 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:37:52,785 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:37:55,623 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:37:57,786 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:38:01,137 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:38:01,162 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:38:02,295 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:38:03,592 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:38:03,594 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:38:06,508 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:38:08,594 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:38:11,524 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:38:12,359 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:38:13,801 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:38:16,525 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:38:18,803 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:38:22,200 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:38:22,427 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:38:24,366 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:38:27,202 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:38:29,368 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:38:31,490 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:38:33,080 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:38:33,593 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:38:34,593 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:38:38,507 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:38:39,595 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:38:41,561 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:38:43,508 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:38:44,682 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:38:49,336 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:38:49,336 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:38:50,630 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:38:51,624 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:38:54,337 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:38:55,631 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:39:00,198 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:39:01,592 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:39:01,685 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:39:03,593 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:39:05,199 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:39:06,594 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:39:09,227 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 09:39:09,228 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 09:39:09,229 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 09:39:09,232 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 09:39:09,739 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 09:39:11,055 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:39:11,749 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:39:12,234 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:39:17,237 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:39:17,435 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:39:17,436 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:39:19,813 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:39:22,437 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:39:22,792 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:39:27,793 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:39:28,100 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:39:29,883 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:39:33,101 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:39:33,353 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:39:33,594 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:39:38,284 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:39:38,419 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:39:39,949 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:39:43,285 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:39:43,420 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:39:48,436 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:39:49,060 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:39:50,020 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:39:53,669 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:39:54,061 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:39:59,120 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:39:59,526 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:40:00,085 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:40:03,604 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:40:04,121 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:40:04,605 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:40:09,644 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:40:09,909 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:40:10,165 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:40:14,645 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:40:14,910 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:40:19,961 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:40:20,113 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:40:20,228 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:40:25,114 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:40:26,386 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:40:30,287 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:40:30,611 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:40:31,388 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:40:33,594 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:40:36,595 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:40:37,281 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:40:37,282 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:40:40,387 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:40:42,009 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:40:42,282 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:40:47,011 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:40:47,300 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:40:50,467 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:40:52,301 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:40:52,672 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:40:57,674 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:40:58,075 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:41:00,535 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:41:03,076 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:41:03,231 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:41:03,595 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:41:08,471 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:41:08,720 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:41:10,603 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:41:13,471 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:41:13,721 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:41:18,883 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:41:19,290 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:41:20,667 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:41:23,884 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:41:24,291 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:41:29,002 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:41:29,470 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:41:29,731 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:41:33,596 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:41:34,003 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:41:34,597 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:41:39,794 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:41:39,854 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:41:40,360 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:41:45,361 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:41:45,954 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:41:47,847 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:41:50,955 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:41:51,181 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:41:56,183 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:41:56,416 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:41:57,908 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:42:01,417 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:42:01,786 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:42:03,597 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:42:06,974 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:42:07,114 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:42:07,977 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:42:11,975 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:42:12,115 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:42:17,382 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:42:17,685 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:42:18,043 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:42:22,384 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:42:22,687 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:42:25,384 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 09:42:25,386 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 09:42:25,387 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 09:42:25,389 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 09:42:26,092 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 09:42:27,097 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:42:28,101 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:42:28,195 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:42:28,391 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:42:33,393 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:42:33,598 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:42:33,686 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:42:33,687 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:42:33,687 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:42:36,157 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:42:38,688 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:42:39,163 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:42:44,164 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:42:44,401 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:42:46,228 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:42:49,402 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:42:49,628 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:42:54,741 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:42:54,903 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:42:56,283 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:42:59,742 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:42:59,904 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:43:03,598 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:43:05,182 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:43:05,687 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:43:06,341 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:43:10,183 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:43:10,715 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:43:15,827 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:43:16,405 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:43:16,507 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:43:20,828 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:43:21,917 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:43:21,917 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:43:24,455 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:43:26,130 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:43:26,918 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:43:31,131 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:43:32,363 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:43:33,599 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:43:34,526 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:43:36,600 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:43:37,364 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:43:41,601 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:43:42,920 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:43:44,593 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:43:47,158 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:43:47,921 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:43:52,776 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:43:53,499 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:43:53,499 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:43:54,664 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:43:57,777 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:43:58,499 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:44:03,072 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:44:03,599 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:44:03,688 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:44:04,725 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:44:08,601 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:44:10,203 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:44:13,974 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:44:14,789 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:44:15,203 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:44:18,976 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:44:20,527 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:44:24,720 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:44:24,865 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:44:25,528 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:44:29,721 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:44:30,913 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:44:32,919 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:44:33,600 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:44:35,601 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:44:35,914 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:44:40,602 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:44:40,991 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:44:41,979 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:44:46,007 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:44:46,194 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:44:51,662 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:44:51,663 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:44:52,012 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:44:52,042 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:44:56,663 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:44:57,013 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:45:02,101 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:45:02,258 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:45:02,532 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:45:03,601 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:45:07,603 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:45:08,768 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:45:12,187 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:45:13,020 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:45:13,772 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:45:18,022 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:45:19,462 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:45:20,242 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:45:23,589 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:45:24,467 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:45:28,590 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:45:30,213 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:45:30,318 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:45:33,621 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:45:33,623 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:45:35,214 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:45:38,623 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:45:39,290 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 09:45:39,291 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 09:45:39,292 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 09:45:39,294 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 09:45:39,390 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 09:45:40,396 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:45:41,086 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:45:41,399 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:45:44,297 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:45:46,088 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:45:49,298 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:45:50,460 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:45:51,679 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:45:54,427 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:45:57,896 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:46:00,115 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:46:00,520 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:46:02,897 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:46:03,621 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:46:05,622 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:46:07,898 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:46:10,623 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:46:12,904 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:46:14,603 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:46:16,060 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:46:17,905 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:46:21,062 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:46:23,780 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:46:23,781 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:46:24,736 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:46:27,057 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:46:28,781 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:46:32,058 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:46:33,621 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:46:34,534 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:46:34,809 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:46:37,622 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:46:41,046 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:46:43,278 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:46:44,876 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:46:46,047 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:46:48,279 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:46:51,541 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:46:51,541 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:46:52,941 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:46:53,804 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:46:56,542 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:46:58,806 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:47:02,105 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:47:03,008 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:47:03,622 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:47:04,622 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:47:07,106 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:47:09,624 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:47:12,606 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:47:13,077 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:47:14,788 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:47:17,607 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:47:19,790 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:47:22,377 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:47:23,541 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:47:24,830 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:47:29,218 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:47:29,832 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:47:32,528 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:47:33,622 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:47:34,219 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:47:35,623 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:47:39,499 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:47:39,499 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:47:40,580 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:47:40,684 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:47:44,500 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:47:45,685 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:47:49,990 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:47:50,649 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:47:51,220 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:47:54,991 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:47:56,222 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:48:00,681 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:48:00,714 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:48:01,885 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:48:03,623 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:48:05,682 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:48:07,624 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:48:10,783 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:48:11,280 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:48:13,622 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:48:17,127 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:48:19,301 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:48:20,838 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:48:22,128 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:48:24,302 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:48:27,622 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:48:29,982 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:48:30,908 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:48:32,622 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:48:33,623 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:48:35,625 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:48:38,079 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:48:38,079 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:48:38,960 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:48:41,457 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:48:43,109 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:48:46,459 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:48:49,033 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:48:49,064 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:48:51,652 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:48:54,034 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:48:56,654 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:48:57,366 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 09:48:57,368 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 09:48:57,368 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 09:48:57,371 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 09:48:58,129 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 09:48:59,060 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:48:59,135 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:49:00,139 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:49:02,373 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:49:03,624 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:49:04,062 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:49:08,217 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:49:09,079 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:49:09,205 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:49:13,218 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:49:15,507 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:49:18,271 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:49:18,748 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:49:20,509 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:49:23,749 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:49:26,137 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:49:28,335 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:49:29,208 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:49:31,138 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:49:33,626 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:49:34,628 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:49:36,261 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:49:36,392 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:49:40,459 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:49:41,270 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:49:46,419 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:49:46,456 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:49:46,839 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:49:51,420 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:49:51,840 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:49:56,517 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:49:57,000 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:49:57,195 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:50:02,196 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:50:03,539 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:50:03,650 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:50:06,581 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:50:07,916 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:50:08,541 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:50:12,917 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:50:14,093 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:50:16,647 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:50:18,317 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:50:19,094 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:50:23,320 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:50:24,964 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:50:24,964 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:50:26,710 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:50:29,216 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:50:29,964 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:50:33,627 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:50:34,764 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:50:34,783 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:50:35,425 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:50:39,783 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:50:40,438 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:50:44,835 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:50:44,870 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:50:45,620 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:50:49,871 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:50:50,640 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:50:54,913 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:50:54,914 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:50:56,553 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:50:56,554 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:50:59,915 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:51:02,496 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:51:03,628 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:51:04,995 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:51:05,676 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:51:07,497 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:51:10,677 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:51:12,498 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:51:15,678 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:51:17,499 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:51:20,679 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:51:22,499 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:51:25,680 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:51:27,500 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:51:30,680 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:51:33,383 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:51:33,629 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:51:36,300 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:51:37,168 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:51:38,384 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:51:41,301 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:51:43,413 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:51:46,524 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:51:47,240 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:51:48,414 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:51:51,524 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:51:54,280 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:51:54,281 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:51:56,621 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:51:57,309 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:51:59,300 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:52:01,622 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:52:03,630 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:52:04,338 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:52:07,377 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:52:07,505 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:52:09,339 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:52:12,507 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:52:14,626 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:52:17,434 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:52:17,947 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:52:19,627 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:52:22,949 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:52:25,401 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:52:26,496 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:52:28,742 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:52:30,402 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:52:33,631 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:52:34,124 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:52:34,125 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 09:52:34,126 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 09:52:34,126 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 09:52:34,129 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 09:52:34,545 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:52:34,546 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 09:52:35,920 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:52:36,554 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:52:39,131 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:52:40,922 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:52:44,605 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:52:44,722 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:52:46,601 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:52:49,723 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:52:51,884 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:52:54,672 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:52:55,118 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:52:56,885 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:53:00,119 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:53:02,527 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:53:03,631 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:53:04,746 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:53:05,632 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:53:07,528 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:53:10,634 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:53:13,292 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:53:14,810 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:53:16,444 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:53:18,293 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:53:21,445 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:53:22,868 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:53:23,401 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:53:26,735 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:53:28,515 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:53:32,454 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:53:32,935 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:53:33,632 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:53:34,103 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:53:37,633 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:53:40,486 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:53:40,487 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:53:42,977 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:53:42,995 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:53:45,488 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:53:47,978 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:53:50,663 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:53:53,106 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:53:53,956 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:53:55,665 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:53:58,957 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:54:01,474 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:54:03,163 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:54:03,632 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:54:04,633 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:54:06,475 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:54:09,634 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:54:12,088 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:54:13,230 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:54:15,443 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:54:17,106 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:54:20,445 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:54:22,996 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:54:23,294 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:54:26,336 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:54:27,997 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:54:31,864 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:54:33,358 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:54:33,633 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:54:33,641 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:54:37,635 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:54:39,499 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:54:42,734 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:54:43,583 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:54:44,500 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:54:47,735 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:54:50,079 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:54:51,631 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:54:53,337 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:54:55,079 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:54:58,339 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:55:00,562 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:55:00,694 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:55:03,633 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:55:03,634 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:55:05,563 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:55:08,635 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:55:10,756 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:55:11,088 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:55:14,373 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:55:17,441 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:55:19,594 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:55:20,826 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:55:22,442 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:55:24,595 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:55:28,185 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:55:28,185 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:55:28,899 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:55:30,396 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:55:33,186 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:55:33,634 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:55:35,635 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:55:38,782 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:55:38,969 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:55:40,937 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:55:43,783 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:55:45,938 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:55:47,514 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 09:55:47,516 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 09:55:47,516 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 09:55:47,519 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 09:55:48,032 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:55:48,033 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 09:55:49,041 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:55:49,284 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:55:51,521 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:55:54,286 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:55:56,522 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:55:59,121 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:55:59,937 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:56:02,174 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:56:03,634 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:56:05,709 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:56:07,858 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:56:09,192 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:56:10,710 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:56:12,859 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:56:15,994 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:56:17,259 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:56:18,166 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:56:20,995 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:56:23,167 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:56:26,937 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:56:27,330 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:56:28,206 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:56:31,956 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:56:33,208 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:56:33,636 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:56:37,400 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:56:37,510 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:56:38,778 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:56:43,780 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:56:43,993 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:56:43,994 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:56:47,459 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:56:48,994 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:56:49,147 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:56:54,149 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:56:54,298 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:56:55,595 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:56:59,299 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:56:59,581 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:57:03,636 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:57:04,729 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:57:04,988 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:57:05,665 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:57:09,730 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:57:09,989 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:57:15,703 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:57:15,735 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:57:15,959 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:57:20,704 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:57:20,960 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:57:25,800 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:57:25,937 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:57:25,937 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:57:26,522 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:57:31,524 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:57:31,707 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:57:33,637 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:57:34,880 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:57:36,638 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:57:36,708 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:57:41,639 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:57:42,127 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:57:42,127 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:57:42,937 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:57:47,128 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:57:47,375 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:57:52,525 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:57:52,740 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:57:53,008 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:57:57,526 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:57:57,740 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:58:02,985 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:58:03,084 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:58:03,266 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:58:03,637 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:58:07,988 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:58:08,639 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:58:13,127 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:58:13,159 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:58:14,203 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:58:19,234 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:58:19,380 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:58:21,213 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:58:24,382 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:58:24,558 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:58:29,559 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:58:29,865 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:58:29,866 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:58:31,277 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:58:33,641 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:58:34,641 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:58:34,867 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:58:39,643 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:58:40,739 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:58:41,345 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:58:44,914 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:58:45,740 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:58:50,498 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:58:51,259 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:58:51,415 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:58:55,499 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:58:56,261 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:58:59,094 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 09:58:59,096 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 09:58:59,096 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 09:58:59,099 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 09:58:59,470 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 09:59:00,475 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:59:01,100 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:59:01,480 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:59:01,917 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:59:03,638 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:59:06,639 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:59:07,679 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:59:09,536 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:59:11,877 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:59:12,680 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:59:16,878 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:59:18,353 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:59:19,607 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:59:22,468 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:59:23,354 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:59:27,469 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:59:28,717 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:59:29,675 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:59:33,010 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:59:33,649 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 09:59:33,718 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:59:38,502 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:59:39,249 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:59:39,745 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:59:43,503 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:59:44,251 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:59:49,365 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:59:49,805 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 09:59:50,211 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:59:54,366 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:59:56,337 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 09:59:59,547 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 09:59:59,871 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:00:01,339 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:00:03,639 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:00:04,640 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:00:06,870 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:00:09,936 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:00:10,279 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:00:11,871 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:00:15,280 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:00:17,575 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:00:17,991 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:00:20,817 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:00:22,576 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:00:25,819 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:00:27,046 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:00:27,717 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:00:31,280 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:00:33,037 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:00:33,640 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:00:36,641 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:00:39,191 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:00:41,141 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:00:41,654 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:00:44,193 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:00:46,655 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:00:49,694 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:00:51,209 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:00:51,832 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:00:54,695 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:00:56,833 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:01:00,257 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:01:00,258 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:01:01,268 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:01:02,487 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:01:03,641 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:01:05,259 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:01:07,643 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:01:10,713 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:01:11,338 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:01:12,973 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:01:15,714 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:01:17,974 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:01:21,191 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:01:21,402 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:01:23,435 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:01:26,193 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:01:28,436 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:01:29,464 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:01:31,875 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:01:33,641 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:01:33,643 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:01:37,634 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:01:38,644 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:01:39,565 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:01:42,635 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:01:43,899 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:01:48,414 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:01:49,571 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:01:49,639 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:01:53,415 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:01:54,572 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:01:58,875 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:01:59,705 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:02:00,181 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:02:03,642 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:02:03,957 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:02:05,643 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:02:09,441 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:02:09,772 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:02:10,702 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:02:14,443 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:02:15,704 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:02:17,702 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 10:02:17,703 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 10:02:17,703 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 10:02:17,706 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 10:02:17,833 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 10:02:18,837 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:02:19,505 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:02:19,842 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:02:20,708 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:02:25,710 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:02:25,909 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:02:29,902 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:02:30,910 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:02:31,149 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:02:33,643 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:02:36,243 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:02:36,421 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:02:37,955 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:02:41,254 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:02:41,421 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:02:47,094 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:02:47,332 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:02:48,016 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:02:52,095 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:02:52,333 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:02:57,504 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:02:57,721 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:02:58,073 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:03:02,723 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:03:03,646 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:03:03,771 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:03:06,143 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:03:07,961 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:03:08,772 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:03:12,963 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:03:14,299 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:03:14,300 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:03:16,211 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:03:18,579 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:03:19,300 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:03:23,580 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:03:24,633 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:03:26,264 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:03:28,814 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:03:29,634 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:03:33,645 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:03:34,646 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:03:35,063 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:03:36,331 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:03:39,647 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:03:40,064 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:03:45,092 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:03:45,340 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:03:45,396 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:03:50,341 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:03:51,553 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:03:53,447 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:03:55,763 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:03:56,554 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:04:00,764 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:04:01,555 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:04:03,646 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:04:06,556 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:04:06,647 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:04:11,647 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:04:12,085 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:04:16,648 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:04:18,051 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:04:18,052 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:04:21,807 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:04:22,188 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:04:23,053 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:04:27,190 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:04:28,540 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:04:28,541 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:04:31,946 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:04:32,869 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:04:33,542 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:04:33,646 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:04:38,648 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:04:39,066 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:04:42,004 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:04:44,067 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:04:44,351 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:04:49,999 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:04:50,349 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:04:52,099 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:04:55,000 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:04:55,350 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:05:00,186 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:05:00,385 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:05:00,529 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:05:03,647 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:05:05,648 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:05:06,851 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:05:09,503 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:05:11,110 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:05:11,853 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:05:16,112 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:05:17,257 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:05:17,257 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:05:19,568 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:05:21,622 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:05:22,258 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:05:26,623 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:05:27,619 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:05:28,040 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:05:32,197 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:05:33,041 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:05:33,650 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:05:37,686 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:05:38,017 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:05:38,538 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:05:43,018 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:05:43,540 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:05:46,513 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 10:05:46,515 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 10:05:46,515 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 10:05:46,517 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 10:05:46,756 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:05:46,757 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 10:05:47,761 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:05:48,519 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:05:49,347 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:05:53,520 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:05:54,573 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:05:57,832 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:05:58,745 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:05:59,574 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:06:03,650 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:06:04,653 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:06:05,306 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:06:07,905 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:06:09,678 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:06:10,307 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:06:14,679 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:06:15,962 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:06:15,988 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:06:20,274 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:06:20,990 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:06:25,685 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:06:26,025 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:06:26,626 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:06:30,686 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:06:31,627 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:06:33,651 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:06:36,084 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:06:36,260 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:06:36,822 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:06:41,261 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:06:41,823 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:06:46,155 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:06:46,989 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:06:47,029 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:06:52,030 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:06:53,495 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:06:56,214 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:06:57,972 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:06:58,497 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:07:02,974 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:07:03,499 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:07:03,499 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:07:03,652 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:07:06,272 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:07:08,500 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:07:08,722 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:07:13,724 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:07:13,989 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:07:13,990 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:07:16,346 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:07:19,050 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:07:19,319 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:07:24,292 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:07:24,321 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:07:26,422 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:07:29,293 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:07:29,388 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:07:33,654 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:07:34,731 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:07:34,922 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:07:36,485 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:07:39,732 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:07:39,923 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:07:44,594 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:07:44,990 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:07:45,350 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:07:50,351 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:07:51,702 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:07:53,651 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:07:56,050 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:07:56,703 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:08:01,051 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:08:02,322 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:08:02,322 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:08:03,654 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:08:03,716 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:08:06,656 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:08:07,350 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:08:11,658 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:08:12,383 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:08:13,785 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:08:16,767 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:08:17,384 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:08:21,769 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:08:22,624 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:08:23,891 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:08:26,939 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:08:27,625 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:08:31,941 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:08:32,702 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:08:32,702 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:08:33,655 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:08:33,966 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:08:37,656 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:08:37,703 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:08:42,658 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:08:42,768 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:08:44,042 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:08:47,769 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:08:48,176 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:08:53,313 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:08:53,568 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:08:54,107 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:08:58,314 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:08:58,569 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:09:02,392 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 10:09:02,393 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 10:09:02,394 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 10:09:02,397 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 10:09:03,176 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:09:03,177 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 10:09:03,655 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:09:03,656 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:09:03,671 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:09:04,181 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:09:08,657 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:09:08,672 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:09:13,949 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:09:14,224 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:09:14,251 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:09:19,226 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:09:20,370 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:09:24,317 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:09:24,599 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:09:25,369 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:09:29,601 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:09:30,470 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:09:30,471 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:09:33,655 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:09:34,383 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:09:34,656 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:09:35,472 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:09:39,658 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:09:40,820 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:09:42,445 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:09:45,227 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:09:45,821 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:09:50,228 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:09:51,309 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:09:52,536 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:09:55,489 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:09:56,310 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:10:00,774 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:10:01,471 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:10:02,600 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:10:03,656 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:10:06,657 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:10:08,033 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:10:10,650 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:10:12,231 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:10:13,034 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:10:17,233 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:10:18,733 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:10:18,733 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:10:20,721 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:10:22,896 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:10:23,734 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:10:27,898 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:10:29,115 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:10:30,783 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:10:33,270 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:10:33,657 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:10:34,116 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:10:38,950 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:10:39,395 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:10:39,960 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:10:43,950 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:10:44,397 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:10:48,018 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:10:49,245 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:10:50,097 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:10:54,247 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:10:56,048 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:10:59,586 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:11:00,107 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:11:01,049 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:11:03,657 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:11:04,658 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:11:07,002 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:11:08,163 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:11:10,257 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:11:12,003 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:11:15,259 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:11:17,097 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:11:18,224 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:11:20,669 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:11:22,415 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:11:25,671 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:11:28,009 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:11:28,291 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:11:31,177 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:11:33,659 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:11:34,254 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:11:34,254 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:11:36,416 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:11:36,566 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:11:39,256 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:11:41,417 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:11:44,824 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:11:46,636 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:11:47,183 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:11:49,825 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:11:52,184 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:11:55,671 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:11:56,707 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:11:57,885 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:12:00,672 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:12:02,887 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:12:03,660 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:12:06,192 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:12:06,778 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:12:08,551 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:12:12,662 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:12:13,552 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:12:13,992 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 10:12:13,993 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 10:12:13,994 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 10:12:13,996 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 10:12:14,840 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 10:12:15,847 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:12:16,851 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:12:17,663 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:12:18,998 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:12:23,428 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:12:24,873 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:12:26,918 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:12:28,429 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:12:29,873 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:12:33,660 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:12:34,002 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:12:34,083 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:12:35,661 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:12:39,084 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:12:40,662 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:12:44,069 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:12:44,946 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:12:46,211 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:12:51,213 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:12:51,445 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:12:51,446 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:12:54,135 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:12:56,447 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:12:56,730 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:13:01,732 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:13:01,779 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:13:03,660 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:13:04,205 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:13:06,780 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:13:07,661 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:13:11,954 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:13:12,264 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:13:13,112 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:13:16,955 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:13:18,113 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:13:22,218 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:13:22,339 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:13:23,447 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:13:27,219 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:13:28,448 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:13:32,406 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:13:32,742 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:13:33,661 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:13:33,662 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:13:38,663 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:13:39,194 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:13:42,471 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:13:44,195 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:13:44,455 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:13:49,457 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:13:49,545 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:13:49,546 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:13:50,518 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:13:54,546 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:13:54,689 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:13:59,802 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:13:59,954 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:14:00,785 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:14:03,661 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:14:04,818 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:14:05,663 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:14:10,114 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:14:10,324 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:14:11,566 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:14:15,326 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:14:16,567 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:14:20,184 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:14:20,545 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:14:22,337 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:14:26,864 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:14:27,339 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:14:30,255 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:14:31,866 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:14:33,279 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:14:33,662 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:14:37,545 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:14:37,546 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:14:38,829 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:14:40,314 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:14:42,546 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:14:43,830 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:14:47,958 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:14:48,373 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:14:49,170 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:14:52,959 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:14:54,172 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:14:58,449 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:14:58,512 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:14:59,777 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:15:03,513 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:15:03,662 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:15:05,663 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:15:09,726 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:15:10,664 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:15:12,530 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:15:14,728 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:15:15,947 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:15:19,802 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:15:19,802 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:15:20,949 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:15:22,595 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:15:24,803 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:15:25,974 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:15:30,206 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:15:30,740 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 10:15:30,742 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 10:15:30,742 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 10:15:30,745 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 10:15:31,660 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:15:31,661 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 10:15:31,747 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:15:32,668 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:15:33,663 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:15:35,217 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:15:37,665 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:15:41,159 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:15:42,734 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:15:43,359 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:15:46,160 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:15:48,361 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:15:51,802 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:15:52,789 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:15:54,044 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:15:58,314 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:15:59,045 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:16:00,839 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:16:03,325 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:16:03,664 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:16:04,665 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:16:09,025 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:16:09,026 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:16:10,163 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:16:10,897 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:16:14,026 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:16:15,165 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:16:19,223 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:16:20,498 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:16:20,952 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:16:24,224 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:16:25,500 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:16:29,730 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:16:31,022 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:16:31,079 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:16:33,664 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:16:34,731 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:16:36,678 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:16:39,075 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:16:40,373 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:16:42,582 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:16:45,917 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:16:48,327 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:16:49,140 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:16:50,918 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:16:53,328 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:16:56,858 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:16:59,082 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:16:59,205 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:17:01,865 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:17:03,665 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:17:04,668 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:17:07,309 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:17:08,284 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:17:10,245 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:17:12,310 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:17:15,247 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:17:17,674 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:17:18,343 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:17:20,844 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:17:22,675 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:17:25,846 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:17:27,945 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:17:28,417 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:17:31,184 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:17:33,667 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:17:34,453 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:17:36,471 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:17:36,627 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:17:39,455 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:17:41,628 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:17:44,778 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:17:46,530 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:17:47,039 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:17:49,780 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:17:52,040 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:17:55,518 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:17:56,603 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:17:57,865 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:18:00,519 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:18:02,866 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:18:03,668 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:18:06,310 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:18:06,672 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:18:08,394 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:18:12,649 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:18:13,395 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:18:14,734 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:18:17,650 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:18:18,908 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:18:23,177 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:18:23,178 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:18:24,444 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:18:24,796 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:18:28,178 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:18:29,445 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:18:33,578 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:18:33,669 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:18:34,669 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:18:34,864 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:18:38,579 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:18:39,671 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:18:42,806 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 10:18:42,807 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 10:18:42,808 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 10:18:42,811 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 10:18:42,914 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 10:18:43,920 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:18:44,500 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:18:44,813 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:18:44,925 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:18:49,502 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:18:49,815 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:18:55,123 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:18:55,245 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:18:55,246 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:18:55,447 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:19:00,449 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:19:00,463 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:19:03,177 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:19:03,669 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:19:05,464 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:19:05,670 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:19:10,671 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:19:10,874 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:19:10,874 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:19:13,250 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:19:15,875 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:19:16,158 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:19:21,245 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:19:21,633 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:19:23,320 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:19:26,393 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:19:26,634 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:19:31,374 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:19:31,675 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:19:31,878 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:19:33,669 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:19:37,671 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:19:38,091 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:19:41,442 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:19:43,092 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:19:43,301 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:19:48,302 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:19:48,400 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:19:50,511 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:19:53,400 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:19:53,641 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:19:59,280 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:19:59,537 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:20:00,653 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:20:03,670 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:20:04,280 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:20:04,671 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:20:08,724 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:20:09,558 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:20:10,666 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:20:15,665 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:20:15,667 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:20:18,802 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:20:20,667 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:20:20,819 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:20:25,820 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:20:25,881 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:20:25,882 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:20:26,857 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:20:30,882 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:20:31,096 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:20:33,672 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:20:36,557 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:20:36,778 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:20:36,925 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:20:41,598 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:20:41,779 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:20:46,818 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:20:47,088 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:20:47,577 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:20:51,819 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:20:53,971 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:20:53,972 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:20:57,163 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:20:57,187 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:20:58,973 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:21:02,189 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:21:03,672 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:21:04,372 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:21:07,234 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:21:07,552 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:21:09,373 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:21:12,554 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:21:14,629 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:21:15,291 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:21:17,868 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:21:19,630 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:21:22,869 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:21:25,058 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:21:25,350 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:21:28,273 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:21:31,377 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:21:33,616 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:21:33,672 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:21:35,405 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:21:36,378 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:21:38,673 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:21:42,221 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:21:44,382 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:21:45,471 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:21:47,222 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:21:49,383 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:21:52,051 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 10:21:52,052 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 10:21:52,053 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 10:21:52,055 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 10:21:52,519 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 10:21:52,648 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:21:53,523 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:21:54,532 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:21:55,057 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:21:57,649 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:22:00,058 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:22:03,177 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:22:03,589 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:22:03,673 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:22:05,674 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:22:09,511 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:22:09,512 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:22:11,643 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:22:12,659 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:22:14,512 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:22:16,644 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:22:19,772 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:22:20,708 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:22:22,049 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:22:24,773 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:22:27,050 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:22:30,093 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:22:30,769 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:22:32,258 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:22:33,673 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:22:35,094 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:22:37,675 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:22:40,841 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:22:40,927 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:22:43,074 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:22:47,284 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:22:48,075 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:22:50,896 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:22:52,285 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:22:53,538 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:22:57,794 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:22:57,794 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:22:58,957 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:22:59,024 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:23:02,795 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:23:03,674 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:23:04,675 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:23:07,927 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:23:09,023 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:23:10,350 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:23:13,167 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:23:15,352 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:23:18,672 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:23:19,092 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:23:20,823 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:23:23,674 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:23:25,824 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:23:29,155 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:23:29,353 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:23:31,527 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:23:33,677 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:23:34,716 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:23:36,873 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:23:37,209 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:23:39,718 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:23:41,874 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:23:45,121 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:23:47,361 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:23:47,371 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:23:50,121 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:23:52,373 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:23:55,764 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:23:55,764 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:23:57,422 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:23:57,952 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:24:00,765 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:24:02,953 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:24:03,676 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:24:06,049 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:24:07,487 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:24:08,720 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:24:11,050 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:24:13,721 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:24:15,546 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:24:16,138 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:24:19,320 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:24:22,428 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:24:24,680 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:24:25,609 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:24:27,433 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:24:29,681 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:24:32,492 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:24:33,677 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:24:34,756 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:24:35,680 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:24:37,494 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:24:39,756 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:24:43,140 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:24:43,140 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:24:45,349 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:24:45,739 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:24:48,140 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:24:50,350 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:24:53,764 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:24:54,800 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:24:56,009 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:24:58,820 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:25:01,010 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:25:03,098 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 10:25:03,100 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 10:25:03,100 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 10:25:03,103 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 10:25:03,677 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:25:03,871 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 10:25:04,709 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:25:04,877 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:25:06,678 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:25:09,711 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:25:11,680 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:25:14,951 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:25:15,556 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:25:17,449 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:25:21,984 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:25:22,451 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:25:25,021 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:25:26,985 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:25:28,404 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:25:32,610 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:25:33,678 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:25:33,678 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:25:35,089 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:25:37,612 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:25:38,679 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:25:43,107 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:25:43,108 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:25:44,068 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:25:45,146 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:25:48,120 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:25:49,069 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:25:53,765 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:25:53,765 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:25:54,225 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:25:55,209 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:25:59,077 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:25:59,226 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:26:03,264 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:26:03,678 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:26:04,290 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:26:04,679 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:26:09,291 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:26:09,680 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:26:13,331 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:26:15,253 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:26:15,346 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:26:20,348 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:26:20,442 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:26:23,390 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:26:25,443 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:26:25,816 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:26:30,817 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:26:30,832 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:26:33,455 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:26:33,679 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:26:35,833 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:26:36,680 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:26:41,195 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:26:41,511 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:26:42,381 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:26:46,196 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:26:47,382 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:26:51,449 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:26:51,573 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:26:52,691 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:26:56,450 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:26:57,693 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:27:01,629 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:27:01,940 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:27:03,184 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:27:03,679 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:27:08,326 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:27:08,681 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:27:11,691 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:27:13,327 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:27:14,524 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:27:18,828 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:27:20,226 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:27:21,750 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:27:23,829 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:27:25,227 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:27:29,395 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:27:29,809 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:27:30,796 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:27:33,680 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:27:34,396 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:27:36,681 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:27:39,436 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:27:39,887 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:27:41,744 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:27:44,437 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:27:46,746 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:27:49,954 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:27:50,050 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:27:52,266 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:27:56,535 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:27:56,535 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:27:57,267 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:28:00,018 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:28:01,536 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:28:02,842 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:28:03,680 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:28:07,018 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:28:08,393 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:28:09,079 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:28:12,026 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:28:13,394 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:28:17,149 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 10:28:17,151 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 10:28:17,151 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 10:28:17,154 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 10:28:17,968 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:28:18,143 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:28:18,146 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 10:28:19,149 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:28:19,156 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:28:22,969 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:28:24,158 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:28:27,228 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:28:28,271 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:28:29,617 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:28:33,273 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:28:33,695 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:28:34,698 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:28:38,325 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:28:39,313 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:28:39,942 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:28:43,326 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:28:44,944 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:28:47,364 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:28:48,407 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:28:50,642 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:28:54,730 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:28:55,643 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:28:57,431 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:28:59,732 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:29:00,966 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:29:03,683 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:29:05,099 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:29:06,298 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:29:07,507 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:29:10,100 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:29:11,298 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:29:15,322 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:29:15,566 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:29:16,486 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:29:20,322 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:29:21,487 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:29:25,323 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:29:26,488 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:29:30,840 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:29:31,651 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:29:32,274 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:29:33,683 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:29:35,842 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:29:37,685 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:29:40,847 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:29:41,712 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:29:43,383 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:29:45,866 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:29:48,384 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:29:51,103 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:29:53,385 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:29:56,104 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:29:58,386 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:30:01,334 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:30:01,838 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:30:03,684 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:30:03,685 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:30:06,336 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:30:08,688 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:30:11,617 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:30:11,909 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:30:14,058 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:30:16,622 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:30:19,060 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:30:21,972 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:30:22,475 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:30:24,678 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:30:27,984 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:30:30,253 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:30:32,046 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:30:32,985 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:30:33,685 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:30:35,686 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:30:38,975 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:30:41,248 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:30:42,122 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:30:43,976 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:30:46,249 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:30:49,373 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:30:50,169 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:30:51,576 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:30:54,374 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:30:56,578 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:30:59,637 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:30:59,638 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:31:00,235 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:31:01,835 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:31:03,685 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:31:06,008 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:31:07,475 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:31:09,293 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:31:11,009 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:31:12,476 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:31:16,957 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:31:18,304 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:31:19,387 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:31:21,958 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:31:23,305 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:31:27,504 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:31:27,637 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:31:28,867 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:31:32,702 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:31:33,686 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:31:34,687 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:31:37,978 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:31:39,606 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:31:40,169 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:31:42,979 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:31:45,170 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:31:47,520 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 10:31:47,522 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 10:31:47,523 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 10:31:47,525 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 10:31:47,665 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 10:31:48,347 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:31:48,671 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:31:49,678 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:31:50,526 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:31:53,348 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:31:55,528 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:31:58,638 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:31:59,737 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:32:01,438 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:32:03,686 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:32:04,056 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:32:06,688 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:32:07,790 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:32:09,713 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:32:11,972 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:32:14,715 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:32:16,974 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:32:19,874 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:32:20,262 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:32:22,154 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:32:26,770 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:32:26,771 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:32:26,771 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:32:27,156 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:32:29,943 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:32:31,772 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:32:32,854 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:32:33,687 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:32:37,080 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:32:38,005 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:32:38,182 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:32:42,080 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:32:43,183 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:32:47,280 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:32:48,081 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:32:48,323 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:32:52,281 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:32:53,325 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:32:57,408 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:32:58,152 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:32:58,616 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:33:02,410 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:33:03,618 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:33:03,690 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:33:08,130 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:33:08,225 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:33:09,364 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:33:13,578 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:33:13,578 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:33:14,365 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:33:16,289 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:33:18,579 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:33:19,753 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:33:23,983 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:33:25,307 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:33:26,349 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:33:28,984 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:33:30,308 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:33:33,689 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:33:34,863 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:33:36,083 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:33:36,413 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:33:39,865 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:33:41,085 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:33:45,371 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:33:45,372 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:33:45,480 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:33:46,526 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:33:50,373 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:33:51,527 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:33:55,548 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:33:55,848 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:33:57,210 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:34:00,849 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:34:02,212 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:34:03,690 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:34:05,615 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:34:06,400 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:34:07,691 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:34:12,404 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:34:12,693 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:34:15,677 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:34:17,405 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:34:18,619 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:34:22,911 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:34:23,735 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:34:24,129 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:34:27,912 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:34:29,130 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:34:33,584 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:34:33,690 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:34:33,805 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:34:34,691 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:34:38,585 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:34:39,817 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:34:39,837 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:34:40,842 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:34:43,586 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:34:44,818 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:34:45,867 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:34:46,874 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:34:48,587 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:34:50,275 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:34:54,190 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:34:54,317 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 10:34:54,321 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 10:34:54,322 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 10:34:54,324 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 10:34:54,923 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:34:54,924 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 10:34:55,327 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:34:55,927 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:34:59,191 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:35:00,327 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:35:03,691 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:35:04,478 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:35:05,709 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:35:05,993 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:35:09,478 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:35:10,710 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:35:14,052 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:35:14,807 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:35:16,006 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:35:21,008 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:35:21,131 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:35:24,121 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:35:26,132 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:35:26,313 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:35:31,314 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:35:31,368 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:35:31,369 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:35:33,691 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:35:34,187 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:35:36,408 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:35:36,692 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:35:42,283 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:35:42,535 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:35:44,253 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:35:47,284 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:35:47,536 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:35:52,306 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:35:52,575 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:35:52,756 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:35:57,757 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:35:59,056 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:35:59,057 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:36:02,370 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:36:03,354 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:36:03,692 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:36:04,056 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:36:08,693 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:36:09,682 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:36:12,441 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:36:13,927 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:36:14,683 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:36:18,928 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:36:20,085 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:36:20,497 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:36:24,266 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:36:25,086 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:36:30,038 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:36:30,383 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:36:30,571 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:36:33,692 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:36:35,384 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:36:35,693 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:36:39,637 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:36:41,025 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:36:41,203 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:36:46,205 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:36:47,376 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:36:47,377 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:36:49,709 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:36:51,522 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:36:52,378 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:36:56,523 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:36:57,406 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:36:57,407 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:36:59,781 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:37:01,965 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:37:02,595 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:37:03,693 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:37:07,413 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:37:07,834 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:37:08,081 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:37:12,414 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:37:13,082 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:37:17,796 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:37:17,896 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:37:18,602 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:37:22,798 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:37:23,604 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:37:27,963 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:37:28,711 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:37:29,407 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:37:33,694 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:37:34,696 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:37:34,932 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:37:38,042 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:37:39,933 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:37:40,165 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:37:45,166 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:37:45,824 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:37:45,824 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:37:48,115 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:37:50,183 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:37:50,825 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:37:55,184 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:37:56,173 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:37:56,312 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:38:00,673 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:38:01,313 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:38:03,708 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:38:04,921 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 10:38:04,922 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 10:38:04,923 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 10:38:04,926 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 10:38:05,230 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 10:38:05,927 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:38:06,236 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:38:06,741 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:38:07,242 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:38:10,928 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:38:11,743 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:38:16,311 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:38:16,885 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:38:17,578 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:38:21,887 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:38:23,149 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:38:26,387 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:38:27,398 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:38:28,150 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:38:32,399 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:38:33,696 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:38:33,760 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:38:36,454 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:38:37,915 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:38:38,761 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:38:42,917 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:38:44,230 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:38:44,511 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:38:48,456 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:38:49,231 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:38:54,395 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:38:54,578 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:38:54,611 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:38:59,395 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:38:59,612 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:39:03,696 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:39:04,697 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:39:04,745 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:39:05,264 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:39:09,698 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:39:10,266 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:39:14,807 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:39:15,328 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:39:15,579 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:39:20,330 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:39:21,507 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:39:23,870 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:39:25,721 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:39:26,509 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:39:30,722 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:39:32,164 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:39:32,165 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:39:33,697 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:39:33,930 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:39:36,698 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:39:37,165 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:39:41,699 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:39:42,760 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:39:43,993 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:39:46,959 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:39:47,761 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:39:52,042 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:39:52,703 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:39:53,515 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:39:56,057 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:39:58,516 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:39:58,556 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:40:03,698 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:40:03,698 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:40:04,072 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:40:04,120 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:40:08,699 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:40:09,074 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:40:14,187 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:40:14,240 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:40:14,579 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:40:19,241 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:40:20,605 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:40:22,242 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:40:24,739 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:40:25,606 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:40:29,740 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:40:31,241 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:40:31,242 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:40:32,315 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:40:33,698 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:40:35,699 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:40:36,242 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:40:40,700 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:40:41,841 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:40:42,382 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:40:46,059 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:40:46,842 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:40:51,865 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:40:52,058 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:40:52,442 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:40:56,866 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:40:57,059 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:41:02,479 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:41:02,480 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:41:02,522 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:41:02,722 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:41:03,699 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:41:08,700 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:41:08,950 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:41:12,582 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:41:13,951 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:41:14,264 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:41:19,265 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:41:19,468 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:41:19,658 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 10:41:19,659 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 10:41:19,660 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 10:41:19,663 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 10:41:20,653 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 10:41:21,660 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:41:22,664 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:41:24,469 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:41:24,665 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:41:29,479 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:41:29,479 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:41:30,047 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:41:30,723 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:41:33,699 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:41:34,835 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:41:35,700 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:41:40,136 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:41:40,784 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:41:41,339 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:41:45,137 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:41:46,341 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:41:50,299 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:41:50,856 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:41:51,646 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:41:55,301 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:41:56,647 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:42:00,479 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:42:00,917 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:42:02,390 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:42:03,700 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:42:06,688 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:42:07,701 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:42:10,991 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:42:11,690 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:42:12,952 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:42:17,351 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:42:17,352 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:42:18,549 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:42:20,051 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:42:22,352 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:42:23,550 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:42:27,480 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:42:28,648 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:42:30,117 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:42:32,481 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:42:32,482 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:42:33,649 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:42:33,700 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:42:38,314 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:42:39,561 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:42:40,181 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:42:43,315 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:42:44,562 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:42:48,242 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:42:48,863 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:42:48,863 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:42:50,256 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:42:55,258 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:42:55,363 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:42:58,306 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:43:00,364 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:43:00,583 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:43:03,703 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:43:05,705 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:43:05,837 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:43:05,838 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:43:08,373 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:43:10,839 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:43:11,167 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:43:15,863 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:43:16,757 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:43:18,439 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:43:21,496 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:43:21,758 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:43:26,740 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:43:26,957 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:43:28,503 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:43:31,741 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:43:31,958 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:43:33,704 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:43:36,578 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:43:37,344 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:43:37,638 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:43:42,345 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:43:42,639 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:43:47,347 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:43:47,641 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:43:52,696 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:43:53,103 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:43:53,565 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:43:58,567 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:43:59,197 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:44:02,764 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:44:03,704 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:44:03,705 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:44:04,198 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:44:08,707 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:44:09,474 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:44:10,825 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:44:14,475 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:44:14,673 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:44:19,960 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:44:20,603 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:44:20,895 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:44:24,961 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:44:25,604 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:44:30,217 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:44:30,958 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:44:31,450 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:44:33,705 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:44:35,218 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:44:36,706 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:44:38,885 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 10:44:38,886 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 10:44:38,886 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 10:44:38,889 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 10:44:39,009 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 10:44:40,016 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:44:40,608 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:44:41,024 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:44:41,891 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:44:45,610 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:44:46,892 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:44:51,093 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:44:51,572 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:44:52,861 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:44:56,962 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:44:57,862 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:45:01,172 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:45:01,963 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:45:03,215 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:45:03,706 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:45:07,471 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:45:08,740 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:45:09,221 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:45:12,472 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:45:13,741 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:45:17,776 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:45:18,280 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:45:18,966 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:45:22,777 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:45:23,968 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:45:27,881 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:45:28,350 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:45:28,971 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:45:32,883 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:45:33,707 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:45:34,709 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:45:38,432 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:45:38,880 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:45:40,053 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:45:44,525 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:45:45,055 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:45:48,496 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:45:49,526 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:45:50,847 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:45:55,099 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:45:56,289 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:45:56,553 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:46:00,100 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:46:01,289 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:46:03,709 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:46:05,718 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:46:05,718 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:46:06,616 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:46:06,873 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:46:10,719 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:46:11,874 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:46:15,865 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:46:15,866 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:46:16,691 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:46:16,936 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:46:21,537 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:46:21,937 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:46:26,752 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:46:27,166 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:46:27,359 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:46:32,361 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:46:32,448 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:46:33,709 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:46:34,810 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:46:37,450 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:46:37,710 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:46:42,711 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:46:42,763 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:46:44,882 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:46:47,764 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:46:47,966 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:46:53,250 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:46:53,393 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:46:54,939 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:46:58,251 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:46:58,394 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:47:03,699 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:47:03,709 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:47:03,710 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:47:05,005 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:47:08,701 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:47:08,723 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:47:13,058 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:47:13,795 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:47:14,463 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:47:18,797 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:47:20,127 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:47:23,116 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:47:24,631 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:47:25,128 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:47:29,632 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:47:30,866 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:47:30,867 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:47:33,185 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:47:33,720 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:47:34,721 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:47:35,950 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:47:39,722 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:47:41,319 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:47:43,279 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:47:45,516 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:47:46,320 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:47:50,141 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 10:47:50,145 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 10:47:50,145 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 10:47:50,148 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 10:47:50,331 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 10:47:51,149 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:47:51,985 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:47:52,342 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:47:53,347 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:47:56,150 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:47:58,291 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:48:01,401 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:48:01,512 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:48:03,292 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:48:03,710 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:48:06,712 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:48:08,690 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:48:10,464 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:48:11,933 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:48:13,690 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:48:16,934 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:48:19,076 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:48:19,076 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:48:20,539 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:48:22,315 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:48:24,077 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:48:27,317 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:48:28,593 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:48:29,527 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:48:32,757 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:48:33,711 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:48:34,529 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:48:38,632 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:48:38,667 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:48:40,400 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:48:43,633 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:48:45,720 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:48:48,731 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:48:49,075 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:48:50,722 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:48:54,076 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:48:56,253 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:48:58,796 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:48:59,437 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:49:01,254 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:49:03,712 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:49:04,714 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:49:06,557 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:49:06,557 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:49:06,889 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:49:09,777 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:49:11,558 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:49:14,778 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:49:16,920 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:49:16,954 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:49:20,163 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:49:21,922 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:49:25,781 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:49:27,027 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:49:27,660 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:49:30,782 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:49:32,938 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:49:33,712 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:49:36,218 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:49:37,095 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:49:37,939 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:49:41,219 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:49:43,229 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:49:45,159 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:49:46,349 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:49:48,230 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:49:51,351 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:49:53,450 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:49:55,224 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:49:56,612 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:49:58,451 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:50:01,613 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:50:03,713 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:50:03,925 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:50:05,291 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:50:06,714 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:50:10,310 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:50:11,715 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:50:13,340 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:50:15,311 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:50:16,745 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:50:20,532 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:50:20,532 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:50:21,747 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:50:23,399 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:50:25,533 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:50:26,992 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:50:31,247 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:50:32,457 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:50:33,467 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:50:33,713 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:50:36,248 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:50:37,714 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:50:41,892 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:50:42,922 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:50:43,537 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:50:46,895 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:50:47,924 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:50:52,427 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:50:53,606 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:50:53,612 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:50:58,607 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:50:58,850 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:51:00,062 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 10:51:00,064 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 10:51:00,064 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 10:51:00,067 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 10:51:00,657 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:51:00,658 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 10:51:01,662 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:51:03,714 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:51:03,717 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:51:03,851 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:51:08,718 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:51:09,692 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:51:10,713 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:51:14,091 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:51:14,730 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:51:19,092 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:51:20,427 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:51:20,785 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:51:24,933 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:51:25,581 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:51:30,736 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:51:30,848 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:51:31,289 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:51:33,715 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:51:36,356 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:51:36,716 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:51:40,911 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:51:42,341 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:51:42,589 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:51:47,591 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:51:48,118 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:51:50,975 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:51:53,120 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:51:53,343 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:51:58,345 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:51:59,084 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:52:01,039 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:52:03,715 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:52:03,716 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:52:04,085 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:52:08,717 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:52:09,512 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:52:11,098 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:52:13,762 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:52:14,513 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:52:19,399 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:52:20,092 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:52:21,170 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:52:24,400 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:52:25,095 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:52:29,226 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:52:29,867 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:52:30,771 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:52:33,716 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:52:35,717 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:52:36,027 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:52:36,027 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:52:39,286 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:52:41,028 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:52:41,292 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:52:46,294 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:52:46,790 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:52:49,347 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:52:51,791 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:52:51,981 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:52:57,379 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:52:57,479 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:52:59,424 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:53:02,379 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:53:02,480 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:53:03,718 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:53:07,517 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:53:07,517 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:53:07,753 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:53:09,489 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:53:12,518 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:53:12,754 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:53:18,213 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:53:18,633 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:53:19,558 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:53:23,215 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:53:23,634 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:53:27,607 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:53:29,082 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:53:29,222 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:53:33,720 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:53:34,454 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:53:34,722 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:53:37,674 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:53:39,455 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:53:40,660 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:53:44,867 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:53:46,096 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:53:47,763 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:53:49,868 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:53:51,097 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:53:55,647 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:53:56,826 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:53:57,829 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:54:00,648 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:54:01,827 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:54:03,720 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:54:05,706 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:54:06,916 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:54:07,827 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:54:10,707 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:54:12,829 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:54:14,306 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 10:54:14,307 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 10:54:14,308 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 10:54:14,310 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 10:54:14,968 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:54:14,968 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 10:54:15,974 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:54:16,007 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:54:18,313 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:54:21,008 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:54:23,314 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:54:25,038 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:54:26,876 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:54:29,086 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:54:32,430 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:54:33,722 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:54:34,724 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:54:35,099 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:54:37,435 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:54:39,725 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:54:42,575 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:54:44,742 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:54:45,167 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:54:47,576 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:54:49,743 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:54:52,885 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:54:52,885 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:54:55,140 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:54:55,238 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:54:57,889 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:55:00,141 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:55:03,463 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:55:03,721 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:55:05,302 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:55:05,722 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:55:08,464 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:55:10,724 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:55:13,355 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:55:13,823 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:55:16,001 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:55:20,113 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:55:21,002 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:55:23,426 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:55:25,114 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:55:26,285 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:55:30,742 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:55:31,878 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:55:33,487 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:55:33,722 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:55:35,743 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:55:37,724 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:55:41,344 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:55:41,344 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:55:41,554 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:55:43,530 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:55:46,345 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:55:48,532 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:55:51,626 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:55:51,885 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:55:51,885 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:55:54,188 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:55:57,035 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:55:59,190 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:56:01,686 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:56:02,634 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:56:03,722 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:56:04,723 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:56:08,263 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:56:08,264 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:56:08,266 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:56:10,502 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:56:11,758 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:56:13,265 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:56:15,503 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:56:18,788 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:56:21,076 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:56:21,936 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:56:23,789 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:56:26,078 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:56:29,633 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:56:31,750 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:56:32,003 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:56:33,723 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:56:34,634 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:56:37,725 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:56:39,057 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:56:40,063 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:56:43,406 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:56:45,065 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:56:48,946 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:56:49,123 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:56:50,670 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:56:53,947 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:56:56,465 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:56:56,467 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:56:59,197 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:56:59,747 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:57:01,466 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:57:03,734 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:57:05,736 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:57:07,111 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:57:09,268 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:57:11,334 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:57:12,113 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:57:16,335 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:57:17,519 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:57:19,323 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:57:21,767 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:57:22,538 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:57:26,769 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:57:26,811 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 10:57:26,813 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 10:57:26,814 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 10:57:26,816 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 10:57:27,376 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 10:57:27,618 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:57:27,619 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:57:28,381 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:57:29,390 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:57:31,819 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:57:32,619 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:57:33,725 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:57:37,302 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:57:37,452 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:57:38,044 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:57:42,305 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:57:43,046 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:57:47,516 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:57:48,139 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:57:48,919 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:57:53,139 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:57:53,920 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:57:58,140 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:57:59,454 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:58:03,614 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:58:03,725 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:58:03,726 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:58:04,455 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:58:08,727 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:58:09,753 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:58:09,753 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:58:13,680 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:58:14,218 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:58:14,754 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:58:19,220 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:58:19,914 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:58:23,745 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:58:24,250 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:58:24,915 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:58:29,251 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:58:30,486 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:58:33,726 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:58:33,826 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:58:34,727 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:58:35,487 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:58:39,728 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:58:40,708 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:58:43,901 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:58:44,907 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:58:45,708 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:58:49,909 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:58:51,232 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:58:52,036 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:58:55,610 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:58:56,232 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:59:01,317 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:59:01,784 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:59:02,111 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:59:03,726 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:59:06,727 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:59:06,785 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:59:12,182 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:59:12,258 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:59:12,566 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:59:17,259 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:59:17,567 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:59:22,869 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:59:23,241 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:59:23,417 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:59:27,871 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:59:28,242 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:59:33,477 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:59:33,727 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 10:59:33,728 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:59:33,743 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:59:38,728 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:59:39,400 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:59:41,540 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:59:44,401 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:59:44,524 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:59:49,525 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 10:59:49,842 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:59:51,609 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 10:59:54,842 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 10:59:55,098 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:00:00,630 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:00:00,859 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:00:01,681 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:00:03,727 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:00:05,631 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:00:06,729 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:00:10,755 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:00:11,748 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:00:12,164 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:00:15,961 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:00:17,166 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:00:19,814 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:00:21,634 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:00:22,858 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:00:27,088 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:00:27,860 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:00:29,881 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:00:32,089 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:00:33,357 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:00:33,728 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:00:37,685 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:00:38,884 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:00:39,954 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:00:42,686 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:00:43,885 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:00:47,924 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 11:00:47,926 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 11:00:47,926 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 11:00:47,929 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 11:00:48,015 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 11:00:48,643 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:00:48,930 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:00:50,028 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:00:51,033 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:00:53,644 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:00:53,931 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:00:58,814 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:00:58,815 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:00:58,981 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:01:00,094 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:01:03,728 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:01:03,815 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:01:04,729 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:01:09,185 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:01:10,159 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:01:10,380 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:01:15,381 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:01:15,603 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:01:18,216 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:01:20,605 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:01:20,910 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:01:25,911 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:01:26,266 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:01:26,267 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:01:28,281 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:01:31,267 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:01:31,418 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:01:33,730 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:01:36,539 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:01:36,678 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:01:38,345 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:01:41,540 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:01:41,679 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:01:47,313 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:01:47,644 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:01:48,417 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:01:52,315 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:01:52,660 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:01:57,484 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:01:57,956 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:01:58,219 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:02:03,220 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:02:03,386 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:02:03,731 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:02:05,542 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:02:08,387 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:02:08,560 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:02:13,561 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:02:13,724 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:02:13,724 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:02:15,620 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:02:18,725 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:02:18,997 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:02:23,766 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:02:24,988 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:02:25,683 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:02:28,767 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:02:29,989 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:02:33,730 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:02:34,222 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:02:35,738 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:02:35,752 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:02:39,223 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:02:40,740 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:02:44,933 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:02:44,933 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:02:45,816 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:02:46,178 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:02:51,179 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:02:51,488 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:02:53,871 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:02:56,489 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:02:56,764 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:03:01,766 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:03:01,917 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:03:03,731 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:03:03,943 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:03:06,918 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:03:07,732 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:03:12,552 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:03:13,696 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:03:14,004 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:03:17,553 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:03:18,698 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:03:23,277 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:03:24,087 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:03:24,498 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:03:29,500 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:03:29,773 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:03:29,775 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:03:32,158 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:03:33,731 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:03:34,732 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:03:34,775 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:03:39,733 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:03:40,105 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:03:42,232 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:03:45,106 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:03:45,247 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:03:50,466 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:03:50,743 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:03:52,298 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:03:55,467 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:03:55,744 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:03:59,127 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 11:03:59,129 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 11:03:59,129 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 11:03:59,132 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 11:03:59,347 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 11:04:00,352 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:04:00,868 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:04:01,134 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:04:01,357 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:04:03,732 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:04:06,733 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:04:07,218 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:04:10,423 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:04:12,219 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:04:12,338 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:04:17,339 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:04:17,762 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:04:20,484 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:04:22,762 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:04:22,953 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:04:28,111 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:04:28,321 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:04:30,557 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:04:33,112 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:04:33,322 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:04:33,732 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:04:38,610 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:04:38,928 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:04:39,093 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:04:44,095 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:04:45,259 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:04:45,259 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:04:47,685 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:04:49,552 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:04:50,260 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:04:54,553 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:04:55,850 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:04:57,745 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:05:00,082 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:05:00,851 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:05:03,735 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:05:05,395 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:05:06,199 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:05:07,812 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:05:10,396 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:05:11,200 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:05:15,398 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:05:16,568 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:05:17,879 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:05:20,741 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:05:21,570 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:05:26,402 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:05:27,036 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:05:27,941 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:05:31,403 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:05:32,038 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:05:33,737 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:05:36,009 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:05:37,054 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:05:37,664 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:05:42,056 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:05:43,362 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:05:43,362 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:05:46,073 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:05:47,600 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:05:48,363 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:05:52,602 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:05:53,902 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:05:56,133 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:05:58,164 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:05:58,902 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:06:03,165 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:06:03,736 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:06:04,596 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:06:06,197 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:06:08,738 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:06:09,597 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:06:14,423 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:06:15,252 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:06:16,261 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:06:19,424 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:06:21,769 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:06:24,330 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:06:25,084 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:06:26,770 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:06:30,085 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:06:32,518 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:06:33,763 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:06:34,395 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:06:35,764 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:06:37,518 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:06:40,766 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:06:43,149 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:06:44,477 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:06:46,611 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:06:48,200 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:06:51,613 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:06:53,944 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:06:54,539 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:06:57,145 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:07:00,443 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:07:00,444 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:07:02,654 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:07:03,738 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:07:04,615 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:07:05,445 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:07:07,739 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:07:10,859 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:07:11,039 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 11:07:11,040 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 11:07:11,040 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 11:07:11,043 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 11:07:11,656 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 11:07:12,661 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:07:13,045 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:07:13,667 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:07:15,860 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:07:18,047 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:07:21,595 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:07:22,735 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:07:23,848 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:07:26,596 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:07:28,850 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:07:32,069 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:07:32,798 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:07:33,738 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:07:34,739 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:07:38,492 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:07:40,637 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:07:41,854 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:07:43,493 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:07:45,638 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:07:48,825 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:07:49,922 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:07:51,069 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:07:53,826 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:07:56,070 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:07:59,224 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:07:59,987 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:08:01,458 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:08:03,739 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:08:04,225 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:08:06,740 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:08:10,056 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:08:10,140 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:08:12,413 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:08:16,632 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:08:16,633 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:08:17,414 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:08:20,119 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:08:21,634 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:08:22,947 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:08:27,253 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:08:28,177 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:08:30,181 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:08:32,535 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:08:33,739 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:08:33,740 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:08:38,339 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:08:38,742 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:08:42,261 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:08:43,340 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:08:43,749 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:08:48,750 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:08:49,192 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:08:50,322 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:08:54,249 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:08:54,362 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:08:59,347 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:08:59,348 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:08:59,363 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:09:00,396 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:09:03,748 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:09:04,348 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:09:04,749 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:09:09,785 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:09:10,095 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:09:10,462 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:09:14,786 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:09:15,096 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:09:20,192 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:09:20,366 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:09:20,562 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:09:25,202 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:09:25,368 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:09:30,357 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:09:30,621 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:09:31,020 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:09:33,749 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:09:36,487 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:09:36,750 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:09:40,693 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:09:41,488 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:09:41,779 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:09:46,781 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:09:47,375 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:09:47,376 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:09:50,762 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:09:52,088 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:09:52,376 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:09:57,089 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:09:57,610 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:10:00,837 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:10:02,611 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:10:02,795 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:10:03,750 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:10:07,993 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:10:08,233 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:10:08,892 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:10:12,993 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:10:13,234 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:10:18,586 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:10:18,727 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:10:18,960 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:10:23,729 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:10:25,145 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:10:26,334 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 11:10:26,335 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 11:10:26,335 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 11:10:26,338 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 11:10:27,021 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:10:27,021 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 11:10:29,032 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:10:29,340 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:10:30,146 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:10:33,751 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:10:34,753 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:10:35,916 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:10:38,099 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:10:40,120 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:10:40,917 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:10:45,122 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:10:45,998 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:10:45,999 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:10:48,168 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:10:50,397 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:10:50,999 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:10:55,399 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:10:56,555 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:10:58,225 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:11:00,811 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:11:01,556 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:11:03,752 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:11:06,161 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:11:06,280 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:11:06,940 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:11:11,162 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:11:13,403 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:11:16,339 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:11:16,603 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:11:18,405 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:11:21,604 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:11:23,709 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:11:26,404 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:11:26,929 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:11:28,710 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:11:31,930 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:11:33,752 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:11:34,353 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:11:34,470 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:11:37,754 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:11:39,354 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:11:42,755 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:11:44,404 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:11:44,538 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:11:48,257 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:11:50,127 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:11:53,258 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:11:54,607 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:11:55,937 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:11:58,312 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:12:01,518 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:12:01,518 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:12:03,753 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:12:03,753 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:12:04,674 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:12:06,519 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:12:08,754 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:12:12,476 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:12:14,692 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:12:14,741 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:12:17,477 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:12:19,692 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:12:22,478 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:12:24,694 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:12:27,542 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:12:28,815 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:12:29,988 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:12:32,543 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:12:33,757 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:12:35,760 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:12:37,573 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:12:38,881 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:12:41,505 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:12:42,574 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:12:46,507 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:12:47,957 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:12:47,958 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:12:48,948 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:12:51,617 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:12:52,959 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:12:56,619 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:12:58,337 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:12:59,018 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:13:01,634 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:13:03,338 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:13:03,757 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:13:06,760 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:13:08,676 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:13:09,089 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:13:11,851 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:13:14,988 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:13:14,988 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:13:17,209 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:13:19,160 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:13:19,989 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:13:22,210 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:13:25,533 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:13:27,211 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:13:27,815 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:13:30,534 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:13:32,817 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:13:33,758 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:13:36,194 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:13:37,279 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:13:38,343 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:13:41,195 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:13:43,345 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:13:45,277 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 11:13:45,279 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 11:13:45,279 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 11:13:45,282 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 11:13:45,332 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 11:13:46,336 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:13:47,045 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:13:47,046 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:13:47,340 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:13:49,284 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:13:53,542 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:13:54,285 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:13:56,402 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:13:58,544 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:13:59,834 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:14:03,758 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:14:03,973 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:14:03,973 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:14:05,194 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:14:06,463 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:14:08,974 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:14:10,195 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:14:14,045 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:14:14,045 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:14:14,522 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:14:15,502 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:14:19,358 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:14:20,503 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:14:24,586 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:14:24,789 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:14:26,091 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:14:31,092 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:14:31,232 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:14:31,233 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:14:33,759 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:14:34,650 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:14:36,263 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:14:36,760 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:14:41,761 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:14:42,051 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:14:44,720 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:14:47,052 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:14:47,175 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:14:52,595 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:14:52,779 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:14:52,782 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:14:57,597 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:14:57,780 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:15:02,845 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:15:03,185 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:15:03,186 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:15:03,339 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:15:03,759 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:15:08,761 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:15:09,555 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:15:12,917 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:15:13,845 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:15:14,556 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:15:18,847 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:15:20,192 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:15:23,044 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:15:24,424 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:15:25,193 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:15:29,426 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:15:30,208 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:15:33,116 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:15:33,760 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:15:34,761 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:15:35,209 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:15:40,518 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:15:40,673 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:15:41,173 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:15:45,519 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:15:45,674 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:15:50,774 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:15:51,028 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:15:51,245 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:15:55,775 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:15:56,029 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:16:01,185 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:16:01,309 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:16:01,626 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:16:03,771 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:16:06,774 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:16:07,244 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:16:11,380 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:16:12,245 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:16:12,499 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:16:17,500 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:16:17,836 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:16:17,837 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:16:19,434 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:16:22,837 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:16:23,100 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:16:28,697 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:16:28,987 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:16:29,497 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:16:33,698 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:16:33,761 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:16:34,762 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:16:38,557 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:16:39,068 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:16:40,272 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:16:45,273 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:16:45,409 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:16:45,409 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:16:45,410 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:16:48,666 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:16:50,410 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:16:50,630 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:16:55,631 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:16:55,712 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:16:56,379 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 11:16:56,381 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 11:16:56,381 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 11:16:56,384 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 11:16:56,716 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 11:16:57,722 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:16:58,727 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:17:00,713 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:17:01,386 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:17:03,762 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:17:06,598 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:17:06,852 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:17:06,876 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:17:11,599 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:17:11,852 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:17:16,665 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:17:16,878 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:17:16,954 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:17:21,666 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:17:21,879 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:17:27,023 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:17:27,237 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:17:27,494 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:17:32,495 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:17:33,658 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:17:33,659 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:17:33,764 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:17:37,103 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:17:37,916 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:17:38,660 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:17:42,918 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:17:43,989 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:17:45,503 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:17:48,110 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:17:48,990 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:17:53,112 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:17:54,356 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:17:55,575 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:17:58,612 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:17:59,357 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:18:03,764 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:18:03,765 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:18:04,636 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:18:04,655 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:18:08,766 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:18:09,657 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:18:14,514 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:18:14,701 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:18:15,332 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:18:19,528 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:18:20,899 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:18:20,899 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:18:22,758 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:18:25,090 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:18:25,900 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:18:30,091 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:18:31,447 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:18:32,823 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:18:33,765 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:18:35,766 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:18:36,448 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:18:40,767 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:18:42,111 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:18:42,884 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:18:46,275 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:18:47,112 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:18:51,703 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:18:52,484 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:18:52,945 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:18:56,704 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:18:57,500 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:19:01,919 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:19:02,655 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:19:03,001 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:19:03,765 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:19:07,767 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:19:09,370 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:19:13,060 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:19:13,562 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:19:14,371 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:19:18,563 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:19:19,373 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:19:23,563 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:19:24,374 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:19:28,564 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:19:29,706 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:19:33,565 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:19:33,766 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:19:34,707 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:19:38,767 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:19:39,708 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:19:43,768 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:19:45,105 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:19:45,696 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:19:49,165 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:19:50,106 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:19:54,166 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:19:54,765 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:19:55,954 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:19:59,261 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:20:00,956 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:20:03,766 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:20:04,833 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:20:04,955 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:20:06,698 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:20:06,699 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:20:09,956 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:20:12,819 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:20:14,903 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:20:15,010 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:20:17,820 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:20:20,011 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:20:23,535 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:20:25,775 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:20:26,986 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:20:28,537 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:20:30,776 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:20:33,768 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:20:34,355 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:20:34,355 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:20:34,639 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 11:20:34,640 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 11:20:34,641 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 11:20:34,650 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 11:20:35,036 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 11:20:36,040 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:20:36,652 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:20:37,045 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:20:39,356 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:20:41,654 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:20:45,188 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:20:47,108 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:20:47,633 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:20:50,190 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:20:52,634 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:20:55,166 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:20:55,891 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:20:58,161 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:21:00,893 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:21:03,162 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:21:03,768 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:21:05,233 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:21:06,359 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:21:08,973 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:21:12,154 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:21:14,595 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:21:15,298 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:21:17,155 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:21:19,596 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:21:22,832 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:21:22,833 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:21:25,084 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:21:25,354 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:21:27,834 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:21:30,085 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:21:33,355 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:21:33,355 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:21:33,769 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:21:35,419 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:21:35,771 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:21:38,732 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:21:40,773 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:21:44,445 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:21:45,492 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:21:45,861 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:21:49,446 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:21:50,862 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:21:54,880 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:21:55,552 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:21:56,107 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:22:01,108 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:22:01,314 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:22:03,769 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:22:05,626 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:22:06,315 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:22:06,770 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:22:11,772 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:22:11,924 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:22:15,690 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:22:16,925 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:22:17,274 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:22:22,275 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:22:22,673 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:22:22,673 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:22:25,756 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:22:27,674 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:22:27,969 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:22:33,345 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:22:33,525 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:22:33,770 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:22:33,812 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:22:38,346 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:22:38,771 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:22:43,876 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:22:44,068 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:22:44,283 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:22:49,070 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:22:49,284 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:22:52,943 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:22:54,352 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:22:54,833 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:22:59,354 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:23:00,848 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:23:03,019 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:23:03,771 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:23:04,772 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:23:05,849 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:23:09,773 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:23:11,546 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:23:13,079 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:23:14,931 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:23:16,550 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:23:19,933 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:23:22,464 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:23:22,464 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:23:23,140 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:23:25,165 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:23:27,466 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:23:30,167 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:23:32,739 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:23:33,208 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:23:33,771 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:23:35,772 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:23:37,740 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:23:40,773 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:23:43,280 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:23:43,300 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:23:46,601 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:23:48,301 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:23:51,568 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 11:23:51,569 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 11:23:51,570 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 11:23:51,573 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 11:23:52,338 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:23:52,339 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 11:23:52,574 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:23:53,340 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:23:53,340 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:23:53,343 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:23:57,575 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:23:59,781 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:24:03,052 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:24:03,410 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:24:03,771 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:24:04,782 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:24:08,773 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:24:09,783 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:24:13,774 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:24:14,784 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:24:18,774 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:24:20,131 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:24:20,132 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:24:21,526 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:24:24,590 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:24:25,133 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:24:29,604 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:24:30,141 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:24:31,585 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:24:33,772 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:24:34,773 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:24:35,141 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:24:39,774 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:24:40,917 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:24:41,653 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:24:45,086 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:24:45,918 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:24:50,761 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:24:51,132 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:24:51,724 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:24:55,762 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:24:58,097 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:25:01,488 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:25:01,784 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:25:03,098 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:25:03,772 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:25:06,774 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:25:08,644 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:25:08,644 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:25:09,841 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:25:11,892 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:25:13,645 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:25:16,894 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:25:19,131 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:25:19,132 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:25:19,907 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:25:22,583 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:25:24,223 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:25:27,584 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:25:29,940 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:25:29,974 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:25:33,198 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:25:33,773 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:25:34,942 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:25:39,099 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:25:40,045 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:25:40,851 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:25:44,100 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:25:46,456 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:25:49,826 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:25:50,109 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:25:51,457 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:25:54,827 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:25:56,601 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:25:59,164 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:25:59,847 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:26:01,615 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:26:03,773 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:26:05,774 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:26:06,978 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:26:06,979 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:26:09,234 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:26:11,385 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:26:11,979 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:26:16,387 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:26:17,286 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:26:17,581 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:26:21,842 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:26:22,582 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:26:27,346 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:26:27,510 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:26:27,914 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:26:32,511 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:26:32,915 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:26:33,775 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:26:37,420 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:26:37,652 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:26:38,367 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:26:42,653 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:26:43,369 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:26:47,486 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:26:48,362 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:26:49,064 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:26:53,363 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:26:54,936 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:26:54,937 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:26:57,545 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:26:59,115 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:26:59,938 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:27:03,777 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:27:04,778 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:27:05,332 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:27:07,622 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:27:10,333 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:27:10,701 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:27:15,334 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:27:15,702 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:27:20,368 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:27:20,368 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:27:20,403 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 11:27:20,405 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 11:27:20,405 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 11:27:20,408 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 11:27:20,708 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 11:27:21,409 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:27:21,712 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:27:22,717 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:27:25,582 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:27:26,410 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:27:31,228 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:27:31,558 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:27:31,783 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:27:33,776 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:27:36,229 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:27:36,777 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:27:41,843 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:27:42,024 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:27:42,230 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:27:47,231 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:27:48,472 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:27:51,916 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:27:52,850 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:27:53,473 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:27:57,851 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:27:59,286 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:28:01,980 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:28:03,465 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:28:03,777 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:28:04,286 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:28:08,779 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:28:09,751 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:28:09,751 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:28:10,038 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:28:14,021 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:28:14,752 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:28:19,897 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:28:20,102 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:28:20,368 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:28:24,897 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:28:25,531 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:28:30,166 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:28:30,473 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:28:31,280 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:28:33,777 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:28:35,779 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:28:37,088 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:28:37,088 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:28:40,247 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:28:41,330 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:28:42,089 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:28:46,332 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:28:47,630 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:28:50,306 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:28:52,018 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:28:52,631 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:28:57,020 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:28:57,750 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:29:00,376 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:29:02,089 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:29:02,751 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:29:03,778 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:29:07,722 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:29:08,337 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:29:09,442 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:29:12,723 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:29:13,338 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:29:18,341 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:29:19,156 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:29:19,506 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:29:23,342 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:29:25,574 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:29:25,574 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:29:29,318 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:29:29,568 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:29:30,575 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:29:33,778 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:29:34,779 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:29:35,891 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:29:37,628 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:29:40,148 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:29:40,892 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:29:45,150 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:29:46,116 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:29:49,724 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:29:50,544 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:29:51,117 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:29:55,546 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:29:56,863 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:29:57,776 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:30:01,123 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:30:01,864 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:30:03,781 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:30:06,644 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:30:07,339 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:30:07,847 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:30:11,645 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:30:12,359 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:30:17,033 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:30:17,789 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:30:17,924 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:30:22,034 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:30:22,791 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:30:27,103 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:30:27,999 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:30:28,014 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:30:32,104 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:30:33,781 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:30:34,608 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:30:36,042 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 11:30:36,044 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 11:30:36,044 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 11:30:36,046 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 11:30:36,057 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 11:30:37,062 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:30:38,048 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:30:38,124 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:30:39,609 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:30:43,049 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:30:44,751 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:30:48,200 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:30:49,003 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:30:49,752 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:30:54,005 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:30:55,230 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:30:55,230 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:30:56,331 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:30:59,399 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:31:00,231 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:31:03,782 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:31:04,783 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:31:05,579 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:31:06,398 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:31:09,892 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:31:10,580 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:31:15,268 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:31:16,052 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:31:16,466 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:31:20,269 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:31:22,541 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:31:22,541 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:31:22,541 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:31:25,714 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:31:25,855 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:31:27,542 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:31:30,857 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:31:32,572 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:31:33,782 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:31:35,787 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:31:36,850 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:31:37,574 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:31:41,852 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:31:43,134 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:31:45,859 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:31:47,515 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:31:48,135 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:31:52,517 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:31:53,541 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:31:53,915 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:31:58,059 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:31:58,736 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:32:03,476 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:32:03,783 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:32:03,983 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:32:04,218 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:32:08,785 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:32:10,770 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:32:10,770 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:32:14,055 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:32:14,055 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:32:15,771 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:32:19,057 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:32:21,189 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:32:22,110 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:32:24,364 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:32:26,190 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:32:29,365 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:32:31,996 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:32:32,180 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:32:33,783 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:32:34,784 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:32:36,997 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:32:39,786 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:32:42,252 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:32:42,396 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:32:45,692 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:32:48,807 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:32:51,009 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:32:52,315 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:32:53,808 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:32:56,010 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:32:59,131 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:33:00,371 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:33:01,286 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:33:03,784 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:33:04,131 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:33:06,785 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:33:09,605 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:33:10,436 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:33:11,848 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:33:14,606 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:33:16,851 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:33:19,715 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:33:20,502 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:33:21,879 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:33:24,717 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:33:26,881 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:33:30,013 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:33:30,559 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:33:32,145 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:33:33,784 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:33:36,509 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:33:37,672 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:33:40,630 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:33:41,511 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:33:42,673 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:33:46,827 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:33:47,019 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 11:33:47,020 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 11:33:47,021 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 11:33:47,022 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 11:33:47,676 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 11:33:48,042 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:33:48,681 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:33:50,691 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:33:51,828 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:33:53,044 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:33:56,828 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:33:58,045 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:34:01,830 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:34:03,046 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:34:03,784 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:34:06,831 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:34:08,786 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:34:11,832 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:34:13,786 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:34:16,833 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:34:18,787 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:34:21,834 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:34:23,788 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:34:26,835 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:34:28,789 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:34:31,835 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:34:33,786 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:34:34,786 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:34:36,836 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:34:39,787 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:34:41,837 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:34:44,788 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:34:46,838 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:34:49,788 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:34:51,839 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:34:54,789 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:34:56,840 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:34:59,790 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:35:01,841 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:35:03,786 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:35:05,786 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:35:06,841 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:35:10,787 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:35:11,843 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:35:15,788 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:35:16,843 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:35:20,790 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:35:21,850 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:35:25,791 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:35:26,860 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:35:30,792 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:35:31,860 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:35:33,797 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:35:35,798 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:35:36,861 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:35:40,799 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:35:41,862 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:35:45,799 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:35:46,863 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:35:50,800 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:35:51,864 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:35:55,801 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:35:56,864 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:36:00,801 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:36:01,865 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:36:03,786 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:36:06,787 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:36:06,866 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:36:11,788 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:36:11,867 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:36:16,789 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:36:16,868 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:36:21,790 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:36:21,869 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:36:26,790 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:36:26,870 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:36:31,791 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:36:31,870 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:36:33,790 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:36:36,871 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:36:37,791 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:36:41,872 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:36:42,792 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:36:46,873 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:36:47,793 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:36:51,873 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:36:52,793 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:36:56,874 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:36:57,794 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:37:01,880 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:37:02,795 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:37:03,789 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:37:06,881 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:37:08,790 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:37:11,882 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:37:13,791 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:37:16,883 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:37:18,792 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:37:21,884 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:37:23,793 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:37:26,885 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:37:28,793 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:37:31,885 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:37:33,790 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:37:34,791 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:37:36,886 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:37:39,792 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:37:41,887 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:37:44,793 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:37:46,888 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:37:49,794 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:37:51,888 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:37:54,794 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:37:56,889 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:37:59,795 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:38:01,890 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:38:03,791 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:38:05,792 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:38:06,891 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:38:10,792 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:38:11,892 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:38:15,793 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:38:16,892 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:38:20,793 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:38:21,893 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:38:25,794 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:38:26,894 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:38:30,795 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:38:31,895 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:38:33,791 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:38:36,793 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:38:36,895 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:38:41,794 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:38:41,896 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:38:46,794 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:38:46,897 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:38:51,795 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:38:51,898 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:38:56,796 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:38:56,899 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:39:01,796 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:39:01,899 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:39:03,792 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:39:06,900 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:39:07,793 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:39:11,901 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:39:12,793 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:39:16,902 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:39:17,794 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:39:21,903 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:39:22,795 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:39:26,903 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:39:27,796 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:39:31,904 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:39:32,797 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:39:33,792 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:39:36,905 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:39:38,793 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:39:41,906 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:39:43,794 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:39:46,907 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:39:48,795 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:39:51,907 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:39:53,795 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:39:56,908 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:39:58,796 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:40:01,909 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:40:03,794 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:40:04,795 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:40:06,910 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:40:09,796 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:40:11,911 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:40:14,797 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:40:16,911 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:40:19,797 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:40:21,912 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:40:24,798 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:40:26,913 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:40:29,799 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:40:31,914 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:40:33,796 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:40:34,804 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:40:36,915 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:40:39,805 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:40:41,915 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:40:44,805 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:40:46,916 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:40:49,806 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:40:51,917 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:40:54,807 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:40:56,918 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:40:59,807 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:41:01,919 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:41:03,796 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:41:05,797 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:41:06,919 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:41:10,798 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:41:11,920 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:41:15,799 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:41:16,921 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:41:20,799 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:41:21,922 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:41:25,800 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:41:26,923 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:41:30,801 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:41:31,923 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:41:33,797 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:41:36,798 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:41:36,924 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:41:41,799 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:41:41,925 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:41:46,799 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:41:46,926 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:41:51,800 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:41:51,927 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:41:56,801 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:41:56,927 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:42:01,801 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:42:01,928 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:42:03,798 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:42:06,929 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:42:07,799 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:42:11,930 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:42:12,800 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:42:16,931 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:42:17,801 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:42:21,932 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:42:22,801 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:42:26,932 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:42:27,802 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:42:31,933 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:42:32,803 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:42:33,798 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:42:36,934 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:42:38,799 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:42:41,935 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:42:43,800 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:42:46,936 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:42:48,801 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:42:51,937 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:42:53,802 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:42:56,937 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:42:58,802 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:43:01,938 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:43:03,798 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:43:04,799 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:43:06,939 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:43:09,800 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:43:11,940 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:43:14,800 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:43:16,941 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:43:19,801 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:43:21,941 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:43:24,802 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:43:26,942 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:43:29,803 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:43:31,943 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:43:33,799 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:43:35,800 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:43:36,944 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:43:40,800 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:43:41,945 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:43:45,801 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:43:46,946 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:43:47,686 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 11:43:47,689 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 11:43:47,689 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 11:43:47,692 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 11:43:48,410 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 11:43:49,415 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:43:51,694 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:43:51,947 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:43:56,695 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:43:56,948 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:44:01,696 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:44:01,948 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:44:03,799 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:44:06,801 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:44:06,949 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:44:11,801 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:44:11,950 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:44:16,577 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:44:16,876 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:44:16,994 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:44:21,877 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:44:22,464 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:44:26,878 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:44:27,807 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:44:31,879 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:44:33,096 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:44:33,800 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:44:37,801 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:44:38,514 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:44:42,802 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:44:44,059 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:44:47,803 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:44:49,113 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:44:52,804 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:44:54,854 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:44:57,805 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:45:02,136 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:45:02,933 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:45:03,195 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:45:03,800 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:45:03,940 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:45:08,701 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:45:08,729 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:45:08,737 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:45:08,738 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:45:09,970 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:45:13,746 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:45:18,747 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:45:20,069 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:45:23,747 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:45:28,748 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:45:33,749 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:45:33,801 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:45:36,754 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:45:36,755 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:45:38,106 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:45:38,768 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:45:41,755 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:45:43,769 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:45:46,756 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:45:48,769 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:45:51,757 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:45:53,770 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:45:56,757 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:45:58,771 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:46:01,758 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:46:03,771 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:46:03,801 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:46:06,759 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:46:08,802 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:46:11,760 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:46:13,803 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:46:16,761 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:46:18,804 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:46:21,762 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:46:23,804 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:46:26,763 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:46:28,805 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:46:31,763 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:46:33,802 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:46:34,802 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:46:35,324 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:46:36,764 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:46:39,803 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:46:44,380 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:46:44,805 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:46:47,391 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:46:49,381 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:46:50,038 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:46:55,040 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:46:55,565 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:46:59,456 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:47:00,160 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:47:00,567 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:47:03,802 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:47:05,804 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:47:06,087 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:47:09,534 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:47:11,088 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:47:11,460 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:47:16,462 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:47:17,064 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:47:19,599 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:47:22,065 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:47:22,274 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:47:27,275 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:47:27,663 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:47:29,659 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:47:32,664 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:47:33,076 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:47:33,803 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:47:38,163 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:47:38,164 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:47:38,597 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:47:38,720 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:47:43,164 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:47:43,598 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:47:48,783 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:47:49,103 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:47:49,304 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:47:54,104 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:47:54,305 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:47:58,842 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:48:00,079 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:48:00,299 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:48:03,804 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:48:05,081 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:48:05,806 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:48:08,903 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:48:10,164 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:48:11,445 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:48:16,447 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:48:16,807 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:48:18,973 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:48:21,814 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:48:22,034 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:48:27,035 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:48:27,138 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:48:27,138 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:48:29,034 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:48:32,139 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:48:32,429 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:48:33,804 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:48:37,164 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:48:37,806 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:48:39,101 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:48:43,127 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:48:43,548 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:48:49,052 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:48:49,173 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:48:49,424 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:48:54,053 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:48:54,425 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:48:59,235 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:48:59,878 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:49:00,158 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:49:03,805 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:49:04,862 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:49:05,806 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:49:09,298 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:49:10,164 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:49:11,798 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:49:16,026 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:49:16,799 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:49:19,366 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:49:21,027 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:49:22,572 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:49:26,551 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:49:26,552 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:49:27,574 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:49:29,426 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:49:31,553 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:49:32,996 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:49:33,806 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:49:37,164 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:49:38,740 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:49:39,498 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:49:42,494 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:49:43,741 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:49:48,246 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:49:48,913 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:49:49,566 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:49:53,247 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:49:53,913 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:49:57,949 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 11:49:57,952 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 11:49:57,956 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 11:49:57,959 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 11:49:58,528 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:49:58,633 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 11:49:58,961 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:49:59,640 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:50:03,529 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:50:03,806 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:50:04,807 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:50:09,165 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:50:09,713 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:50:10,775 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:50:14,494 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:50:15,777 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:50:19,787 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:50:20,326 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:50:21,727 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:50:25,336 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:50:26,729 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:50:29,861 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:50:30,387 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:50:31,742 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:50:33,807 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:50:35,388 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:50:36,808 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:50:38,919 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:50:41,046 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:50:42,377 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:50:46,898 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:50:47,378 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:50:48,996 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:50:51,899 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:50:53,190 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:50:57,532 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:50:58,750 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:50:59,055 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:51:02,533 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:51:03,751 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:51:03,807 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:51:08,123 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:51:09,131 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:51:09,478 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:51:13,124 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:51:14,479 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:51:18,315 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:51:19,189 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:51:19,729 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:51:23,378 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:51:24,730 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:51:28,643 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:51:29,247 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:51:29,800 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:51:33,644 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:51:33,807 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:51:34,808 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:51:39,131 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:51:39,315 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:51:40,535 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:51:44,133 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:51:45,536 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:51:49,391 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:51:49,929 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:51:51,130 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:51:55,641 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:51:55,642 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:51:56,132 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:51:59,453 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:52:00,642 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:52:01,218 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:52:03,808 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:52:05,643 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:52:06,809 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:52:10,704 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:52:11,810 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:52:15,705 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:52:16,811 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:52:21,384 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:52:21,987 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:52:23,579 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:52:26,385 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:52:26,988 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:52:31,787 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:52:32,114 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:52:33,636 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:52:33,808 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:52:36,835 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:52:37,810 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:52:42,451 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:52:42,811 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:52:45,817 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:52:47,459 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:52:47,913 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:52:52,915 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:52:53,362 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:52:53,362 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:52:55,879 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:52:58,363 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:52:58,616 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:53:03,397 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:53:03,617 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:53:03,809 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:53:05,944 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:53:08,398 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:53:08,810 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:53:13,548 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:53:13,815 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:53:16,012 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:53:18,549 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:53:18,995 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:53:24,235 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:53:24,562 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:53:26,070 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:53:29,236 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:53:29,570 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:53:33,199 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 11:53:33,200 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 11:53:33,201 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 11:53:33,203 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 11:53:33,809 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:53:34,133 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:53:34,133 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 11:53:34,810 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:53:34,831 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:53:35,138 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:53:39,812 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:53:39,893 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:53:45,158 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:53:45,239 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:53:45,779 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:53:50,160 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:53:51,665 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:53:55,302 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:53:55,919 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:53:56,667 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:54:00,920 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:54:02,231 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:54:03,453 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:54:03,810 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:54:06,811 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:54:07,232 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:54:11,812 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:54:12,539 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:54:15,532 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:54:17,075 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:54:17,540 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:54:22,077 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:54:22,826 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:54:25,620 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:54:27,174 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:54:27,827 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:54:32,175 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:54:32,908 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:54:33,810 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:54:35,677 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:54:37,811 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:54:37,910 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:54:42,813 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:54:43,719 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:54:45,733 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:54:48,124 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:54:48,719 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:54:53,125 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:54:53,720 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:54:58,126 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:54:59,112 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:54:59,818 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:55:03,325 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:55:03,811 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:55:04,113 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:55:08,910 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:55:09,363 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:55:09,889 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:55:13,911 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:55:14,653 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:55:19,424 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:55:19,832 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:55:19,951 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:55:24,425 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:55:25,837 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:55:25,837 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:55:29,993 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:55:30,055 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:55:30,838 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:55:33,813 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:55:35,816 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:55:36,379 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:55:38,109 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:55:41,380 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:55:41,551 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:55:46,875 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:55:47,279 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:55:48,176 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:55:51,876 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:55:52,279 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:55:57,079 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:55:57,080 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:55:57,408 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:55:58,245 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:56:02,081 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:56:02,409 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:56:03,815 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:56:07,307 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:56:07,363 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:56:08,145 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:56:12,796 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:56:13,146 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:56:17,371 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:56:18,529 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:56:18,719 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:56:23,720 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:56:23,866 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:56:23,866 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:56:27,438 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:56:28,867 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:56:29,320 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:56:33,815 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:56:34,712 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:56:34,817 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:56:37,502 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:56:39,712 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:56:40,345 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:56:45,204 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:56:45,347 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:56:47,591 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:56:50,207 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:56:50,559 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:56:55,211 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 11:56:55,212 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 11:56:55,213 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 11:56:55,215 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 11:56:55,646 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 11:56:55,783 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:56:56,217 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:56:56,650 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:56:57,655 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:57:00,784 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:57:01,217 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:57:03,818 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:57:05,717 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:57:06,575 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:57:06,760 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:57:11,761 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:57:13,153 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:57:13,153 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:57:15,778 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:57:17,305 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:57:18,157 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:57:22,307 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:57:23,589 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:57:25,848 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:57:27,979 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:57:28,590 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:57:32,981 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:57:33,817 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:57:34,356 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:57:35,902 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:57:38,819 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:57:39,356 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:57:43,821 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:57:44,705 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:57:45,977 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:57:49,052 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:57:49,706 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:57:54,867 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:57:55,031 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:57:56,069 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:58:00,032 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:58:00,591 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:58:03,829 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:58:05,816 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:58:06,030 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:58:06,160 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:58:10,816 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:58:11,031 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:58:16,033 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:58:16,673 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:58:18,234 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:58:21,034 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:58:21,675 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:58:26,295 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:58:26,615 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:58:26,868 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:58:31,616 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:58:32,988 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:58:33,818 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:58:36,365 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:58:37,138 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:58:37,996 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:58:42,139 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:58:43,991 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:58:46,430 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:58:47,142 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:58:48,992 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:58:52,143 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:58:54,252 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:58:55,496 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:58:57,555 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:58:59,253 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:59:02,557 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:59:03,819 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:59:05,054 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:59:05,556 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:59:07,820 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:59:10,055 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:59:12,842 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:59:13,613 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:59:15,651 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:59:17,843 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:59:21,139 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:59:23,524 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:59:23,671 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:59:26,140 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:59:28,525 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:59:31,236 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:59:33,732 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:59:33,820 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 11:59:33,821 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:59:36,260 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:59:38,822 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:59:41,954 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:59:41,954 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:59:43,794 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:59:44,452 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:59:46,955 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:59:49,453 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:59:52,846 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 11:59:54,025 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 11:59:55,050 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 11:59:57,846 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:00:00,051 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:00:03,330 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:00:03,820 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:00:04,087 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:00:05,822 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:00:08,332 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:00:10,823 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:00:11,343 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 12:00:11,344 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 12:00:11,345 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 12:00:11,348 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 12:00:12,154 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:00:12,154 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 12:00:13,994 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:00:13,994 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:00:14,163 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:00:16,351 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:00:19,978 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:00:22,349 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:00:24,236 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:00:24,988 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:00:27,350 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:00:30,061 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:00:32,460 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:00:33,821 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:00:34,307 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:00:35,062 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:00:37,822 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:00:40,716 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:00:43,751 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:00:44,365 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:00:45,717 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:00:48,752 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:00:51,141 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:00:54,430 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:00:54,467 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:00:56,181 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:00:59,470 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:01:01,813 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:01:02,492 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:01:03,821 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:01:04,823 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:01:06,814 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:01:09,824 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:01:11,994 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:01:14,562 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:01:15,556 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:01:17,708 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:01:20,558 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:01:23,117 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:01:24,623 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:01:26,484 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:01:28,119 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:01:31,668 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:01:32,072 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:01:33,808 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:01:33,822 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:01:37,823 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:01:39,570 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:01:41,736 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:01:43,255 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:01:44,571 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:01:48,257 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:01:49,712 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:01:51,807 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:01:53,890 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:01:54,713 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:01:58,891 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:02:00,404 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:02:01,874 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:02:03,823 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:02:04,824 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:02:05,405 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:02:09,825 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:02:10,857 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:02:11,952 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:02:15,071 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:02:15,858 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:02:20,549 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:02:21,141 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:02:22,017 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:02:25,550 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:02:26,288 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:02:30,080 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:02:31,079 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:02:31,920 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:02:33,824 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:02:36,826 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:02:36,922 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:02:41,998 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:02:42,146 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:02:42,707 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:02:47,708 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:02:48,276 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:02:52,266 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:02:53,277 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:02:53,476 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:02:58,477 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:02:58,788 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:02:58,789 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:03:00,313 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:03:03,789 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:03:03,824 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:03:03,825 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:03:09,473 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:03:09,788 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:03:10,382 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:03:14,474 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:03:14,789 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:03:19,983 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:03:20,365 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:03:20,445 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:03:24,985 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:03:25,367 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:03:27,900 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 12:03:27,901 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 12:03:27,902 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 12:03:27,904 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 12:03:28,512 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:03:28,513 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 12:03:30,524 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:03:30,658 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:03:30,906 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:03:33,825 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:03:36,826 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:03:36,918 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:03:40,590 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:03:41,919 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:03:42,240 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:03:47,241 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:03:47,770 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:03:50,653 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:03:52,771 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:03:53,134 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:03:58,317 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:03:58,317 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:03:58,837 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:04:00,718 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:04:03,318 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:04:03,825 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:04:04,826 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:04:09,232 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:04:10,553 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:04:10,806 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:04:14,234 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:04:15,554 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:04:17,856 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:04:20,022 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:04:21,211 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:04:25,996 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:04:26,213 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:04:27,924 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:04:30,997 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:04:31,361 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:04:33,844 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:04:36,620 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:04:36,847 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:04:38,003 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:04:41,621 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:04:41,871 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:04:47,078 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:04:47,307 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:04:48,071 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:04:52,079 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:04:52,308 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:04:57,082 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:04:57,309 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:05:02,083 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:05:02,310 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:05:03,844 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:05:07,474 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:05:07,845 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:05:12,734 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:05:12,845 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:05:17,847 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:05:18,401 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:05:22,250 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:05:23,402 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:05:23,444 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:05:28,443 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:05:29,793 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:05:29,793 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:05:32,318 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:05:33,844 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:05:33,845 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:05:34,795 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:05:38,846 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:05:39,889 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:05:42,397 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:05:44,206 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:05:44,890 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:05:49,208 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:05:50,775 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:05:52,461 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:05:55,004 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:05:55,775 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:06:00,005 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:06:01,324 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:06:02,531 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:06:03,845 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:06:05,846 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:06:06,325 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:06:10,847 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:06:11,699 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:06:12,593 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:06:15,885 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:06:16,700 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:06:21,414 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:06:22,167 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:06:22,656 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:06:26,415 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:06:27,169 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:06:31,640 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:06:32,716 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:06:33,063 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:06:33,845 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:06:36,846 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:06:39,281 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:06:42,821 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:06:42,824 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:06:44,283 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:06:47,822 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:06:49,698 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:06:52,898 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:06:52,964 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:06:54,699 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:06:57,966 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:07:00,697 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:07:00,698 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:07:02,961 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:07:03,001 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:07:03,846 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:07:05,698 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:07:08,847 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:07:10,627 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 12:07:10,629 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 12:07:10,630 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 12:07:10,632 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 12:07:11,016 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 12:07:11,343 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:07:13,026 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:07:14,634 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:07:16,428 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:07:19,635 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:07:22,083 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:07:22,099 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:07:25,059 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:07:27,100 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:07:30,060 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:07:32,879 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:07:33,846 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:07:34,163 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:07:35,847 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:07:37,880 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:07:40,849 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:07:42,214 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:07:43,344 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:07:46,797 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:07:48,559 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:07:52,280 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:07:52,297 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:07:54,165 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:07:57,297 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:07:59,853 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:07:59,853 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:08:02,344 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:08:03,020 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:08:03,847 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:08:04,855 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:08:08,848 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:08:10,490 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:08:12,401 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:08:14,753 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:08:15,491 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:08:19,760 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:08:20,974 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:08:22,469 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:08:25,709 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:08:25,975 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:08:30,985 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:08:30,985 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:08:31,270 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:08:32,522 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:08:33,847 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:08:35,985 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:08:36,848 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:08:40,587 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:08:41,344 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:08:42,655 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:08:47,656 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:08:47,707 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:08:50,660 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:08:52,708 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:08:52,872 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:08:57,873 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:08:58,079 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:08:58,079 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:09:00,724 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:09:03,080 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:09:03,351 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:09:03,848 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:09:08,420 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:09:09,269 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:09:10,795 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:09:14,007 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:09:14,269 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:09:18,854 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:09:19,384 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:09:19,533 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:09:24,535 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:09:25,645 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:09:28,921 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:09:29,865 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:09:30,646 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:09:33,850 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:09:35,852 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:09:36,027 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:09:38,981 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:09:41,028 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:09:41,251 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:09:46,029 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:09:46,252 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:09:51,132 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:09:51,566 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:09:53,068 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:09:56,132 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:09:56,567 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:10:01,948 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:10:02,249 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:10:03,159 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:10:03,850 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:10:06,950 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:10:07,852 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:10:11,218 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:10:12,079 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:10:12,919 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:10:17,920 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:10:18,075 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:10:20,280 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:10:23,076 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:10:23,728 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:10:28,191 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:10:28,730 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:10:29,390 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 12:10:29,391 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 12:10:29,392 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 12:10:29,394 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 12:10:30,347 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:10:30,348 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 12:10:31,353 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:10:33,193 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:10:33,850 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:10:33,851 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:10:38,624 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:10:38,852 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:10:40,413 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:10:43,625 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:10:43,872 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:10:49,186 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:10:49,422 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:10:50,476 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:10:54,187 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:10:54,423 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:10:58,527 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:10:59,192 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:10:59,975 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:11:03,851 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:11:05,852 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:11:06,095 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:11:08,595 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:11:11,097 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:11:11,290 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:11:16,291 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:11:16,736 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:11:16,737 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:11:18,664 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:11:21,738 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:11:21,795 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:11:27,259 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:11:27,608 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:11:28,729 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:11:32,260 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:11:32,609 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:11:33,852 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:11:36,785 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:11:37,870 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:11:38,138 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:11:42,871 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:11:43,139 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:11:46,865 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:11:48,522 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:11:48,522 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:11:48,795 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:11:53,796 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:11:54,027 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:11:56,930 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:11:59,028 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:11:59,256 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:12:03,854 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:12:04,665 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:12:04,794 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:12:06,993 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:12:09,666 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:12:09,795 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:12:14,984 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:12:15,249 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:12:17,062 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:12:19,985 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:12:20,249 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:12:25,119 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:12:25,259 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:12:25,822 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:12:30,823 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:12:31,922 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:12:31,923 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:12:33,853 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:12:35,190 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:12:35,854 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:12:36,924 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:12:40,856 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:12:42,513 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:12:45,250 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:12:46,826 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:12:47,513 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:12:51,827 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:12:53,057 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:12:53,302 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:12:57,250 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:12:58,058 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:13:02,814 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:13:03,363 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:13:03,650 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:13:03,854 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:13:07,855 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:13:09,966 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:13:13,207 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:13:13,429 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:13:14,967 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:13:18,208 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:13:20,306 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:13:20,307 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:13:22,572 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:13:23,509 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:13:25,307 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:13:28,511 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:13:30,650 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:13:30,782 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:13:33,854 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:13:33,856 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:13:35,695 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:13:38,857 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:13:39,324 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 12:13:39,325 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 12:13:39,325 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 12:13:39,328 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 12:13:39,842 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 12:13:40,848 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:13:41,105 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:13:41,851 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:13:44,330 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:13:47,360 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:13:47,360 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:13:49,463 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:13:50,909 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:13:52,361 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:13:54,464 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:13:57,833 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:13:58,971 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:13:59,980 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:14:02,834 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:14:03,855 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:14:05,856 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:14:08,145 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:14:09,032 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:14:11,381 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:14:13,146 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:14:16,382 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:14:18,563 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:14:19,093 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:14:21,710 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:14:24,833 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:14:27,093 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:14:27,154 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:14:29,834 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:14:32,094 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:14:33,857 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:14:35,570 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:14:35,571 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:14:37,216 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:14:37,735 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:14:40,572 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:14:42,736 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:14:45,651 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:14:47,274 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:14:48,026 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:14:50,838 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:14:53,028 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:14:55,897 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:14:57,338 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:14:58,147 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:15:00,910 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:15:03,149 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:15:03,858 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:15:06,277 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:15:07,410 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:15:08,682 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:15:12,793 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:15:13,690 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:15:15,455 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:15:17,793 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:15:19,008 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:15:23,336 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:15:24,565 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:15:25,526 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:15:28,337 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:15:29,566 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:15:33,722 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:15:33,722 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:15:33,858 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:15:34,858 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:15:35,582 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:15:38,722 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:15:39,860 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:15:42,633 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:15:44,092 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:15:45,309 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:15:49,094 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:15:50,310 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:15:54,705 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:15:54,942 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:15:55,533 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:16:00,535 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:16:01,030 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:16:02,764 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:16:03,858 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:16:05,859 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:16:06,031 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:16:10,861 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:16:11,513 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:16:12,837 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:16:16,514 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:16:16,770 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:16:22,070 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:16:22,261 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:16:22,904 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:16:27,071 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:16:27,262 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:16:32,387 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:16:32,722 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:16:32,981 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:16:33,859 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:16:37,860 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:16:38,007 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:16:43,054 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:16:43,126 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:16:43,919 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:16:48,127 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:16:49,230 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:16:49,231 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:16:50,362 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 12:16:50,363 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 12:16:50,364 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 12:16:50,366 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 12:16:51,110 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:16:51,111 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 12:16:52,117 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:16:53,368 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:16:54,232 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:16:58,370 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:16:59,829 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:17:01,173 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:17:03,862 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:17:03,863 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:17:04,830 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:17:08,865 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:17:10,565 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:17:11,234 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:17:14,713 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:17:15,566 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:17:20,641 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:17:20,641 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:17:20,711 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:17:21,299 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:17:25,713 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:17:27,051 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:17:29,360 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:17:31,386 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:17:32,053 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:17:33,870 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:17:36,871 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:17:37,892 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:17:39,426 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:17:42,126 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:17:42,893 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:17:47,127 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:17:48,513 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:17:49,506 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:17:52,711 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:17:53,514 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:17:58,028 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:17:58,829 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:17:59,573 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:18:03,029 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:18:03,861 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:18:04,035 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:18:08,979 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:18:09,636 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:18:09,736 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:18:13,980 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:18:16,123 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:18:19,333 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:18:19,696 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:18:21,170 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:18:24,334 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:18:26,900 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:18:29,762 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:18:30,102 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:18:31,901 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:18:33,862 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:18:35,865 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:18:36,810 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:18:37,436 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:18:37,437 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:18:41,690 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:18:42,437 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:18:46,869 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:18:47,223 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:18:47,641 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:18:52,224 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:18:54,420 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:18:56,975 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:18:57,655 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:18:59,421 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:19:02,656 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:19:03,864 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:19:05,034 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:19:05,034 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:19:07,307 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:19:08,197 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:19:10,035 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:19:13,198 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:19:15,407 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:19:17,362 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:19:18,825 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:19:20,408 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:19:23,828 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:19:25,420 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:19:25,460 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:19:29,615 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:19:30,460 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:19:33,864 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:19:34,687 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:19:35,487 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:19:36,034 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:19:39,688 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:19:42,841 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:19:45,056 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:19:45,554 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:19:47,842 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:19:50,056 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:19:53,267 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:19:53,268 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:19:55,504 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:19:55,625 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:19:58,268 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:20:00,506 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:20:02,994 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 12:20:02,995 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 12:20:02,996 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 12:20:02,998 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 12:20:03,682 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:20:03,683 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 12:20:03,741 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:20:03,865 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:20:05,690 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:20:05,865 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:20:08,742 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:20:10,867 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:20:13,797 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:20:15,753 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:20:16,017 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:20:18,798 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:20:21,019 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:20:23,813 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:20:24,252 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:20:26,462 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:20:30,672 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:20:31,463 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:20:33,865 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:20:33,878 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:20:35,673 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:20:36,867 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:20:41,153 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:20:42,359 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:20:43,943 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:20:46,154 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:20:47,360 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:20:50,996 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:20:51,569 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:20:51,570 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:20:52,763 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:20:56,570 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:20:57,764 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:21:01,067 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:21:02,398 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:21:03,632 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:21:03,868 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:21:08,678 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:21:08,678 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:21:08,870 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:21:11,128 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:21:13,679 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:21:14,862 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:21:19,094 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:21:20,384 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:21:21,201 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:21:24,095 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:21:25,385 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:21:29,253 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:21:29,643 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:21:30,889 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:21:33,866 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:21:34,644 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:21:36,868 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:21:39,314 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:21:40,058 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:21:42,278 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:21:46,364 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:21:46,364 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:21:47,279 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:21:49,380 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:21:51,365 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:21:52,571 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:21:57,151 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:21:58,542 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:21:59,455 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:22:02,152 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:22:03,543 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:22:03,869 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:22:07,511 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:22:07,700 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:22:07,700 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:22:08,957 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:22:12,701 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:22:13,958 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:22:17,581 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:22:18,348 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:22:19,646 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:22:24,648 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:22:24,747 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:22:24,748 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:22:27,649 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:22:29,749 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:22:29,861 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:22:33,870 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:22:34,872 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:22:35,098 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:22:37,719 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:22:40,099 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:22:40,349 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:22:45,527 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:22:45,753 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:22:45,775 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:22:50,528 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:22:50,753 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:22:55,839 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:22:56,278 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:22:56,395 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:23:01,396 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:23:02,596 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:23:02,597 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:23:03,870 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:23:05,912 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:23:06,950 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:23:07,598 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:23:11,951 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:23:12,998 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:23:13,173 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 12:23:13,175 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 12:23:13,175 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 12:23:13,178 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 12:23:13,971 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:23:13,972 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 12:23:15,982 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:23:17,180 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:23:17,999 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:23:22,181 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:23:23,492 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:23:23,493 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:23:24,027 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:23:27,661 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:23:28,493 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:23:33,240 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:23:33,870 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:23:33,948 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:23:34,103 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:23:38,872 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:23:38,949 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:23:44,165 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:23:44,533 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:23:44,825 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:23:49,534 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:23:49,825 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:23:54,827 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:23:55,025 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:23:55,026 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:23:57,244 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:24:00,026 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:24:00,136 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:24:03,873 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:24:05,401 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:24:05,832 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:24:07,308 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:24:10,402 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:24:10,833 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:24:15,363 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:24:15,876 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:24:16,087 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:24:20,884 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:24:21,088 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:24:25,432 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:24:26,644 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:24:26,761 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:24:31,763 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:24:33,198 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:24:33,873 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:24:35,505 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:24:37,485 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:24:38,199 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:24:42,486 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:24:43,542 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:24:45,576 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:24:47,733 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:24:48,543 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:24:52,735 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:24:53,625 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:24:53,807 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:24:57,938 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:24:58,808 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:25:03,339 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:25:03,690 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:25:03,874 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:25:04,117 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:25:08,875 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:25:09,119 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:25:13,752 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:25:14,194 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:25:14,965 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:25:19,196 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:25:20,957 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:25:23,820 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:25:25,190 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:25:25,958 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:25:30,192 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:25:31,384 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:25:33,874 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:25:33,879 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:25:35,875 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:25:36,385 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:25:40,877 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:25:41,922 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:25:41,936 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:25:46,169 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:25:46,923 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:25:51,581 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:25:51,997 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:25:52,323 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:25:56,582 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:25:57,325 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:26:02,081 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:26:02,189 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:26:03,019 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:26:03,875 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:26:07,877 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:26:08,274 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:26:08,274 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:26:12,147 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:26:13,275 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:26:13,639 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:26:18,640 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:26:18,764 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:26:20,203 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:26:23,765 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:26:23,974 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:26:28,197 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 12:26:28,198 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 12:26:28,199 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 12:26:28,201 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 12:26:28,260 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 12:26:29,038 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:26:29,208 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:26:30,270 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:26:33,875 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:26:34,039 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:26:34,877 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:26:39,274 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:26:40,333 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:26:40,562 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:26:45,563 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:26:45,743 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:26:47,390 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:26:50,804 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:26:51,123 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:26:56,124 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:26:56,695 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:26:56,696 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:26:57,457 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:27:01,698 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:27:01,815 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:27:03,929 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:27:07,011 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:27:07,258 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:27:07,526 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:27:12,012 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:27:12,258 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:27:17,585 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:27:17,669 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:27:17,842 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:27:22,843 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:27:24,058 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:27:24,059 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:27:27,649 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:27:28,426 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:27:29,060 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:27:33,427 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:27:33,878 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:27:34,618 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:27:35,709 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:27:38,837 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:27:39,619 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:27:43,839 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:27:45,375 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:27:45,778 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:27:49,559 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:27:50,376 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:27:55,289 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:27:55,851 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:27:55,994 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:28:00,289 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:28:02,340 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:28:03,878 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:28:05,879 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:28:05,906 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:28:07,341 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:28:10,880 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:28:12,590 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:28:12,591 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:28:13,962 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:28:16,818 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:28:17,592 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:28:21,819 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:28:22,837 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:28:24,035 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:28:27,008 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:28:27,838 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:28:32,089 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:28:32,229 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:28:33,070 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:28:33,879 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:28:37,880 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:28:38,071 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:28:42,156 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:28:43,053 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:28:43,845 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:28:43,845 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:28:48,055 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:28:49,465 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:28:52,221 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:28:53,771 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:28:54,466 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:28:58,772 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:28:59,978 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:29:02,288 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:29:03,879 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:29:03,880 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:29:04,979 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:29:08,881 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:29:10,441 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:29:12,352 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:29:14,648 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:29:15,442 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:29:19,966 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:29:20,401 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:29:20,766 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:29:24,967 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:29:26,974 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:29:26,974 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:29:30,160 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:29:30,462 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:29:31,976 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:29:33,880 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:29:35,881 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:29:37,755 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:29:37,965 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 12:29:37,966 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 12:29:37,967 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 12:29:37,969 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 12:29:38,545 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 12:29:40,554 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:29:40,971 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:29:42,756 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:29:45,977 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:29:48,318 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:29:50,622 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:29:51,632 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:29:53,320 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:29:56,633 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:29:57,684 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:29:58,840 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:30:02,004 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:30:03,841 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:30:03,881 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:30:07,584 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:30:07,749 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:30:09,431 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:30:12,585 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:30:14,627 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:30:14,627 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:30:17,761 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:30:17,842 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:30:19,628 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:30:22,762 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:30:25,140 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:30:25,899 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:30:28,443 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:30:30,141 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:30:33,445 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:30:33,884 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:30:35,887 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:30:35,975 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:30:39,023 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:30:40,888 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:30:44,025 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:30:46,037 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:30:46,491 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:30:49,886 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:30:52,956 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:30:52,956 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:30:55,181 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:30:56,113 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:30:57,957 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:31:00,182 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:31:03,354 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:31:03,883 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:31:06,123 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:31:06,172 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:31:08,390 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:31:11,124 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:31:13,724 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:31:16,130 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:31:16,234 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:31:18,725 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:31:21,131 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:31:24,697 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:31:26,285 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:31:26,873 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:31:29,713 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:31:31,874 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:31:33,884 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:31:35,119 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:31:36,358 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:31:37,034 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:31:40,120 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:31:42,035 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:31:45,516 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:31:45,517 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:31:46,416 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:31:47,716 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:31:50,517 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:31:52,718 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:31:55,841 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:31:56,480 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:31:58,234 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:32:02,426 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:32:03,235 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:32:03,885 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:32:06,598 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:32:07,427 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:32:08,537 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:32:12,428 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:32:13,538 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:32:17,429 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:32:18,539 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:32:22,655 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:32:24,327 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:32:24,702 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:32:27,656 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:32:29,328 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:32:33,146 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:32:33,886 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:32:34,763 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:32:34,887 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:32:38,147 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:32:39,890 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:32:43,604 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:32:43,604 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:32:43,827 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:32:44,893 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:32:48,605 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:32:49,895 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:32:53,888 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:32:54,254 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:32:55,399 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:33:00,400 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:33:00,562 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:33:00,563 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:33:01,761 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 12:33:01,763 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 12:33:01,763 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 12:33:01,766 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 12:33:01,937 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 12:33:03,886 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:33:03,946 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:33:04,951 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:33:05,563 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:33:05,887 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:33:10,616 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:33:10,888 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:33:14,310 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:33:15,617 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:33:15,907 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:33:20,908 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:33:21,260 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:33:24,388 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:33:26,261 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:33:26,451 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:33:31,520 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:33:31,521 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:33:31,747 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:33:32,447 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:33:33,887 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:33:36,521 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:33:36,888 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:33:41,604 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:33:42,204 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:33:42,512 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:33:47,212 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:33:48,402 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:33:52,581 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:33:52,624 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:33:53,403 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:33:57,626 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:33:58,820 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:33:58,820 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:34:00,635 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:34:03,084 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:34:03,821 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:34:03,887 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:34:08,866 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:34:08,888 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:34:10,702 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:34:13,872 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:34:14,187 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:34:19,400 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:34:19,677 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:34:20,773 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:34:24,401 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:34:24,678 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:34:29,820 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:34:30,157 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:34:30,834 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:34:33,888 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:34:35,889 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:34:36,480 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:34:40,904 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:34:41,482 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:34:41,663 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:34:46,665 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:34:46,804 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:34:46,805 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:34:47,963 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:34:51,805 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:34:52,034 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:34:57,214 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:34:57,467 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:34:58,032 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:35:02,215 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:35:02,467 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:35:03,889 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:35:07,576 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:35:07,776 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:35:08,099 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:35:12,577 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:35:12,777 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:35:18,064 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:35:18,167 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:35:18,359 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:35:23,065 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:35:23,360 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:35:28,065 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:35:28,360 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:35:33,066 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:35:33,361 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:35:33,889 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:35:38,068 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:35:38,890 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:35:43,069 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:35:43,891 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:35:48,069 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:35:48,892 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:35:53,070 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:35:53,893 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:35:58,413 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:35:58,428 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:35:59,205 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:36:03,429 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:36:03,891 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:36:04,893 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:36:08,472 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:36:08,838 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:36:10,268 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:36:13,840 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:36:15,269 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:36:18,535 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:36:19,578 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:36:20,921 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:36:25,154 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:36:25,922 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:36:28,594 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:36:30,155 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:36:31,405 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:36:33,891 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:36:35,299 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:36:36,606 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:36:38,656 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:36:40,300 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:36:41,607 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:36:45,898 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:36:45,898 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:36:46,376 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 12:36:46,378 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 12:36:46,378 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 12:36:46,380 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 12:36:46,707 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 12:36:47,389 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:36:48,716 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:36:50,905 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:36:52,390 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:36:56,430 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:36:56,794 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:36:57,643 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:37:01,431 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:37:02,644 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:37:03,893 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:37:06,845 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:37:07,775 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:37:08,874 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:37:11,846 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:37:12,777 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:37:16,923 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:37:17,121 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:37:18,583 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:37:22,122 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:37:23,585 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:37:26,996 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:37:27,832 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:37:29,086 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:37:33,669 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:37:33,670 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:37:33,894 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:37:34,896 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:37:37,066 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:37:38,671 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:37:40,596 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:37:43,983 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:37:46,272 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:37:47,132 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:37:48,985 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:37:51,273 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:37:54,388 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:37:56,189 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:37:56,623 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:37:59,389 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:38:01,625 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:38:03,895 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:38:04,822 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:38:06,251 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:38:07,428 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:38:09,823 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:38:12,430 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:38:15,121 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:38:16,320 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:38:18,042 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:38:22,242 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:38:23,044 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:38:24,374 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:38:27,281 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:38:28,495 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:38:33,094 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:38:33,095 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:38:33,895 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:38:33,896 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:38:34,440 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:38:38,095 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:38:38,896 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:38:43,344 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:38:44,503 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:38:44,620 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:38:48,345 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:38:49,621 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:38:53,346 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:38:54,623 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:38:58,379 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:38:58,596 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:38:59,928 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:39:03,385 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:39:03,897 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:39:05,899 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:39:08,662 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:39:08,715 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:39:10,963 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:39:15,181 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:39:15,181 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:39:15,964 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:39:18,730 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:39:20,182 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:39:21,469 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:39:25,477 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:39:26,644 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:39:26,787 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:39:30,478 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:39:31,645 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:39:33,898 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:39:35,524 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:39:36,777 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:39:36,848 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:39:40,525 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:39:41,778 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:39:45,866 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:39:46,912 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:39:47,084 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:39:50,867 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:39:52,085 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:39:56,231 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:39:56,975 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:39:57,474 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:40:02,475 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:40:02,685 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:40:02,686 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:40:03,898 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:40:03,903 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 12:40:03,904 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 12:40:03,905 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 12:40:03,907 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 12:40:04,031 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 12:40:05,039 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:40:07,687 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:40:07,909 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:40:12,910 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:40:12,999 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:40:15,118 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:40:18,000 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:40:18,243 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:40:23,467 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:40:23,739 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:40:25,177 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:40:28,468 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:40:28,740 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:40:33,898 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:40:33,901 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:40:34,017 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:40:34,017 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:40:35,240 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:40:38,901 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:40:39,019 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:40:44,186 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:40:44,890 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:40:45,298 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:40:49,187 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:40:50,512 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:40:53,377 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:40:54,759 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:40:55,513 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:40:59,761 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:41:00,863 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:41:02,435 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:41:03,899 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:41:04,900 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:41:05,864 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:41:09,901 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:41:11,614 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:41:12,504 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:41:15,810 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:41:16,615 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:41:21,498 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:41:22,320 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:41:22,576 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:41:26,498 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:41:28,786 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:41:30,634 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:41:32,041 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:41:33,787 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:41:33,899 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:41:37,901 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:41:39,158 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:41:40,697 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:41:43,381 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:41:44,159 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:41:48,383 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:41:49,989 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:41:50,770 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:41:54,259 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:41:54,990 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:41:59,899 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:42:00,678 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:42:00,837 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:42:03,900 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:42:04,901 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:42:05,679 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:42:10,809 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:42:10,897 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:42:11,553 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:42:15,810 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:42:17,058 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:42:17,059 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:42:20,964 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:42:21,309 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:42:22,059 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:42:26,310 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:42:27,326 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:42:29,020 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:42:31,447 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:42:32,327 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:42:33,900 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:42:36,902 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:42:37,671 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:42:39,077 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:42:41,910 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:42:42,672 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:42:46,911 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:42:48,059 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:42:49,136 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:42:52,327 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:42:53,091 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:42:57,876 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:42:58,549 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:42:59,204 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:43:02,877 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:43:03,550 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:43:03,903 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:43:08,464 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:43:09,267 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:43:09,347 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:43:13,465 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:43:14,736 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:43:15,985 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 12:43:15,986 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 12:43:15,987 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 12:43:15,989 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 12:43:16,317 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 12:43:17,324 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:43:18,991 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:43:19,742 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:43:23,993 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:43:25,271 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:43:27,405 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:43:29,544 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:43:30,271 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:43:33,903 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:43:34,905 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:43:35,787 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:43:37,474 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:43:40,016 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:43:40,787 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:43:45,252 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:43:46,053 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:43:47,544 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:43:50,253 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:43:52,466 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:43:55,602 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:43:55,708 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:43:57,468 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:44:00,709 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:44:03,377 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:44:03,378 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:44:03,903 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:44:04,670 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:44:05,904 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:44:08,379 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:44:10,906 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:44:13,629 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:44:14,741 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:44:16,842 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:44:18,630 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:44:21,843 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:44:24,102 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:44:24,807 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:44:27,297 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:44:30,585 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:44:32,865 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:44:33,061 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:44:33,904 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:44:35,586 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:44:38,905 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:44:41,136 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:44:42,934 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:44:44,483 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:44:46,137 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:44:49,485 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:44:51,751 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:44:51,751 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:44:53,003 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:44:55,006 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:44:56,752 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:45:00,007 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:45:02,172 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:45:03,069 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:45:03,905 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:45:05,905 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:45:07,173 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:45:11,122 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:45:11,270 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:45:13,064 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:45:16,271 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:45:18,419 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:45:18,420 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:45:21,182 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:45:21,667 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:45:23,420 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:45:26,669 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:45:28,864 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:45:31,248 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:45:32,118 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:45:33,865 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:45:33,905 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:45:37,906 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:45:39,170 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:45:39,315 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:45:43,324 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:45:44,171 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:45:48,835 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:45:49,380 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:45:49,420 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:45:53,836 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:45:56,084 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:45:59,365 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:45:59,451 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:46:01,092 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:46:03,905 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:46:04,906 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:46:06,884 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:46:06,885 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:46:09,514 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:46:10,039 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:46:11,885 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:46:15,040 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:46:17,215 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:46:19,588 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:46:20,505 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:46:22,216 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:46:25,507 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:46:25,869 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 12:46:25,870 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 12:46:25,871 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 12:46:25,873 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 12:46:26,641 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 12:46:27,646 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:46:27,667 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:46:30,882 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:46:33,906 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:46:34,118 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:46:34,118 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:46:36,279 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:46:37,705 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:46:39,119 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:46:41,280 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:46:44,534 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:46:46,760 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:46:47,778 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:46:49,535 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:46:51,762 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:46:54,563 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:46:56,790 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:46:57,847 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:46:59,564 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:47:01,792 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:47:03,907 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:47:05,119 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:47:05,899 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:47:07,318 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:47:10,225 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:47:12,320 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:47:14,957 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:47:15,474 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:47:17,725 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:47:21,926 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:47:22,726 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:47:25,019 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:47:26,927 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:47:28,206 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:47:32,489 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:47:33,754 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:47:33,909 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:47:35,079 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:47:37,490 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:47:38,910 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:47:43,130 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:47:43,262 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:47:44,417 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:47:48,307 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:47:49,418 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:47:53,201 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:47:53,912 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:47:55,108 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:48:00,109 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:48:00,293 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:48:03,270 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:48:03,909 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:48:05,294 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:48:05,910 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:48:10,660 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:48:11,888 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:48:13,325 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:48:15,661 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:48:16,889 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:48:21,229 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:48:21,229 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:48:21,418 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:48:22,463 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:48:26,230 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:48:27,465 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:48:31,484 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:48:31,489 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:48:33,096 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:48:33,910 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:48:36,713 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:48:38,911 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:48:41,549 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:48:41,742 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:48:43,951 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:48:48,193 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:48:48,952 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:48:51,617 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:48:53,194 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:48:54,388 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:48:58,605 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:48:59,840 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:49:01,688 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:49:03,606 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:49:03,910 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:49:04,911 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:49:08,945 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:49:08,946 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:49:09,738 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:49:10,134 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:49:13,946 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:49:15,136 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:49:19,229 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:49:19,812 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:49:20,810 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:49:25,811 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:49:26,070 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:49:29,875 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:49:31,071 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:49:31,294 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:49:33,913 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:49:36,827 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:49:36,827 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:49:36,960 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:49:36,987 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 12:49:36,988 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 12:49:36,988 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 12:49:36,991 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 12:49:37,984 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:49:37,985 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 12:49:39,994 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:49:41,828 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:49:41,992 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:49:47,209 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:49:47,498 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:49:48,052 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:49:52,210 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:49:52,499 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:49:57,613 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:49:57,862 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:49:58,110 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:50:02,615 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:50:02,864 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:50:03,913 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:50:07,176 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:50:08,254 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:50:08,255 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:50:08,614 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:50:13,616 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:50:13,690 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:50:17,234 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:50:18,691 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:50:18,851 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:50:23,852 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:50:24,034 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:50:24,035 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:50:25,294 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:50:29,035 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:50:29,325 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:50:33,913 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:50:34,077 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:50:35,322 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:50:35,361 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:50:39,077 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:50:40,323 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:50:44,706 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:50:45,558 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:50:45,937 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:50:49,707 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:50:50,939 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:50:55,170 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:50:55,618 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:50:56,371 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:51:00,172 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:51:01,372 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:51:03,914 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:51:05,254 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:51:05,690 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:51:06,915 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:51:11,175 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:51:11,916 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:51:13,749 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:51:16,176 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:51:17,381 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:51:22,147 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:51:22,148 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:51:23,380 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:51:23,809 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:51:27,148 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:51:28,380 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:51:32,522 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:51:33,774 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:51:33,887 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:51:33,915 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:51:37,523 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:51:38,916 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:51:42,923 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:51:43,958 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:51:44,121 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:51:47,924 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:51:49,123 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:51:53,325 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:51:53,326 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:51:54,024 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:51:54,600 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:51:58,327 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:51:59,602 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:52:03,915 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:52:04,084 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:52:04,132 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:52:04,916 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:52:09,465 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:52:09,465 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:52:09,918 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:52:12,132 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:52:14,466 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:52:15,712 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:52:20,004 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:52:21,261 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:52:22,243 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:52:25,005 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:52:26,262 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:52:30,165 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:52:31,308 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:52:31,464 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:52:33,916 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:52:35,166 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:52:36,917 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:52:40,965 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:52:41,370 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:52:42,116 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:52:45,966 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:52:47,117 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:52:49,665 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 12:52:49,666 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 12:52:49,667 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 12:52:49,669 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 12:52:50,438 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 12:52:51,325 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:52:51,444 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:52:52,452 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:52:52,671 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:52:56,392 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:52:57,672 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:53:02,674 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:53:02,681 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:53:03,916 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:53:05,539 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:53:07,684 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:53:08,121 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:53:13,122 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:53:13,311 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:53:15,601 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:53:18,312 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:53:18,616 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:53:23,889 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:53:24,169 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:53:25,662 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:53:28,890 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:53:29,178 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:53:33,891 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:53:33,926 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:53:34,927 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:53:39,372 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:53:39,928 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:53:44,373 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:53:44,937 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:53:49,682 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:53:49,683 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:53:49,938 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:53:53,824 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:53:54,683 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:53:55,400 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:54:00,401 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:54:00,761 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:54:03,892 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:54:03,926 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:54:05,762 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:54:05,927 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:54:10,928 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:54:11,336 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:54:11,337 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:54:13,952 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:54:16,337 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:54:16,684 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:54:21,890 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:54:22,378 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:54:24,012 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:54:27,133 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:54:27,379 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:54:32,387 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:54:33,019 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:54:33,927 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:54:34,081 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:54:37,928 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:54:39,527 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:54:39,528 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:54:42,138 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:54:43,687 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:54:44,529 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:54:48,689 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:54:49,926 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:54:52,209 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:54:54,254 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:54:54,927 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:54:59,255 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:55:00,392 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:55:02,273 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:55:03,927 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:55:04,928 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:55:05,393 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:55:10,535 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:55:11,068 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:55:11,068 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:55:12,342 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:55:15,536 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:55:16,076 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:55:21,224 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:55:22,047 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:55:22,411 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:55:26,226 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:55:28,080 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:55:31,442 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:55:32,477 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:55:33,093 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:55:33,928 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:55:36,929 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:55:39,079 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:55:39,080 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:55:41,536 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:55:42,460 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:55:44,080 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:55:47,462 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:55:49,593 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:55:49,792 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:55:53,087 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:55:54,793 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:55:58,088 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:55:59,732 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:56:00,366 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:56:03,690 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:56:03,928 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:56:05,368 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:56:09,575 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:56:09,804 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:56:11,080 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:56:14,576 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:56:17,201 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:56:19,862 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:56:20,443 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:56:22,202 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:56:25,445 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:56:27,928 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:56:27,929 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:56:28,120 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 12:56:28,121 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 12:56:28,121 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 12:56:28,124 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 12:56:28,916 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 12:56:29,920 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:56:31,125 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:56:32,930 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:56:33,929 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:56:36,931 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:56:38,080 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:56:39,987 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:56:42,792 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:56:43,490 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:56:48,049 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:56:48,096 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:56:48,948 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:56:53,097 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:56:53,950 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:56:58,122 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:56:59,026 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:56:59,760 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:57:03,930 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:57:04,931 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:57:05,293 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:57:08,192 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:57:10,294 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:57:10,564 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:57:15,308 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:57:15,566 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:57:18,248 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:57:20,310 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:57:21,504 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:57:25,702 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:57:25,703 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:57:26,973 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:57:28,309 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:57:30,703 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:57:31,973 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:57:33,931 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:57:36,490 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:57:37,661 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:57:38,373 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:57:41,490 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:57:42,663 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:57:47,421 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:57:48,441 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:57:48,621 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:57:53,623 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:57:53,957 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:57:53,958 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:57:56,492 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:57:58,958 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:57:59,241 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:58:03,931 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:58:04,620 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:58:04,888 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:58:06,558 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:58:09,621 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:58:09,889 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:58:14,960 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:58:15,106 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:58:16,622 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:58:19,961 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:58:20,107 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:58:25,338 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:58:25,595 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:58:26,695 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:58:30,339 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:58:30,597 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:58:33,932 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:58:35,537 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:58:35,757 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:58:35,933 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:58:40,934 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:58:41,985 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:58:41,985 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:58:43,806 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:58:46,233 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:58:46,986 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:58:51,234 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:58:52,338 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:58:53,863 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:58:56,770 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:58:57,540 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:59:01,771 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:59:03,121 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:59:03,917 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:59:03,932 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:59:06,938 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:59:08,121 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:59:12,732 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:59:13,524 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:59:13,524 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:59:13,979 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:59:17,733 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:59:19,861 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:59:23,544 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:59:24,054 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:59:24,869 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:59:28,546 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:59:30,448 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:59:33,757 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:59:33,933 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 12:59:34,120 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:59:35,449 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:59:38,935 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:59:41,216 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:59:41,217 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:59:41,550 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 12:59:41,551 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 12:59:41,552 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 12:59:41,555 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 12:59:42,175 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:59:42,176 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 12:59:44,189 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:59:44,557 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:59:46,217 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:59:49,559 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:59:51,620 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:59:52,241 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 12:59:54,748 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 12:59:56,621 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 12:59:59,749 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:00:02,145 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:00:02,314 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:00:03,933 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:00:04,934 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:00:07,147 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:00:09,935 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:00:12,216 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:00:12,379 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:00:15,212 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:00:18,887 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:00:21,060 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:00:22,435 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:00:23,888 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:00:26,061 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:00:29,536 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:00:29,537 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:00:31,909 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:00:32,497 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:00:33,934 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:00:34,537 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:00:36,935 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:00:40,216 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:00:42,633 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:00:42,658 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:00:45,369 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:00:47,635 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:00:50,707 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:00:50,940 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:00:53,152 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:00:55,941 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:00:58,153 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:01:01,237 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:01:02,833 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:01:03,753 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:01:03,934 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:01:06,238 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:01:08,936 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:01:10,888 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:01:12,006 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:01:14,262 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:01:18,023 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:01:19,948 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:01:20,221 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:01:23,025 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:01:25,222 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:01:28,859 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:01:28,860 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:01:30,022 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:01:31,128 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:01:33,861 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:01:33,935 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:01:36,936 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:01:39,282 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:01:40,089 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:01:42,522 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:01:44,283 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:01:47,524 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:01:49,817 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:01:50,166 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:01:53,223 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:01:54,819 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:01:58,218 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:01:58,658 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:02:00,393 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:02:00,394 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:02:03,659 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:02:03,936 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:02:06,315 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:02:08,486 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:02:09,547 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:02:11,316 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:02:14,549 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:02:16,697 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:02:18,549 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:02:19,843 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:02:21,698 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:02:24,844 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:02:27,038 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:02:28,610 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:02:30,556 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:02:32,039 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:02:33,937 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:02:35,939 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:02:37,907 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:02:38,674 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:02:41,216 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:02:44,470 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:02:46,720 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:02:46,733 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:02:49,471 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:02:51,721 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:02:54,835 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:02:55,053 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 13:02:55,055 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 13:02:55,055 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 13:02:55,058 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 13:02:55,796 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 13:02:56,802 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:02:57,060 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:02:59,836 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:03:02,062 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:03:03,939 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:03:04,930 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:03:06,863 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:03:07,163 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:03:09,931 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:03:12,164 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:03:15,286 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:03:15,286 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:03:16,929 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:03:17,510 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:03:20,287 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:03:22,511 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:03:25,662 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:03:26,991 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:03:27,904 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:03:30,664 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:03:32,907 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:03:33,938 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:03:35,664 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:03:37,057 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:03:37,939 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:03:42,153 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:03:42,940 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:03:44,105 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:03:47,154 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:03:48,415 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:03:52,661 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:03:53,916 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:03:54,177 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:03:57,662 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:03:58,916 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:04:03,572 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:04:03,939 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:04:03,940 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:04:04,247 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:04:08,573 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:04:08,940 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:04:13,944 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:04:14,130 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:04:14,329 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:04:18,946 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:04:19,131 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:04:22,383 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:04:24,509 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:04:24,737 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:04:29,739 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:04:30,511 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:04:32,446 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:04:33,939 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:04:34,940 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:04:35,512 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:04:39,942 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:04:40,944 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:04:42,519 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:04:45,424 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:04:46,134 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:04:50,425 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:04:51,570 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:04:52,583 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:04:55,691 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:04:56,570 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:05:01,606 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:05:01,973 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:05:02,643 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:05:03,940 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:05:06,941 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:05:06,974 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:05:12,163 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:05:12,510 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:05:12,705 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:05:17,512 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:05:18,676 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:05:20,764 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:05:22,836 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:05:23,678 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:05:27,838 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:05:29,541 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:05:29,541 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:05:30,833 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:05:33,799 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:05:33,941 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:05:34,542 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:05:38,942 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:05:40,146 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:05:40,893 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:05:44,424 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:05:45,147 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:05:49,728 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:05:50,581 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:05:50,956 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:05:54,729 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:05:57,215 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:05:57,215 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:06:00,422 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:06:01,033 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:06:02,216 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:06:03,941 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:06:05,943 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:06:07,639 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:06:07,878 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 13:06:07,880 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 13:06:07,880 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 13:06:07,883 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 13:06:08,088 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 13:06:09,093 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:06:11,885 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:06:12,640 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:06:16,887 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:06:17,735 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:06:17,735 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:06:21,174 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:06:22,257 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:06:22,736 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:06:27,258 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:06:28,541 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:06:29,230 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:06:32,959 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:06:33,728 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:06:33,942 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:06:38,302 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:06:38,460 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:06:39,210 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:06:43,461 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:06:44,220 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:06:48,366 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:06:49,127 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:06:49,954 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:06:54,129 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:06:55,600 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:06:58,421 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:06:59,748 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:07:00,601 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:07:03,944 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:07:04,946 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:07:06,545 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:07:08,486 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:07:09,949 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:07:11,546 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:07:14,950 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:07:16,557 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:07:16,557 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:07:18,552 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:07:20,889 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:07:21,558 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:07:25,891 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:07:26,634 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:07:28,625 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:07:31,636 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:07:31,874 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:07:33,944 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:07:36,636 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:07:36,945 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:07:42,200 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:07:42,711 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:07:42,724 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:07:47,201 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:07:47,724 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:07:52,775 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:07:53,148 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:07:53,235 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:07:58,236 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:07:59,638 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:07:59,638 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:08:02,841 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:08:03,945 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:08:03,946 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:08:04,640 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:08:08,947 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:08:09,685 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:08:12,910 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:08:13,989 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:08:14,696 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:08:18,991 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:08:20,370 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:08:22,975 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:08:24,694 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:08:25,371 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:08:29,695 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:08:30,639 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:08:33,037 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:08:33,945 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:08:34,946 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:08:36,035 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:08:40,827 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:08:41,102 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:08:41,578 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:08:45,828 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:08:46,580 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:08:51,169 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:08:51,533 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:08:52,345 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:08:56,534 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:08:58,464 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:08:58,465 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:09:01,236 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:09:01,623 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:09:03,466 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:09:03,946 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:09:06,947 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:09:08,731 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:09:11,303 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:09:12,087 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:09:13,732 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:09:17,089 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:09:19,474 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:09:21,366 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:09:22,770 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:09:24,475 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:09:27,771 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:09:28,375 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 13:09:28,377 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 13:09:28,377 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 13:09:28,380 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 13:09:28,421 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 13:09:29,639 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:09:31,439 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:09:33,382 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:09:33,947 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:09:36,638 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:09:38,493 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:09:38,897 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:09:41,640 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:09:43,898 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:09:47,639 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:09:47,639 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:09:48,557 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:09:49,875 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:09:52,640 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:09:54,880 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:09:58,126 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:09:58,612 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:10:00,327 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:10:03,127 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:10:03,948 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:10:05,951 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:10:08,690 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:10:08,750 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:10:11,933 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:10:13,752 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:10:17,581 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:10:18,756 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:10:19,430 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:10:19,430 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:10:22,582 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:10:24,870 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:10:26,809 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:10:28,061 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:10:29,871 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:10:33,063 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:10:33,949 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:10:35,280 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:10:35,281 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:10:36,873 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:10:38,592 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:10:40,281 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:10:43,594 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:10:45,429 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:10:46,938 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:10:48,794 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:10:50,513 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:10:53,795 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:10:55,927 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:10:57,006 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:10:59,129 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:11:00,928 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:11:03,949 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:11:04,631 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:11:06,398 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:11:07,070 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:11:09,632 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:11:12,802 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:11:15,020 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:11:15,131 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:11:17,803 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:11:20,021 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:11:23,092 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:11:25,204 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:11:25,223 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:11:28,093 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:11:30,224 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:11:33,950 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:11:34,049 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:11:34,049 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:11:35,273 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:11:35,951 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:11:39,050 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:11:40,953 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:11:44,512 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:11:45,389 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:11:45,997 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:11:50,998 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:11:51,212 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:11:53,447 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:11:56,213 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:11:56,428 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:12:01,429 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:12:01,832 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:12:01,832 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:12:03,512 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:12:03,951 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:12:06,833 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:12:06,952 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:12:12,244 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:12:12,516 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:12:13,578 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:12:17,245 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:12:17,517 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:12:22,673 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:12:22,906 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:12:23,637 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:12:27,674 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:12:27,907 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:12:32,703 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:12:32,833 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:12:33,901 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:12:33,951 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:12:38,953 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:12:39,090 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:12:40,300 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 13:12:40,301 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 13:12:40,302 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 13:12:40,304 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 13:12:40,749 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:12:40,750 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 13:12:41,754 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:12:44,091 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:12:44,306 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:12:49,307 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:12:49,684 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:12:49,685 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:12:50,821 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:12:54,685 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:12:54,988 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:13:00,264 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:13:00,511 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:13:00,878 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:13:03,952 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:13:05,265 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:13:05,954 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:13:10,835 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:13:10,959 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:13:11,105 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:13:15,837 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:13:16,107 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:13:21,039 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:13:21,818 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:13:22,063 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:13:27,064 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:13:27,271 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:13:29,088 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:13:32,272 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:13:32,535 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:13:33,952 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:13:37,888 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:13:37,954 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:13:39,167 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:13:42,889 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:13:43,337 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:13:48,627 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:13:48,628 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:13:48,886 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:13:49,279 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:13:53,628 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:13:53,886 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:13:59,325 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:13:59,335 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:13:59,469 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:14:03,956 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:14:04,327 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:14:04,959 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:14:09,400 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:14:09,971 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:14:10,165 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:14:15,167 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:14:15,501 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:14:19,463 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:14:20,503 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:14:20,804 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:14:25,805 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:14:26,339 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:14:27,529 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:14:31,340 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:14:31,474 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:14:33,956 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:14:36,975 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:14:37,184 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:14:37,590 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:14:41,976 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:14:42,184 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:14:47,004 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:14:47,235 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:14:47,650 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:14:52,005 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:14:52,236 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:14:57,601 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:14:57,784 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:14:57,801 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:15:02,803 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:15:03,957 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:15:04,055 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:15:04,056 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:15:07,855 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:15:08,354 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:15:09,057 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:15:13,356 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:15:14,678 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:15:17,010 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:15:19,044 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:15:19,679 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:15:24,046 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:15:25,101 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:15:25,627 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:15:29,792 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:15:30,628 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:15:33,968 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:15:35,160 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:15:35,524 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:15:36,223 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:15:36,224 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:15:40,525 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:15:41,226 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:15:45,218 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:15:46,034 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:15:46,627 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:15:46,628 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:15:51,035 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:15:52,293 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:15:53,540 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 13:15:53,541 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 13:15:53,542 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 13:15:53,545 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 13:15:54,406 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 13:15:55,412 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:15:56,547 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:15:57,294 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:16:01,548 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:16:02,884 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:16:02,884 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:16:03,958 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:16:05,473 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:16:06,959 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:16:07,885 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:16:11,960 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:16:13,395 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:16:13,530 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:16:17,654 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:16:18,396 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:16:23,420 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:16:23,598 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:16:24,232 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:16:28,421 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:16:30,529 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:16:33,675 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:16:33,750 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:16:33,958 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:16:35,531 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:16:38,960 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:16:40,832 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:16:43,737 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:16:43,997 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:16:45,833 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:16:48,998 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:16:51,103 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:16:51,103 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:16:51,789 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:16:54,228 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:16:56,104 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:16:59,230 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:17:01,496 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:17:01,863 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:17:03,959 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:17:04,960 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:17:06,497 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:17:10,958 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:17:11,577 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:17:11,932 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:17:15,959 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:17:18,125 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:17:18,126 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:17:21,365 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:17:22,000 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:17:23,127 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:17:26,366 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:17:28,425 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:17:29,052 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:17:31,627 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:17:33,426 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:17:33,960 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:17:36,962 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:17:38,862 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:17:39,120 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:17:42,154 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:17:43,863 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:17:47,155 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:17:49,126 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:17:49,181 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:17:52,809 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:17:54,523 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:17:58,289 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:17:59,252 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:18:00,069 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:18:03,290 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:18:03,961 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:18:05,825 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:18:05,826 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:18:09,152 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:18:09,320 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:18:10,826 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:18:14,153 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:18:16,353 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:18:17,388 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:18:19,606 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:18:21,354 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:18:24,608 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:18:26,637 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:18:27,456 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:18:29,797 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:18:31,639 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:18:33,963 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:18:34,965 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:18:37,212 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:18:37,213 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:18:37,519 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:18:40,554 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:18:42,214 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:18:46,301 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:18:47,580 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:18:47,913 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:18:51,302 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:18:53,535 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:18:53,536 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:18:55,641 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:18:57,050 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:18:58,537 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:19:02,051 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:19:03,964 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:19:04,486 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:19:04,487 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:19:04,487 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:19:05,686 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 13:19:05,687 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 13:19:05,688 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 13:19:05,690 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 13:19:05,713 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 13:19:07,692 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:19:07,727 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:19:09,488 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:19:12,693 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:19:15,014 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:19:17,798 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:19:18,262 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:19:20,015 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:19:23,263 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:19:25,658 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:19:27,867 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:19:29,048 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:19:30,658 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:19:33,965 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:19:34,879 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:19:36,487 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:19:37,936 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:19:39,880 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:19:41,629 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:19:45,606 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:19:45,984 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:19:47,223 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:19:50,607 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:19:53,015 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:19:53,015 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:19:56,049 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:19:56,314 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:19:58,016 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:20:01,315 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:20:03,965 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:20:03,968 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:20:06,114 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:20:06,336 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:20:08,969 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:20:11,337 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:20:14,678 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:20:16,206 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:20:17,062 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:20:19,679 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:20:22,063 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:20:25,205 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:20:25,281 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:20:27,395 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:20:31,468 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:20:32,396 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:20:33,337 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:20:33,966 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:20:36,469 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:20:37,967 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:20:42,340 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:20:43,408 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:20:43,606 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:20:47,341 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:20:48,607 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:20:52,741 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:20:52,741 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:20:53,477 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:20:53,926 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:20:57,742 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:20:58,927 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:21:03,181 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:21:03,550 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:21:03,966 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:21:03,967 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:21:08,181 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:21:08,968 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:21:13,365 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:21:13,612 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:21:14,544 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:21:19,545 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:21:19,694 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:21:23,679 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:21:24,695 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:21:25,041 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:21:30,042 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:21:30,294 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:21:33,749 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:21:33,967 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:21:35,295 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:21:35,968 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:21:40,744 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:21:40,980 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:21:43,921 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:21:45,745 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:21:45,983 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:21:50,746 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:21:50,984 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:21:55,939 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:21:56,401 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:21:58,012 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:22:00,940 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:22:01,401 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:22:03,969 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:22:06,741 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:22:07,046 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:22:08,079 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:22:12,047 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:22:13,331 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:22:16,133 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:22:17,605 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:22:18,332 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:22:22,606 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:22:23,869 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:22:23,870 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:22:24,150 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 13:22:24,151 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 13:22:24,152 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 13:22:24,154 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 13:22:24,186 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 13:22:26,200 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:22:28,156 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:22:28,870 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:22:33,157 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:22:33,969 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:22:34,496 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:22:36,271 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:22:38,971 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:22:39,497 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:22:44,721 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:22:45,419 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:22:46,332 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:22:49,722 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:22:51,936 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:22:51,936 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:22:55,177 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:22:55,399 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:22:56,937 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:23:00,179 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:23:02,847 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:23:03,970 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:23:05,462 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:23:05,971 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:23:07,848 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:23:10,973 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:23:13,275 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:23:13,519 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:23:16,541 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:23:18,276 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:23:21,542 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:23:23,576 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:23:23,863 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:23:27,187 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:23:28,864 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:23:32,957 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:23:33,653 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:23:33,970 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:23:34,737 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:23:37,971 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:23:40,228 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:23:40,228 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:23:43,487 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:23:43,720 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:23:45,229 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:23:48,488 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:23:50,597 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:23:51,781 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:23:53,807 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:23:55,599 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:23:58,808 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:24:00,902 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:24:01,851 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:24:03,971 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:24:03,971 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:24:05,903 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:24:08,973 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:24:11,228 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:24:11,228 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:24:11,912 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:24:14,509 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:24:17,719 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:24:19,972 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:24:20,062 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:24:22,720 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:24:25,063 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:24:27,723 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:24:27,723 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:24:30,039 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:24:30,889 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:24:32,725 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:24:33,971 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:24:35,973 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:24:38,228 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:24:38,229 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:24:40,093 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:24:41,603 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:24:43,289 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:24:46,604 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:24:49,102 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:24:50,170 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:24:52,245 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:24:54,102 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:24:57,246 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:24:59,446 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:25:00,234 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:25:02,921 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:25:03,972 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:25:04,448 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:25:08,342 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:25:10,113 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:25:10,310 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:25:13,343 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:25:15,573 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:25:18,370 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:25:18,924 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:25:20,574 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:25:23,926 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:25:25,862 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:25:25,863 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:25:28,434 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:25:29,031 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:25:30,864 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:25:33,973 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:25:34,975 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:25:36,162 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:25:36,385 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 13:25:36,386 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 13:25:36,387 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 13:25:36,390 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 13:25:36,500 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 13:25:38,509 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:25:40,395 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:25:41,163 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:25:45,396 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:25:46,607 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:25:48,568 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:25:50,850 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:25:51,607 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:25:55,616 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:25:56,075 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:25:56,886 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:26:01,076 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:26:03,146 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:26:03,973 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:26:05,669 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:26:06,270 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:26:08,147 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:26:11,271 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:26:13,494 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:26:13,495 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:26:15,737 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:26:16,793 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:26:18,495 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:26:21,795 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:26:23,534 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:26:25,808 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:26:27,752 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:26:28,534 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:26:32,753 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:26:33,863 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:26:33,966 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:26:33,974 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:26:37,975 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:26:38,967 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:26:43,687 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:26:43,925 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:26:44,395 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:26:48,688 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:26:49,396 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:26:53,689 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:26:54,398 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:26:58,690 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:26:59,401 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:27:03,975 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:27:03,976 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:27:04,077 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:27:05,129 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:27:08,977 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:27:10,545 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:27:10,545 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:27:14,141 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:27:14,855 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:27:15,546 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:27:19,857 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:27:20,982 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:27:22,207 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:27:25,231 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:27:25,983 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:27:30,233 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:27:31,718 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:27:32,269 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:27:33,975 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:27:35,976 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:27:36,719 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:27:40,977 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:27:41,812 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:27:41,812 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:27:42,347 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:27:46,036 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:27:46,813 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:27:51,728 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:27:52,408 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:27:52,411 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:27:56,729 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:27:58,966 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:27:58,966 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:28:02,273 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:28:02,474 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:28:03,968 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:28:03,976 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:28:07,977 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:28:09,305 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:28:12,544 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:28:13,486 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:28:14,306 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:28:18,487 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:28:19,670 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:28:20,599 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:28:23,901 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:28:24,671 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:28:29,462 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:28:30,141 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:28:30,666 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:28:33,976 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:28:34,977 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:28:35,142 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:28:40,176 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:28:40,413 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:28:40,742 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:28:45,414 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:28:46,594 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:28:49,813 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:28:50,867 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:28:51,595 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:28:55,868 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:28:57,104 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:28:57,105 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:28:57,372 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 13:28:57,373 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 13:28:57,374 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 13:28:57,376 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 13:28:57,872 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:28:57,873 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 13:28:58,877 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:29:01,378 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:29:02,105 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:29:03,977 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:29:06,603 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:29:07,469 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:29:07,942 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:29:11,604 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:29:12,470 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:29:16,814 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:29:17,578 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:29:18,010 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:29:21,815 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:29:22,846 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:29:27,222 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:29:27,999 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:29:28,073 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:29:32,223 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:29:33,001 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:29:33,978 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:29:37,948 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:29:38,136 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:29:38,782 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:29:42,950 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:29:44,199 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:29:46,189 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:29:48,470 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:29:49,200 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:29:53,472 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:29:55,062 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:29:56,257 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:29:59,450 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:30:00,063 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:30:03,981 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:30:04,723 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:30:05,594 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:30:06,315 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:30:09,724 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:30:10,594 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:30:15,390 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:30:16,058 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:30:16,058 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:30:16,385 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:30:20,391 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:30:22,548 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:30:25,769 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:30:26,441 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:30:27,599 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:30:30,771 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:30:33,599 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:30:33,980 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:30:36,065 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:30:36,612 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:30:38,600 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:30:41,067 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:30:43,939 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:30:43,940 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:30:46,250 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:30:46,677 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:30:48,940 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:30:51,251 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:30:54,512 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:30:56,752 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:30:56,810 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:30:59,514 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:31:01,812 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:31:03,980 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:31:04,809 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:31:05,222 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:31:06,982 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:31:10,224 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:31:11,983 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:31:14,873 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:31:15,833 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:31:15,834 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:31:17,132 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:31:21,301 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:31:22,133 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:31:24,938 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:31:26,365 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:31:27,539 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:31:32,347 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:31:32,540 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:31:33,981 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:31:33,998 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:31:37,348 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:31:37,982 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:31:42,804 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:31:42,995 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:31:44,060 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:31:47,805 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:31:47,996 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:31:52,110 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:31:53,217 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:31:53,433 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:31:58,218 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:31:58,434 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:32:02,181 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:32:03,917 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:32:03,981 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:32:03,982 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:32:07,470 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 13:32:07,472 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 13:32:07,472 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 13:32:07,475 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 13:32:08,216 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:32:08,216 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 13:32:08,918 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:32:09,221 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:32:09,477 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:32:13,919 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:32:14,246 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:32:14,712 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:32:15,252 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:32:18,920 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:32:19,712 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:32:24,318 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:32:24,859 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:32:25,011 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:32:30,012 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:32:31,234 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:32:33,982 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:32:34,393 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:32:35,983 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:32:36,235 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:32:40,985 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:32:41,834 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:32:41,834 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:32:44,457 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:32:46,167 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:32:46,880 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:32:51,168 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:32:52,860 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:32:54,522 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:32:57,081 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:32:57,860 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:33:02,587 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:33:02,647 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:33:03,309 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:33:03,983 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:33:07,985 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:33:09,825 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:33:12,655 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:33:13,054 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:33:14,827 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:33:18,055 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:33:20,220 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:33:22,720 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:33:23,488 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:33:25,222 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:33:28,489 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:33:30,675 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:33:32,797 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:33:33,934 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:33:33,983 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:33:35,676 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:33:38,984 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:33:40,852 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:33:41,078 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:33:44,238 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:33:46,080 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:33:49,239 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:33:50,924 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:33:51,947 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:33:54,312 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:33:57,640 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:33:57,641 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:33:59,889 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:34:00,989 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:34:02,642 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:34:03,984 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:34:04,985 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:34:07,888 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:34:10,045 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:34:11,062 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:34:12,889 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:34:15,047 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:34:18,593 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:34:20,048 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:34:21,128 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:34:23,596 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:34:25,049 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:34:28,178 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:34:28,641 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:34:30,893 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:34:33,984 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:34:34,569 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:34:35,985 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:34:38,244 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:34:40,261 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:34:41,411 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:34:46,412 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:34:46,501 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:34:46,502 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:34:48,305 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:34:51,503 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:34:51,777 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:34:56,538 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:34:56,779 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:35:00,376 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:35:01,540 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:35:02,033 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:35:03,985 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:35:07,230 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:35:07,396 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:35:08,440 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:35:12,231 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:35:12,397 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:35:16,997 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 13:35:16,998 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 13:35:16,999 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 13:35:17,001 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 13:35:17,498 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 13:35:17,698 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:35:18,003 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:35:18,503 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:35:22,699 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:35:23,003 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:35:28,280 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:35:28,564 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:35:28,566 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:35:30,572 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:35:33,281 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:35:33,985 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:35:33,987 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:35:38,371 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:35:38,627 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:35:39,648 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:35:43,372 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:35:44,649 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:35:48,695 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:35:48,727 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:35:49,904 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:35:53,728 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:35:54,905 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:35:58,756 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:35:59,239 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:36:00,536 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:36:03,988 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:36:04,241 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:36:05,991 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:36:08,825 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:36:09,922 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:36:11,232 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:36:14,923 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:36:16,233 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:36:20,837 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:36:21,234 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:36:22,911 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:36:25,839 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:36:26,245 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:36:30,899 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:36:30,900 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:36:31,247 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:36:32,972 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:36:33,988 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:36:35,900 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:36:36,989 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:36:41,184 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:36:42,406 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:36:43,031 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:36:46,185 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:36:47,407 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:36:51,796 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:36:53,095 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:36:53,131 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:36:56,797 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:36:58,132 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:37:02,622 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:37:03,158 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:37:03,841 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:37:03,989 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:37:08,991 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:37:09,136 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:37:13,229 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:37:14,137 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:37:14,410 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:37:19,411 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:37:19,633 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:37:21,286 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:37:24,633 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:37:24,923 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:37:29,925 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:37:30,591 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:37:30,592 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:37:33,361 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:37:33,990 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:37:34,991 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:37:35,593 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:37:39,992 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:37:41,301 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:37:42,425 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:37:45,819 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:37:46,301 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:37:51,494 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:37:51,813 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:37:52,501 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:37:56,495 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:37:56,814 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:38:02,002 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:38:02,267 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:38:02,565 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:38:03,990 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:38:07,004 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:38:07,992 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:38:12,632 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:38:12,993 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:38:13,294 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:38:18,296 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:38:18,690 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:38:18,691 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:38:20,684 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:38:23,692 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:38:24,021 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:38:29,002 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:38:29,002 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:38:29,022 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:38:30,755 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:38:33,991 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:38:34,250 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:38:34,992 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:38:38,957 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 13:38:38,959 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 13:38:38,960 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 13:38:38,962 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 13:38:39,742 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:38:39,822 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 13:38:40,828 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:38:40,964 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:38:44,743 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:38:45,966 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:38:50,143 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:38:50,888 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:38:51,338 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:38:56,340 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:38:56,634 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:38:58,952 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:39:01,635 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:39:02,017 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:39:03,991 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:39:07,567 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:39:07,723 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:39:09,020 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:39:12,569 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:39:12,724 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:39:18,232 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:39:18,233 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:39:18,356 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:39:19,248 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:39:23,233 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:39:23,357 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:39:28,524 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:39:28,781 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:39:29,310 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:39:33,526 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:39:33,783 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:39:33,994 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:39:38,536 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:39:38,597 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:39:38,788 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:39:43,789 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:39:45,004 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:39:45,005 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:39:45,006 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:39:46,650 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:39:49,153 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:39:50,006 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:39:54,154 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:39:55,289 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:39:56,716 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:39:59,520 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:40:00,290 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:40:03,995 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:40:04,997 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:40:05,945 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:40:06,775 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:40:10,217 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:40:10,947 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:40:15,825 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:40:16,005 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:40:16,859 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:40:20,826 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:40:23,024 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:40:24,912 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:40:26,252 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:40:28,025 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:40:31,254 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:40:33,538 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:40:33,538 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:40:33,995 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:40:34,974 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:40:36,996 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:40:38,539 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:40:41,997 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:40:44,290 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:40:45,061 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:40:47,730 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:40:49,291 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:40:52,732 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:40:54,966 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:40:55,152 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:40:58,451 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:40:59,966 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:41:03,452 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:41:03,995 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:41:05,221 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:41:05,388 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:41:08,997 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:41:10,389 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:41:14,666 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:41:15,334 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:41:16,005 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:41:19,667 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:41:22,254 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:41:25,436 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:41:25,579 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:41:27,255 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:41:30,580 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:41:32,839 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:41:32,840 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:41:33,996 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:41:35,495 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:41:35,997 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:41:37,840 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:41:40,999 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:41:43,455 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:41:45,565 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:41:46,755 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:41:48,455 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:41:51,756 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:41:52,044 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 13:41:52,045 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 13:41:52,046 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 13:41:52,048 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 13:41:52,614 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 13:41:53,622 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:41:53,857 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:41:57,050 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:41:58,859 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:42:02,923 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:42:03,682 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:42:03,997 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:42:04,717 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:42:07,998 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:42:10,137 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:42:12,737 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:42:13,383 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:42:15,138 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:42:18,384 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:42:20,425 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:42:20,426 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:42:22,807 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:42:23,621 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:42:25,427 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:42:28,622 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:42:30,717 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:42:30,858 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:42:33,997 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:42:33,998 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:42:36,172 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:42:39,000 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:42:40,925 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:42:41,602 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:42:44,863 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:42:46,609 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:42:50,577 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:42:50,992 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:42:52,423 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:42:55,578 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:42:57,747 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:43:01,024 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:43:01,067 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:43:02,749 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:43:03,998 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:43:06,999 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:43:08,322 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:43:09,116 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:43:12,623 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:43:13,323 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:43:17,624 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:43:18,600 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:43:19,179 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:43:22,659 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:43:23,601 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:43:28,276 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:43:29,129 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:43:29,248 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:43:33,277 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:43:34,062 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:43:35,565 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:43:38,806 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:43:39,311 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:43:40,566 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:43:43,807 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:43:46,441 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:43:47,369 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:43:49,570 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:43:51,442 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:43:54,572 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:43:56,879 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:43:57,432 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:44:00,153 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:44:01,880 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:44:04,001 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:44:05,979 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:44:07,503 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:44:07,775 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:44:10,980 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:44:14,363 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:44:15,981 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:44:17,569 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:44:19,364 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:44:20,998 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:44:25,340 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:44:26,893 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:44:27,633 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:44:30,341 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:44:31,894 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:44:34,003 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:44:35,601 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:44:35,601 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:44:37,699 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:44:37,860 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:44:40,601 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:44:42,861 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:44:46,269 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:44:47,768 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:44:48,594 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:44:51,270 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:44:53,596 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:44:56,934 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:44:57,824 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:44:59,202 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:45:01,935 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:45:04,004 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:45:04,877 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:45:04,878 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 13:45:04,879 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 13:45:04,880 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 13:45:04,882 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 13:45:05,882 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:45:05,883 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 13:45:07,669 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:45:07,893 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:45:09,885 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:45:13,149 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:45:14,944 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:45:15,400 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:45:18,150 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:45:20,401 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:45:23,657 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:45:25,015 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:45:25,913 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:45:28,658 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:45:30,914 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:45:34,004 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:45:34,577 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:45:34,577 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:45:35,081 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:45:36,005 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:45:39,578 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:45:41,019 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:45:45,140 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:45:45,154 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:45:46,418 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:45:51,419 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:45:51,632 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:45:51,633 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:45:55,216 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:45:56,634 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:45:56,870 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:46:01,871 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:46:02,080 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:46:03,277 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:46:04,004 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:46:07,005 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:46:07,081 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:46:12,573 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:46:12,833 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:46:13,337 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:46:17,574 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:46:17,834 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:46:22,621 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:46:23,407 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:46:23,809 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:46:27,622 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:46:28,828 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:46:33,055 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:46:33,467 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:46:34,005 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:46:34,006 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:46:39,007 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:46:39,480 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:46:41,531 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:46:44,485 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:46:44,674 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:46:49,676 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:46:50,243 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:46:50,244 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:46:51,604 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:46:55,245 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:46:55,333 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:47:00,334 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:47:01,054 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:47:01,669 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:47:04,006 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:47:06,006 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:47:06,060 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:47:11,176 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:47:11,742 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:47:11,896 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:47:16,177 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:47:16,898 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:47:21,227 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:47:21,820 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:47:22,794 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:47:26,228 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:47:28,539 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:47:31,873 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:47:31,892 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:47:33,540 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:47:34,006 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:47:37,008 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:47:39,085 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:47:41,967 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:47:42,343 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:47:44,111 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:47:47,345 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:47:49,793 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:47:52,051 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:47:52,688 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:47:55,031 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:47:57,690 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:48:00,921 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:48:02,119 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:48:03,538 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:48:04,007 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:48:05,922 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:48:09,009 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:48:11,178 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:48:11,498 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:48:14,778 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:48:17,937 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:48:19,214 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 13:48:19,215 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 13:48:19,216 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 13:48:19,218 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 13:48:19,250 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 13:48:20,220 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:48:21,261 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:48:22,267 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:48:22,938 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:48:25,221 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:48:28,381 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:48:29,316 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:48:30,684 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:48:33,382 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:48:34,007 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:48:36,008 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:48:38,400 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:48:38,401 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:48:39,380 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:48:41,812 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:48:43,401 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:48:46,813 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:48:48,794 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:48:48,794 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:48:49,458 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:48:52,470 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:48:54,147 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:48:57,471 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:48:59,483 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:48:59,513 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:49:02,670 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:49:04,008 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:49:06,057 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:49:06,057 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:49:08,428 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:49:09,579 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:49:11,058 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:49:13,429 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:49:16,724 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:49:17,637 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:49:19,002 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:49:21,725 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:49:24,004 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:49:27,259 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:49:27,697 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:49:29,475 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:49:32,260 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:49:34,010 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:49:35,012 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:49:37,755 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:49:37,889 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:49:37,890 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:49:40,218 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:49:42,946 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:49:45,219 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:49:47,822 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:49:48,882 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:49:51,205 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:49:54,629 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:49:54,629 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:49:56,923 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:49:57,891 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:49:59,630 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:50:01,924 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:50:04,012 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:50:04,890 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:50:04,890 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:50:07,578 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:50:07,958 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:50:10,241 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:50:12,579 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:50:16,080 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:50:18,025 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:50:18,243 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:50:21,081 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:50:23,244 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:50:26,076 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:50:26,317 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:50:28,450 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:50:31,327 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:50:33,451 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:50:34,011 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:50:37,144 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:50:37,146 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:50:38,602 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:50:40,154 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:50:42,145 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:50:43,603 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:50:47,395 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:50:48,605 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:50:50,222 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:50:52,396 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:50:53,878 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:50:58,370 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:50:59,603 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:51:00,280 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:51:03,371 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:51:04,012 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:51:05,013 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:51:08,901 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:51:08,901 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:51:09,350 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:51:10,206 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:51:13,902 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:51:15,207 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:51:19,399 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:51:19,410 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:51:20,639 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:51:25,640 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:51:25,889 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:51:27,462 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:51:30,890 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:51:31,134 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:51:34,013 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:51:35,952 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:51:37,016 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:51:37,214 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 13:51:37,215 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 13:51:37,216 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 13:51:37,218 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 13:51:37,519 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:51:37,520 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 13:51:40,535 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:51:40,954 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:51:42,220 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:51:45,999 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:51:47,221 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:51:47,586 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:51:51,000 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:51:52,257 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:51:56,484 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:51:57,658 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:51:57,661 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:52:01,485 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:52:02,659 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:52:04,014 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:52:06,956 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:52:07,727 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:52:08,283 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:52:12,102 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:52:13,284 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:52:17,426 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:52:17,799 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:52:18,714 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:52:22,428 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:52:23,716 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:52:27,863 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:52:27,982 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:52:29,117 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:52:33,423 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:52:34,014 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:52:34,686 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:52:35,919 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:52:38,424 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:52:39,687 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:52:43,468 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:52:44,689 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:52:45,980 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:52:48,469 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:52:49,741 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:52:53,817 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:52:53,817 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:52:54,981 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:52:56,049 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:52:58,818 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:52:59,987 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:53:04,053 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:53:04,410 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:53:05,054 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:53:06,143 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:53:09,411 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:53:10,055 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:53:14,963 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:53:15,213 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:53:16,212 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:53:19,965 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:53:20,215 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:53:25,070 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:53:25,070 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:53:25,470 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:53:26,277 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:53:30,072 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:53:30,471 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:53:34,016 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:53:35,923 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:53:36,017 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:53:36,341 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:53:41,018 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:53:41,363 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:53:41,363 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:53:44,413 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:53:46,364 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:53:46,684 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:53:51,686 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:53:52,008 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:53:54,474 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:53:57,009 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:53:57,276 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:54:02,278 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:54:02,441 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:54:03,542 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:54:04,016 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:54:07,441 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:54:08,017 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:54:13,193 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:54:13,394 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:54:13,607 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:54:18,194 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:54:18,395 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:54:23,657 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:54:23,668 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:54:23,937 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:54:28,938 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:54:30,132 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:54:33,734 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:54:34,017 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:54:34,017 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:54:35,202 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:54:39,019 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:54:40,826 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:54:40,826 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:54:43,800 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:54:44,103 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:54:45,827 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:54:49,104 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:54:50,687 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 13:54:50,689 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 13:54:50,690 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 13:54:50,691 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 13:54:50,847 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 13:54:51,392 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:54:51,852 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:54:54,693 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:54:56,394 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:54:59,695 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:55:01,709 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:55:02,145 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:55:04,017 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:55:05,018 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:55:06,710 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:55:10,018 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:55:11,713 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:55:15,019 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:55:16,714 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:55:20,020 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:55:22,071 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:55:22,074 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:55:25,122 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:55:26,299 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:55:27,657 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:55:30,123 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:55:32,901 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:55:34,018 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:55:35,130 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:55:36,365 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:55:37,902 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:55:40,131 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:55:43,476 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:55:45,598 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:55:46,427 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:55:48,478 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:55:50,599 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:55:54,182 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:55:54,183 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:55:56,309 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:55:56,489 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:55:59,183 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:56:01,311 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:56:04,018 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:56:04,397 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:56:04,548 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:56:07,019 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:56:09,398 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:56:12,021 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:56:15,092 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:56:16,628 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:56:17,342 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:56:20,093 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:56:22,344 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:56:23,683 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:56:25,183 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:56:28,083 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:56:31,445 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:56:33,750 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:56:34,019 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:56:34,020 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:56:36,447 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:56:39,021 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:56:41,613 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:56:41,614 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:56:43,819 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:56:44,828 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:56:46,615 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:56:49,830 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:56:52,183 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:56:52,184 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:56:53,885 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:56:55,664 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:56:57,810 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:57:00,665 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:57:02,814 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:57:04,019 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:57:05,966 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:57:06,020 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:57:07,815 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:57:11,022 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:57:13,044 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:57:14,017 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:57:16,215 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:57:18,045 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:57:21,217 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:57:23,183 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:57:24,081 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:57:26,705 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:57:28,416 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:57:31,706 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:57:33,811 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:57:34,020 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:57:34,194 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:57:37,021 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:57:40,228 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:57:40,228 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:57:42,463 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:57:44,270 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:57:45,246 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:57:47,464 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:57:50,247 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:57:52,466 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:57:55,634 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:57:55,635 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:57:56,338 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:57:58,018 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:58:00,635 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:58:03,020 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:58:04,021 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:58:06,183 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:58:06,403 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:58:08,915 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:58:11,184 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:58:13,917 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:58:16,254 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:58:18,535 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:58:19,462 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:58:21,255 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:58:24,463 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:58:26,256 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:58:29,464 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:58:31,513 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:58:32,023 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 13:58:32,024 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 13:58:32,025 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 13:58:32,028 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 13:58:32,623 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 13:58:34,021 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:58:34,633 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:58:35,022 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:58:36,520 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:58:40,023 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:58:42,474 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:58:42,475 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:58:44,692 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:58:45,739 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:58:47,475 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:58:50,741 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:58:52,681 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:58:56,513 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:58:56,774 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:58:58,203 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:59:01,515 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:59:03,279 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:59:04,021 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:59:06,836 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:59:07,022 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:59:08,281 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:59:12,024 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:59:13,342 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:59:15,902 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:59:17,742 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:59:18,343 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:59:22,744 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:59:23,555 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:59:25,971 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:59:27,918 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:59:28,556 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:59:32,920 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:59:34,023 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 13:59:34,070 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:59:36,040 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:59:37,992 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:59:39,073 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:59:42,994 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:59:44,320 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:59:48,158 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:59:48,708 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:59:49,322 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:59:53,710 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 13:59:54,352 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:59:54,352 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:59:58,227 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 13:59:59,353 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 13:59:59,623 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:00:04,025 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:00:04,620 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:00:05,029 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:00:08,290 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:00:09,621 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:00:10,099 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:00:15,101 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:00:15,115 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:00:18,354 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:00:20,116 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:00:20,371 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:00:25,373 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:00:25,526 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:00:25,527 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:00:28,424 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:00:30,542 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:00:31,048 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:00:34,026 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:00:36,022 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:00:37,029 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:00:38,486 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:00:41,085 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:00:42,266 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:00:46,373 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:00:47,267 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:00:50,565 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:00:51,374 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:00:52,605 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:00:57,057 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:00:57,057 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:00:58,228 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:00:58,618 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:01:02,058 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:01:03,229 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:01:04,027 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:01:07,126 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:01:08,437 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:01:08,682 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:01:12,128 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:01:13,438 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:01:17,953 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:01:18,740 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:01:19,163 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:01:22,954 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:01:24,164 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:01:28,399 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:01:29,637 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:01:30,818 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:01:33,400 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:01:34,027 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:01:35,029 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:01:38,691 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:01:38,871 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:01:40,416 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:01:44,188 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:01:45,418 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:01:48,943 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:01:49,993 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:01:51,229 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:01:55,573 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:01:55,574 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:01:56,231 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:01:56,860 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 14:01:56,861 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 14:01:56,865 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 14:01:56,867 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 14:01:56,993 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 14:01:59,003 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:02:00,575 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:02:01,870 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:02:04,029 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:02:06,285 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:02:07,574 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:02:09,063 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:02:11,287 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:02:12,574 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:02:16,287 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:02:17,575 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:02:22,229 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:02:22,576 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:02:27,230 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:02:27,577 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:02:32,293 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:02:32,578 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:02:34,029 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:02:38,030 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:02:38,451 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:02:38,451 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:02:42,244 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:02:43,452 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:02:43,656 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:02:48,658 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:02:49,365 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:02:52,309 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:02:53,864 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:02:54,366 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:02:58,865 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:02:59,431 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:03:02,367 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:03:04,041 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:03:04,041 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:03:04,433 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:03:09,043 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:03:10,026 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:03:10,027 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:03:10,027 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:03:12,435 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:03:14,451 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:03:15,028 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:03:19,452 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:03:20,261 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:03:22,508 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:03:24,535 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:03:25,262 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:03:29,537 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:03:30,375 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:03:34,030 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:03:34,586 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:03:35,031 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:03:35,376 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:03:40,033 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:03:40,616 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:03:40,616 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:03:40,617 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:03:44,645 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:03:45,617 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:03:45,811 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:03:50,812 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:03:51,131 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:03:52,706 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:03:56,132 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:03:56,579 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:04:02,045 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:04:02,461 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:04:02,792 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:04:04,030 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:04:07,046 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:04:08,032 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:04:12,616 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:04:12,896 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:04:13,211 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:04:17,903 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:04:18,213 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:04:22,955 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:04:23,279 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:04:23,634 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:04:28,281 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:04:28,636 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:04:33,014 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:04:34,031 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:04:34,032 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:04:34,052 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:04:39,033 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:04:39,661 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:04:43,085 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:04:44,663 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:04:44,945 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:04:49,947 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:04:50,350 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:04:53,152 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:04:55,351 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:04:55,673 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:05:01,073 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:05:01,464 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:05:03,221 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:05:04,031 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:05:06,074 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:05:07,032 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:05:11,323 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:05:12,612 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:05:13,278 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:05:16,324 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:05:17,614 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:05:22,124 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:05:22,344 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:05:23,539 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:05:27,125 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:05:28,541 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:05:32,417 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:05:32,706 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:05:33,942 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:05:34,032 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:05:38,232 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:05:39,034 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:05:39,528 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 14:05:39,529 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 14:05:39,529 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 14:05:39,532 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 14:05:40,492 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:05:40,493 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 14:05:43,233 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:05:44,534 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:05:48,981 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:05:50,243 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:05:50,562 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:05:53,982 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:05:55,244 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:05:58,983 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:06:00,245 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:06:04,032 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:06:04,446 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:06:06,033 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:06:06,670 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:06:09,447 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:06:11,035 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:06:14,593 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:06:14,735 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:06:16,810 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:06:19,594 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:06:21,811 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:06:24,595 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:06:26,811 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:06:29,596 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:06:31,812 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:06:34,033 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:06:34,611 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:06:37,035 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:06:39,612 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:06:42,035 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:06:44,613 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:06:47,037 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:06:49,723 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:06:52,917 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:06:52,959 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:06:54,724 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:06:57,918 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:07:00,229 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:07:03,021 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:07:03,721 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:07:04,033 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:07:05,231 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:07:09,035 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:07:10,619 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:07:10,619 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:07:13,089 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:07:14,319 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:07:15,963 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:07:19,321 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:07:21,728 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:07:23,159 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:07:24,958 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:07:26,733 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:07:29,960 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:07:31,797 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:07:33,230 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:07:34,034 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:07:35,035 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:07:36,798 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:07:40,036 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:07:42,555 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:07:43,296 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:07:45,884 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:07:47,557 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:07:51,535 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:07:53,204 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:07:53,360 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:07:56,536 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:07:58,794 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:07:58,794 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:08:01,412 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:08:02,053 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:08:03,795 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:08:04,034 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:08:08,036 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:08:09,316 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:08:11,489 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:08:13,590 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:08:14,317 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:08:18,591 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:08:19,663 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:08:21,560 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:08:23,809 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:08:24,665 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:08:28,810 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:08:30,541 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:08:30,541 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:08:30,748 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:08:33,929 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:08:34,035 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:08:35,543 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:08:39,890 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:08:40,619 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:08:40,823 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:08:44,891 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:08:47,493 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:08:50,246 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:08:50,895 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:08:52,494 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:08:55,247 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:08:57,643 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:08:57,643 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:09:00,926 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:09:00,968 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:09:02,644 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:09:04,036 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:09:06,037 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:09:08,463 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:09:11,035 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:09:11,676 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:09:13,464 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:09:16,677 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:09:18,994 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:09:21,134 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:09:21,726 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:09:23,995 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:09:26,728 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:09:28,827 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 14:09:28,828 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 14:09:28,829 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 14:09:28,831 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 14:09:29,202 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 14:09:29,540 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:09:29,540 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:09:31,214 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:09:31,833 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:09:34,036 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:09:34,541 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:09:37,038 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:09:40,214 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:09:41,289 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:09:42,510 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:09:46,677 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:09:47,511 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:09:49,344 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:09:51,679 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:09:52,873 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:09:57,087 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:09:57,088 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:09:58,386 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:09:59,415 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:10:02,088 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:10:03,387 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:10:04,037 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:10:07,548 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:10:08,770 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:10:09,482 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:10:12,549 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:10:13,771 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:10:18,241 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:10:19,557 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:10:19,558 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:10:23,242 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:10:24,560 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:10:28,287 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:10:29,626 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:10:30,516 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:10:34,038 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:10:34,687 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:10:36,040 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:10:37,676 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:10:39,691 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:10:41,127 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:10:45,473 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:10:45,473 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:10:46,781 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:10:47,743 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:10:50,477 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:10:51,782 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:10:56,087 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:10:56,088 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:10:56,797 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:10:57,488 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:11:01,134 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:11:02,489 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:11:04,042 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:11:06,852 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:11:06,914 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:11:08,256 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:11:13,258 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:11:13,535 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:11:16,921 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:11:18,536 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:11:18,883 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:11:23,901 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:11:24,412 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:11:26,991 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:11:29,101 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:11:29,413 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:11:34,041 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:11:34,458 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:11:34,745 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:11:37,060 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:11:39,459 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:11:39,746 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:11:45,028 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:11:45,028 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:11:45,449 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:11:47,132 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:11:50,029 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:11:50,450 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:11:55,184 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:11:55,571 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:11:56,111 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:12:00,573 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:12:01,112 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:12:04,042 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:12:05,252 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:12:06,422 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:12:07,042 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:12:12,007 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:12:12,008 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:12:12,008 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:12:12,044 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:12:15,313 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:12:17,009 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:12:17,243 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:12:22,245 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:12:22,646 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:12:25,377 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:12:27,647 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:12:27,756 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:12:33,384 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:12:33,663 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:12:34,042 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:12:35,444 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:12:38,385 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:12:39,043 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:12:42,243 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 14:12:42,244 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 14:12:42,245 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 14:12:42,247 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 14:12:42,497 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 14:12:43,502 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:12:43,872 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:12:44,249 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:12:49,250 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:12:50,366 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:12:53,570 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:12:54,503 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:12:55,367 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:12:59,504 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:13:00,709 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:13:00,710 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:13:03,635 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:13:04,043 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:13:05,043 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:13:05,711 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:13:10,045 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:13:11,261 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:13:11,690 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:13:15,519 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:13:16,262 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:13:20,520 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:13:21,389 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:13:21,760 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:13:25,540 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:13:26,390 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:13:31,338 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:13:31,822 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:13:31,898 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:13:34,043 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:13:36,902 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:13:37,044 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:13:42,009 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:13:42,010 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:13:42,045 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:13:43,896 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:13:47,142 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:13:47,696 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:13:52,974 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:13:53,658 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:13:53,960 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:13:57,975 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:13:58,659 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:14:03,021 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:14:03,047 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:14:03,728 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:14:04,044 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:14:08,730 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:14:09,045 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:14:13,085 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:14:14,013 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:14:15,013 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:14:20,014 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:14:20,761 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:14:23,159 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:14:25,129 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:14:25,762 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:14:30,131 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:14:30,980 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:14:30,982 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:14:33,224 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:14:34,044 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:14:35,983 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:14:36,045 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:14:41,047 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:14:41,790 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:14:43,287 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:14:46,050 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:14:46,791 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:14:51,052 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:14:52,608 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:14:53,355 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:14:56,784 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:14:57,609 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:15:02,292 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:15:03,039 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:15:03,426 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:15:04,045 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:15:08,047 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:15:09,679 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:15:13,486 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:15:13,836 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:15:14,680 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:15:18,838 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:15:20,540 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:15:23,560 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:15:24,759 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:15:25,541 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:15:29,760 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:15:31,193 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:15:31,193 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:15:31,614 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:15:34,045 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:15:35,046 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:15:36,194 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:15:40,047 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:15:41,678 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:15:42,013 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:15:42,014 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:15:45,410 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:15:47,117 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:15:51,217 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:15:51,739 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:15:52,996 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:15:56,218 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:15:58,323 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:15:58,323 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:15:58,323 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:15:59,777 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 14:15:59,779 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 14:15:59,779 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 14:15:59,789 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 14:15:59,799 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 14:16:01,791 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:16:01,813 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:16:03,324 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:16:04,046 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:16:07,048 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:16:09,135 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:16:11,874 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:16:12,383 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:16:14,158 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:16:17,385 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:16:20,062 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:16:21,986 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:16:23,278 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:16:25,063 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:16:28,280 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:16:30,039 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:16:30,323 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:16:33,821 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:16:34,047 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:16:35,407 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:16:39,494 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:16:40,101 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:16:41,267 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:16:44,495 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:16:46,691 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:16:46,692 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:16:49,158 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:16:49,932 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:16:51,693 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:16:54,933 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:16:56,973 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:16:59,227 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:17:00,150 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:17:01,974 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:17:04,047 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:17:06,048 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:17:07,374 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:17:09,289 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:17:11,664 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:17:12,375 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:17:16,666 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:17:18,269 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:17:19,357 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:17:22,485 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:17:23,276 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:17:27,486 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:17:28,292 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:17:29,426 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:17:32,521 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:17:33,293 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:17:34,049 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:17:38,097 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:17:38,810 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:17:39,501 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:17:43,098 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:17:43,825 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:17:48,166 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:17:49,561 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:17:49,805 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:17:53,167 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:17:55,300 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:17:57,622 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:17:58,417 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:18:00,301 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:18:03,419 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:18:04,050 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:18:05,847 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:18:07,693 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:18:09,063 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:18:10,848 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:18:14,065 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:18:16,321 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:18:16,322 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:18:17,760 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:18:19,587 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:18:21,323 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:18:24,588 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:18:26,950 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:18:27,827 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:18:30,227 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:18:31,952 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:18:34,050 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:18:36,146 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:18:37,893 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:18:37,942 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:18:41,147 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:18:43,503 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:18:45,946 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:18:46,873 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:18:48,504 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:18:51,874 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:18:54,085 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:18:56,016 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:18:57,315 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:18:59,086 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:19:02,317 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:19:04,051 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:19:04,922 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:19:06,081 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:19:07,372 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:19:09,923 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:19:12,374 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:19:13,986 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 14:19:13,987 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 14:19:13,988 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 14:19:13,990 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 14:19:14,132 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 14:19:15,322 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:19:16,143 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:19:17,994 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:19:22,232 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:19:22,995 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:19:26,207 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:19:27,233 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:19:28,508 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:19:32,868 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:19:32,869 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:19:34,051 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:19:34,052 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:19:36,278 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:19:37,870 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:19:39,053 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:19:42,877 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:19:44,337 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:19:45,012 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:19:47,878 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:19:50,013 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:19:53,416 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:19:53,443 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:19:55,612 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:19:58,444 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:20:00,614 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:20:03,483 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:20:03,859 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:20:03,859 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:20:04,052 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:20:06,053 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:20:10,366 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:20:11,054 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:20:13,557 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:20:15,367 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:20:16,763 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:20:20,539 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:20:20,540 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:20:21,765 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:20:23,630 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:20:25,541 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:20:26,829 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:20:30,859 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:20:30,861 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:20:31,831 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:20:33,696 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:20:34,052 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:20:36,392 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:20:37,054 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:20:41,703 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:20:43,052 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:20:43,771 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:20:46,704 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:20:48,053 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:20:52,550 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:20:53,764 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:20:53,870 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:20:57,551 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:20:58,764 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:21:02,552 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:21:03,766 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:21:04,053 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:21:07,649 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:21:07,963 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:21:08,834 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:21:12,657 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:21:13,836 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:21:17,860 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:21:18,037 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:21:18,919 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:21:23,954 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:21:24,074 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:21:28,101 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:21:29,075 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:21:29,340 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:21:34,054 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:21:34,506 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:21:34,506 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:21:34,705 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:21:36,156 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:21:39,507 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:21:39,706 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:21:44,925 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:21:45,211 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:21:46,228 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:21:49,926 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:21:50,212 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:21:55,875 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:21:56,097 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:21:56,296 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:22:01,098 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:22:02,268 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:22:02,269 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:22:02,269 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:22:04,055 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:22:06,357 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:22:07,056 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:22:07,270 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:22:12,058 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:22:12,783 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:22:14,413 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:22:17,080 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:22:17,784 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:22:22,081 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:22:23,236 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:22:23,592 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:22:27,455 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:22:28,237 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:22:31,980 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 14:22:31,982 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 14:22:31,982 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 14:22:31,984 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 14:22:32,658 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 14:22:32,992 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:22:33,269 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:22:33,663 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:22:34,055 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:22:38,057 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:22:38,746 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:22:43,395 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:22:43,724 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:22:44,369 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:22:48,396 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:22:49,370 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:22:53,397 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:22:54,371 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:22:58,398 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:23:00,104 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:23:03,412 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:23:03,836 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:23:04,056 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:23:05,105 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:23:09,057 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:23:10,140 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:23:13,896 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:23:14,448 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:23:15,142 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:23:19,449 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:23:20,767 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:23:20,768 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:23:22,070 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:23:25,128 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:23:25,768 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:23:30,129 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:23:31,332 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:23:32,133 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:23:34,056 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:23:36,058 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:23:36,332 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:23:41,589 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:23:41,734 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:23:42,198 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:23:46,591 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:23:46,736 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:23:52,260 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:23:52,322 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:23:52,690 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:23:57,692 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:23:58,222 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:24:02,337 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:24:02,909 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:24:03,224 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:24:04,057 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:24:08,058 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:24:08,298 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:24:08,299 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:24:10,390 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:24:13,300 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:24:13,452 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:24:18,322 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:24:18,323 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:24:18,453 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:24:20,456 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:24:23,765 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:24:24,009 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:24:29,370 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:24:29,554 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:24:30,523 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:24:34,057 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:24:34,371 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:24:35,058 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:24:39,402 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:24:40,598 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:24:40,756 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:24:44,403 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:24:45,757 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:24:49,884 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:24:50,662 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:24:51,077 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:24:56,079 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:24:56,233 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:24:57,715 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:25:01,234 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:25:01,456 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:25:04,059 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:25:06,627 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:25:06,628 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:25:06,830 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:25:07,900 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:25:11,628 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:25:11,831 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:25:17,049 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:25:17,323 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:25:17,969 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:25:22,050 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:25:22,323 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:25:27,987 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:25:28,037 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:25:28,216 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:25:33,217 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:25:34,060 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:25:34,442 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:25:34,443 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:25:38,099 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:25:38,698 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:25:39,444 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:25:43,700 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:25:44,843 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:25:46,179 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:25:49,050 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:25:49,844 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:25:54,051 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:25:54,210 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 14:25:54,212 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 14:25:54,213 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 14:25:54,216 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 14:25:54,246 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 14:25:55,063 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:25:56,259 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:25:59,218 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:26:00,070 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:26:04,060 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:26:05,061 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:26:05,443 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:26:06,320 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:26:10,061 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:26:10,816 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:26:15,180 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:26:16,380 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:26:16,816 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:26:20,181 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:26:22,266 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:26:22,266 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:26:24,438 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:26:25,472 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:26:27,267 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:26:30,473 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:26:32,541 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:26:34,060 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:26:34,517 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:26:36,070 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:26:37,542 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:26:41,071 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:26:42,868 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:26:44,584 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:26:46,149 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:26:47,869 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:26:51,150 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:26:53,469 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:26:53,470 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:26:54,657 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:26:56,761 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:26:58,472 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:27:02,405 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:27:02,706 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:27:04,061 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:27:04,283 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:27:08,062 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:27:09,616 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:27:09,616 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:27:12,774 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:27:13,972 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:27:14,617 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:27:18,974 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:27:20,167 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:27:22,835 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:27:24,423 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:27:25,168 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:27:29,424 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:27:30,527 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:27:32,918 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:27:34,061 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:27:35,062 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:27:35,528 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:27:39,965 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:27:40,494 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:27:41,247 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:27:45,495 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:27:46,249 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:27:50,055 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:27:51,245 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:27:51,470 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:27:56,247 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:27:58,017 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:28:00,120 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:28:01,289 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:28:03,018 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:28:04,062 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:28:07,063 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:28:08,596 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:28:08,596 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:28:10,189 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:28:12,852 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:28:13,597 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:28:17,854 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:28:19,076 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:28:20,252 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:28:23,311 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:28:24,076 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:28:28,879 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:28:29,615 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:28:30,317 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:28:33,880 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:28:34,062 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:28:36,184 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:28:36,184 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:28:36,185 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:28:38,372 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:28:39,479 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:28:41,185 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:28:44,480 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:28:46,639 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:28:48,440 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:28:49,899 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:28:51,640 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:28:54,901 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:28:56,923 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:28:58,496 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:29:00,114 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:29:01,924 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:29:04,064 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:29:06,066 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:29:07,185 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:29:07,484 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 14:29:07,485 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 14:29:07,486 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 14:29:07,488 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 14:29:07,556 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 14:29:08,563 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:29:11,490 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:29:12,428 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:29:16,492 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:29:17,831 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:29:18,625 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:29:22,094 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:29:22,832 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:29:28,006 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:29:28,593 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:29:28,699 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:29:33,007 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:29:34,065 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:29:35,143 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:29:38,307 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:29:38,764 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:29:40,145 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:29:43,308 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:29:45,785 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:29:48,830 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:29:48,990 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:29:50,786 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:29:53,992 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:29:56,143 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:29:56,144 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:29:56,887 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:29:59,405 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:30:01,144 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:30:04,065 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:30:04,648 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:30:06,185 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:30:06,186 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:30:06,958 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:30:09,649 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:30:11,462 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:30:15,328 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:30:17,017 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:30:17,149 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:30:20,334 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:30:22,363 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:30:22,364 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:30:25,069 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:30:25,539 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:30:27,364 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:30:30,541 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:30:32,879 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:30:34,072 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:30:35,135 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:30:36,072 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:30:37,880 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:30:41,074 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:30:43,252 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:30:44,190 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:30:46,463 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:30:48,253 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:30:51,464 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:30:53,366 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:30:54,251 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:30:57,221 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:30:58,934 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:31:02,222 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:31:04,066 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:31:04,315 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:31:04,559 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:31:08,067 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:31:11,115 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:31:13,375 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:31:14,390 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:31:16,117 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:31:18,376 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:31:21,663 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:31:22,440 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:31:23,929 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:31:26,664 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:31:28,930 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:31:31,899 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:31:34,066 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:31:34,067 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:31:34,512 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:31:36,900 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:31:39,068 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:31:41,941 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:31:41,942 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:31:42,566 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:31:44,093 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:31:46,953 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:31:49,095 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:31:52,295 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:31:52,623 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:31:54,546 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:31:57,296 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:31:59,547 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:32:02,612 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:32:02,684 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:32:04,078 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:32:05,079 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:32:09,005 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:32:09,006 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:32:10,080 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:32:12,750 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:32:14,006 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:32:15,265 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:32:19,815 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:32:19,971 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 14:32:19,973 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 14:32:19,973 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 14:32:19,975 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 14:32:20,364 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:32:20,803 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:32:20,803 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 14:32:22,811 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:32:24,816 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:32:26,205 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:32:29,820 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:32:31,206 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:32:34,068 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:32:34,818 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:32:37,069 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:32:39,818 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:32:42,069 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:32:44,819 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:32:47,070 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:32:49,820 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:32:52,071 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:32:54,820 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:32:57,072 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:32:59,821 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:33:02,072 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:33:04,072 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:33:04,822 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:33:07,073 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:33:09,823 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:33:12,074 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:33:14,823 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:33:17,074 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:33:19,824 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:33:22,075 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:33:24,825 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:33:27,076 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:33:29,826 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:33:32,076 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:33:34,071 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:33:34,827 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:33:38,072 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:33:39,828 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:33:43,073 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:33:44,829 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:33:48,074 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:33:49,830 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:33:53,075 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:33:54,830 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:33:58,075 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:33:59,831 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:34:03,076 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:34:04,072 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:34:04,832 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:34:09,073 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:34:09,833 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:34:14,073 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:34:14,833 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:34:19,074 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:34:19,834 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:34:24,075 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:34:24,835 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:34:29,076 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:34:29,836 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:34:34,072 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:34:34,837 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:34:35,073 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:34:39,837 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:34:40,074 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:34:44,838 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:34:45,074 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:34:49,839 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:34:50,075 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:34:54,840 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:34:55,076 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:34:59,841 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:35:00,077 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:35:04,073 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:35:04,842 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:35:06,073 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:35:09,843 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:35:11,074 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:35:14,844 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:35:16,075 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:35:19,844 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:35:21,075 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:35:24,845 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:35:26,076 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:35:29,846 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:35:31,077 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:35:34,073 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:35:34,853 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:35:36,083 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:35:39,854 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:35:41,099 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:35:44,855 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:35:46,100 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:35:49,856 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:35:51,101 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:35:54,857 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:35:56,101 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:35:59,858 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:36:01,102 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:36:04,074 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:36:04,861 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:36:07,075 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:36:09,862 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:36:12,080 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:36:14,862 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:36:17,081 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:36:19,863 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:36:22,081 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:36:24,866 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:36:27,082 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:36:29,866 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:36:32,083 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:36:34,074 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:36:34,867 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:36:38,075 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:36:39,868 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:36:43,077 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:36:44,869 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:36:48,078 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:36:49,870 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:36:53,078 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:36:54,871 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:36:58,079 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:36:59,872 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:37:03,080 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:37:04,075 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:37:04,873 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:37:09,076 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:37:09,874 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:37:14,076 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:37:14,874 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:37:19,077 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:37:19,875 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:37:24,078 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:37:24,876 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:37:29,079 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:37:29,877 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:37:34,075 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:37:34,877 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:37:35,076 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:37:39,878 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:37:40,077 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:37:44,879 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:37:45,077 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:37:49,880 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:37:50,078 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:37:54,881 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:37:55,079 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:37:59,882 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:38:00,079 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:38:04,076 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:38:04,883 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:38:06,076 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:38:09,883 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:38:11,077 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:38:14,884 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:38:16,078 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:38:19,885 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:38:21,079 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:38:24,886 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:38:26,079 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:38:29,887 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:38:31,080 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:38:34,076 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:38:34,888 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:38:37,077 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:38:39,888 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:38:42,078 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:38:44,889 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:38:47,078 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:38:49,890 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:38:52,079 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:38:54,891 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:38:57,080 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:38:59,892 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:39:02,081 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:39:04,077 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:39:04,893 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:39:08,079 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:39:09,893 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:39:13,080 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:39:14,894 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:39:18,080 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:39:19,895 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:39:23,081 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:39:24,896 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:39:28,082 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:39:29,897 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:39:33,082 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:39:34,077 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:39:34,898 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:39:39,078 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:39:39,898 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:39:44,079 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:39:44,899 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:39:49,080 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:39:49,900 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:39:54,080 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:39:54,901 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:39:59,081 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:39:59,902 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:40:04,078 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:40:04,903 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:40:05,079 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:40:09,904 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:40:10,080 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:40:14,904 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:40:15,081 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:40:19,905 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:40:20,082 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:40:24,906 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:40:25,082 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:40:29,907 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:40:30,083 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:40:34,078 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:40:34,908 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:40:36,079 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:40:39,909 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:40:41,080 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:40:44,910 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:40:46,081 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:40:49,911 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:40:51,082 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:40:54,911 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:40:56,082 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:40:59,912 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:41:01,083 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:41:04,079 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:41:04,913 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:41:07,080 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:41:09,914 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:41:12,080 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:41:14,915 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:41:17,081 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:41:19,915 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:41:22,081 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:41:24,916 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:41:27,082 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:41:29,917 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:41:32,083 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:41:34,079 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:41:34,918 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:41:38,080 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:41:39,919 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:41:43,081 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:41:44,920 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:41:48,081 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:41:49,921 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:41:53,082 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:41:54,921 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:41:58,083 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:41:59,922 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:42:03,084 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:42:04,080 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:42:04,923 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:42:09,081 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:42:09,924 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:42:14,082 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:42:14,925 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:42:15,603 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 14:42:15,607 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 14:42:15,607 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 14:42:15,611 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 14:42:16,556 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 14:42:17,563 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:42:19,613 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:42:19,926 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:42:24,614 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:42:24,927 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:42:29,615 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:42:29,928 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:42:34,080 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:42:34,929 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:42:35,081 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:42:39,988 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:42:40,712 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:42:41,724 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:42:45,713 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:42:46,007 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:42:50,713 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:42:51,829 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:42:55,714 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:42:57,034 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:43:00,715 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:43:03,072 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:43:04,081 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:43:06,081 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:43:08,667 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:43:11,082 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:43:13,903 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:43:16,083 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:43:19,455 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:43:21,084 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:43:25,656 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:43:25,657 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:43:25,657 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:43:26,725 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:43:26,972 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:43:27,979 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:43:31,726 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:43:34,081 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:43:37,082 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:43:39,728 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:43:41,114 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:43:42,777 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:43:47,777 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:43:52,778 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:43:52,800 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:43:57,779 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:44:02,780 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:44:04,082 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:44:06,774 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:44:06,775 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:44:07,799 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:44:09,245 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:44:11,775 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:44:12,800 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:44:16,776 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:44:17,801 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:44:21,777 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:44:22,801 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:44:26,778 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:44:27,802 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:44:31,778 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:44:32,803 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:44:34,084 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:44:36,779 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:44:38,085 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:44:41,780 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:44:43,085 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:44:46,781 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:44:48,086 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:44:51,782 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:44:53,087 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:44:56,782 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:44:58,088 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:45:01,783 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:45:03,452 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:45:04,019 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:45:04,083 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:45:06,784 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:45:09,085 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:45:13,114 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:45:14,086 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:45:15,529 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:45:18,115 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:45:19,794 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:45:24,090 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:45:24,091 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:45:24,795 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:45:27,601 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:45:29,153 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:45:30,703 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:45:34,086 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:45:34,254 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:45:36,669 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:45:37,682 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:45:39,255 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:45:41,670 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:45:45,243 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:45:47,618 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:45:47,756 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:45:50,266 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:45:52,620 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:45:55,643 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:45:55,643 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:45:57,830 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:45:58,038 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:46:00,644 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:46:03,040 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:46:04,087 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:46:06,126 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:46:07,896 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:46:08,588 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:46:11,138 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:46:13,590 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:46:16,415 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:46:17,970 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:46:19,070 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:46:21,416 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:46:24,071 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:46:26,844 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:46:26,844 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:46:29,245 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:46:30,050 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:46:31,845 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:46:34,088 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:46:35,091 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:46:36,887 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:46:38,175 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:46:40,102 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:46:41,888 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:46:45,103 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:46:47,490 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:46:50,243 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:46:50,636 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:46:52,491 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:46:55,637 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:46:58,275 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:46:58,275 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:47:00,301 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:47:01,626 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:47:03,276 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:47:04,088 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:47:07,090 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:47:09,224 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:47:09,369 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:47:12,423 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:47:14,226 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:47:17,425 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:47:19,430 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:47:19,862 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:47:22,433 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:47:25,837 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:47:27,483 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:47:28,063 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:47:30,838 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:47:33,064 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:47:34,090 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:47:36,489 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:47:37,557 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:47:38,687 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:47:41,490 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:47:43,688 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:47:47,431 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:47:47,622 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:47:49,059 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:47:52,432 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:47:54,061 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:47:57,659 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:47:57,694 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:47:59,892 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:48:02,660 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:48:04,092 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:48:05,094 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:48:07,768 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:48:08,471 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:48:10,816 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:48:13,472 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:48:15,816 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:48:18,473 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:48:20,818 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:48:23,587 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:48:23,876 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:48:25,998 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:48:28,589 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:48:30,999 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:48:31,933 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 14:48:31,938 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 14:48:31,939 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 14:48:31,943 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 14:48:31,948 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 14:48:33,962 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:48:34,091 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:48:34,445 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:48:36,093 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:48:39,447 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:48:41,094 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:48:44,026 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:48:45,197 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:48:46,457 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:48:50,788 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:48:51,458 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:48:54,094 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:48:55,789 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:48:57,066 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:49:01,627 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:49:02,876 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:49:04,093 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:49:04,157 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:49:06,628 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:49:08,094 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:49:12,179 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:49:13,453 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:49:14,220 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:49:17,180 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:49:18,454 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:49:22,274 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:49:22,873 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:49:24,054 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:49:29,056 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:49:29,417 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:49:29,418 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:49:32,332 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:49:34,092 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:49:34,094 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:49:34,418 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:49:39,095 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:49:39,710 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:49:42,391 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:49:44,768 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:49:44,868 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:49:49,870 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:49:50,634 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:49:52,445 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:49:55,322 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:49:55,635 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:50:00,323 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:50:00,855 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:50:00,856 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:50:02,506 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:50:04,093 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:50:05,857 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:50:06,094 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:50:11,095 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:50:11,295 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:50:11,562 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:50:16,158 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:50:16,764 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:50:21,160 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:50:21,824 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:50:23,632 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:50:26,546 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:50:26,825 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:50:31,547 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:50:32,421 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:50:33,702 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:50:34,094 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:50:37,095 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:50:37,422 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:50:42,573 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:50:43,065 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:50:43,770 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:50:47,574 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:50:48,066 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:50:51,822 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:50:53,360 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:50:53,514 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:50:58,515 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:50:59,824 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:51:01,894 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:51:04,094 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:51:04,095 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:51:04,825 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:51:09,096 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:51:10,452 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:51:11,957 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:51:14,708 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:51:15,454 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:51:19,710 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:51:21,079 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:51:22,026 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:51:25,583 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:51:26,080 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:51:30,584 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:51:31,512 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:51:32,082 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:51:34,095 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:51:36,096 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:51:36,513 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:51:41,537 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:51:42,150 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:51:42,288 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:51:46,538 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:51:48,852 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:51:50,265 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 14:51:50,266 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 14:51:50,267 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 14:51:50,269 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 14:51:51,215 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 14:51:52,220 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:51:52,271 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:51:53,853 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:51:57,272 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:51:59,582 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:51:59,583 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:52:02,289 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:52:02,843 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:52:04,095 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:52:04,583 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:52:08,097 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:52:10,080 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:52:10,345 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:52:13,363 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:52:15,081 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:52:18,365 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:52:20,406 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:52:20,682 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:52:23,958 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:52:25,682 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:52:28,959 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:52:30,574 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:52:30,797 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:52:30,798 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:52:34,061 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:52:34,095 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:52:35,799 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:52:39,970 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:52:40,644 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:52:41,558 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:52:44,970 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:52:47,450 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:52:50,643 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:52:50,706 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:52:52,451 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:52:55,644 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:52:57,797 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:52:57,798 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:52:59,766 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:53:00,677 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:53:02,897 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:53:04,096 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:53:06,097 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:53:07,902 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:53:09,831 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:53:11,114 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:53:12,917 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:53:16,115 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:53:18,281 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:53:19,891 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:53:21,647 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:53:23,282 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:53:26,648 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:53:28,797 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:53:29,951 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:53:31,791 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:53:33,939 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:53:34,096 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:53:37,098 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:53:39,318 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:53:40,014 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:53:42,670 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:53:44,319 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:53:47,671 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:53:49,970 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:53:50,130 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:53:53,222 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:53:54,971 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:53:58,182 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:53:58,565 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:54:00,423 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:54:03,566 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:54:04,097 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:54:06,329 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:54:08,824 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:54:10,264 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:54:11,330 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:54:13,825 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:54:16,497 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:54:18,826 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:54:20,332 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:54:21,505 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:54:23,827 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:54:27,366 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:54:28,398 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:54:29,509 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:54:32,367 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:54:34,099 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:54:35,101 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:54:37,933 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:54:38,458 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:54:40,340 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:54:42,934 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:54:45,341 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:54:48,517 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:54:48,664 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:54:50,921 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:54:55,184 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:54:55,922 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:54:58,577 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:55:00,185 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:55:01,497 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:55:04,100 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:55:05,610 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:55:06,792 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:55:06,793 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 14:55:06,794 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 14:55:06,795 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 14:55:06,798 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 14:55:07,647 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 14:55:08,651 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:55:10,611 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:55:11,805 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:55:16,523 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:55:16,523 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:55:16,807 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:55:18,719 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:55:21,524 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:55:21,948 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:55:26,572 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:55:27,847 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:55:28,793 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:55:31,573 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:55:32,848 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:55:34,102 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:55:37,291 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:55:38,481 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:55:38,859 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:55:42,292 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:55:43,483 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:55:47,786 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:55:47,787 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:55:47,931 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:55:49,063 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:55:52,788 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:55:54,065 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:55:57,798 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:55:58,000 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:55:59,718 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:56:04,144 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:56:04,592 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:56:05,146 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:56:08,074 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:56:09,593 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:56:10,882 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:56:14,856 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:56:14,856 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:56:15,883 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:56:18,151 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:56:19,857 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:56:21,288 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:56:24,993 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:56:26,290 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:56:28,227 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:56:30,380 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:56:31,395 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:56:34,144 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:56:36,271 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:56:37,145 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:56:40,292 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:56:41,272 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:56:42,816 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:56:46,306 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:56:48,671 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:56:50,362 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:56:51,307 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:56:53,672 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:56:56,984 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:56:58,420 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:56:59,216 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:57:01,985 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:57:04,145 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:57:05,147 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:57:07,750 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:57:08,486 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:57:10,916 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:57:12,751 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:57:15,918 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:57:18,409 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:57:18,555 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:57:21,730 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:57:23,410 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:57:27,406 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:57:28,677 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:57:28,913 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:57:32,407 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:57:34,147 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:57:35,237 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:57:37,937 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:57:38,748 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:57:40,239 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:57:42,938 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:57:45,708 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:57:45,708 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:57:47,994 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:57:48,818 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:57:50,709 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:57:52,995 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:57:56,230 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:57:58,453 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:57:58,877 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:58:01,231 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:58:03,456 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:58:04,148 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:58:06,417 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:58:08,596 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:58:08,946 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:58:11,417 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:58:13,597 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:58:17,323 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:58:17,323 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:58:18,727 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:58:19,018 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:58:22,330 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:58:23,729 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:58:26,619 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 14:58:26,621 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 14:58:26,621 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 14:58:26,623 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 14:58:27,071 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 14:58:27,913 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:58:27,914 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:58:28,076 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:58:29,080 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:58:29,625 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:58:33,101 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:58:34,149 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:58:35,151 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:58:38,134 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:58:38,426 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:58:40,671 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:58:43,427 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:58:45,672 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:58:48,198 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:58:48,673 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:58:50,852 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:58:55,180 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:58:55,853 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:58:58,278 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:59:00,181 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:59:01,548 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:59:04,150 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:59:05,924 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:59:07,049 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:59:08,342 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:59:10,925 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:59:12,050 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:59:16,736 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:59:16,736 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:59:18,010 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:59:18,405 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:59:21,737 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:59:23,012 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:59:26,460 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:59:27,456 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:59:28,712 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:59:33,713 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:59:33,941 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:59:34,151 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 14:59:36,527 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:59:38,942 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:59:39,701 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:59:44,480 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:59:44,481 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:59:44,481 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:59:44,702 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:59:46,599 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:59:49,482 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:59:49,728 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:59:54,729 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 14:59:54,992 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 14:59:56,656 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 14:59:59,993 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:00:00,376 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:00:04,153 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:00:05,876 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:00:05,980 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:00:06,725 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:00:10,877 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:00:10,981 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:00:16,185 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:00:16,453 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:00:16,790 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:00:21,187 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:00:21,454 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:00:26,856 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:00:26,910 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:00:27,063 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:00:31,912 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:00:32,064 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:00:34,153 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:00:36,934 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:00:37,201 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:00:37,616 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:00:42,203 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:00:43,980 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:00:46,989 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:00:47,234 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:00:48,982 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:00:52,236 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:00:54,422 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:00:57,049 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:00:57,700 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:00:59,423 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:01:02,702 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:01:04,153 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:01:05,149 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:01:07,108 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:01:08,592 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:01:10,211 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:01:13,594 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:01:15,482 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:01:17,177 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:01:19,556 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:01:21,165 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:01:24,993 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:01:26,755 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:01:27,245 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:01:29,994 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:01:32,165 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:01:32,166 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:01:34,153 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:01:35,154 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:01:35,307 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:01:37,167 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:01:40,155 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:01:42,717 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:01:44,031 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 15:01:44,032 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 15:01:44,033 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 15:01:44,035 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 15:01:44,367 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:01:44,368 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 15:01:46,036 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:01:46,377 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:01:47,718 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:01:51,038 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:01:53,493 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:01:54,432 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:01:56,737 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:01:58,494 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:02:01,738 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:02:03,937 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:02:04,154 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:02:04,490 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:02:07,168 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:02:08,938 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:02:12,170 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:02:14,482 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:02:14,482 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:02:14,615 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:02:17,621 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:02:19,503 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:02:22,622 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:02:24,679 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:02:25,446 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:02:28,374 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:02:30,448 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:02:33,375 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:02:34,154 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:02:34,742 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:02:36,343 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:02:39,156 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:02:41,344 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:02:44,157 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:02:46,526 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:02:46,526 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:02:48,832 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:02:50,047 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:02:51,527 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:02:55,057 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:02:57,330 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:02:58,896 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:03:00,651 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:03:02,331 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:03:04,158 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:03:06,161 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:03:08,081 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:03:08,956 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:03:11,568 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:03:13,127 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:03:16,570 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:03:18,270 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:03:18,271 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:03:19,022 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:03:21,574 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:03:23,271 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:03:26,576 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:03:29,079 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:03:29,158 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:03:32,383 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:03:34,159 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:03:34,160 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:03:38,014 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:03:39,151 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:03:39,849 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:03:43,015 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:03:45,131 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:03:45,132 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:03:48,353 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:03:49,224 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:03:50,133 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:03:53,354 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:03:55,383 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:03:57,279 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:03:58,568 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:04:00,384 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:04:03,569 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:04:04,160 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:04:05,780 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:04:07,339 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:04:08,992 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:04:10,780 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:04:13,993 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:04:15,875 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:04:17,401 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:04:19,026 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:04:20,876 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:04:24,028 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:04:26,531 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:04:27,470 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:04:29,693 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:04:32,937 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:04:34,170 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:04:35,171 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:04:35,529 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:04:37,938 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:04:40,171 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:04:43,211 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:04:44,581 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:04:45,454 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:04:48,212 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:04:50,456 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:04:53,794 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:04:54,641 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:04:56,080 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:04:58,795 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:05:01,081 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:05:02,942 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 15:05:02,943 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 15:05:02,944 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 15:05:02,947 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 15:05:03,696 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 15:05:04,160 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:05:04,589 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:05:04,702 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:05:06,161 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:05:09,653 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:05:11,163 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:05:14,771 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:05:15,653 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:05:16,912 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:05:21,104 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:05:21,913 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:05:24,833 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:05:26,106 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:05:27,352 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:05:31,589 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:05:31,590 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:05:32,892 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:05:32,946 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:05:34,161 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:05:36,752 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:05:38,163 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:05:42,129 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:05:42,979 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:05:43,371 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:05:47,130 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:05:48,372 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:05:52,591 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:05:53,041 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:05:53,879 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:05:57,592 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:05:58,880 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:06:02,749 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:06:03,102 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:06:04,001 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:06:04,161 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:06:09,163 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:06:09,184 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:06:13,163 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:06:14,185 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:06:14,383 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:06:19,384 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:06:19,598 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:06:19,599 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:06:21,221 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:06:24,599 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:06:24,889 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:06:30,147 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:06:30,443 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:06:31,294 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:06:34,162 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:06:35,148 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:06:36,163 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:06:40,417 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:06:41,367 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:06:41,577 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:06:45,418 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:06:46,578 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:06:51,056 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:06:51,435 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:06:52,245 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:06:56,057 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:06:57,247 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:07:01,498 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:07:01,602 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:07:02,883 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:07:04,162 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:07:07,065 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:07:08,164 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:07:09,548 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:07:12,066 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:07:13,270 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:07:17,689 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:07:18,877 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:07:19,635 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:07:22,690 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:07:23,878 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:07:28,264 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:07:29,521 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:07:29,706 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:07:33,265 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:07:34,163 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:07:35,164 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:07:38,385 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:07:38,764 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:07:40,580 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:07:43,385 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:07:45,582 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:07:48,831 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:07:49,056 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:07:51,511 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:07:55,822 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:07:56,512 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:07:58,893 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:08:00,824 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:08:02,046 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:08:04,165 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:08:06,517 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:08:06,518 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:08:07,659 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:08:08,959 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:08:11,519 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:08:12,660 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:08:16,362 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 15:08:16,363 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 15:08:16,363 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 15:08:16,367 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 15:08:17,018 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:08:17,019 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 15:08:17,164 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:08:18,368 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:08:19,028 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:08:22,165 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:08:23,370 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:08:27,081 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:08:27,692 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:08:28,975 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:08:32,700 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:08:33,976 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:08:34,166 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:08:37,147 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:08:38,508 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:08:38,509 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:08:39,600 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:08:43,994 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:08:44,602 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:08:47,218 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:08:48,997 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:08:50,291 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:08:54,584 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:08:55,842 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:08:57,282 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:08:59,585 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:09:00,842 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:09:04,166 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:09:04,939 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:09:05,334 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:09:06,167 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:09:09,940 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:09:11,168 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:09:15,390 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:09:15,557 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:09:16,717 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:09:19,411 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:09:20,597 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:09:22,081 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:09:26,501 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:09:27,460 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:09:27,852 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:09:31,503 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:09:32,854 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:09:34,166 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:09:35,540 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:09:36,508 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:09:38,439 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:09:42,780 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:09:43,440 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:09:45,606 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:09:47,781 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:09:48,983 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:09:53,597 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:09:53,598 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:09:54,808 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:09:55,681 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:09:58,599 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:09:59,809 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:10:04,030 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:10:04,167 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:10:05,168 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:10:05,748 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:10:09,030 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:10:10,169 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:10:14,940 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:10:15,860 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:10:16,140 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:10:21,141 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:10:21,323 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:10:21,324 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:10:23,134 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:10:26,325 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:10:26,553 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:10:31,554 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:10:31,564 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:10:33,192 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:10:34,167 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:10:36,565 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:10:37,169 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:10:42,148 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:10:42,305 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:10:43,264 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:10:47,148 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:10:47,315 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:10:52,671 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:10:52,955 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:10:53,610 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:10:57,957 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:10:59,102 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:11:01,662 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:11:03,307 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:11:04,104 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:11:04,168 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:11:09,169 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:11:09,264 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:11:09,265 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:11:11,718 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:11:14,266 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:11:14,345 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:11:19,346 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:11:19,672 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:11:19,672 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:11:21,791 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:11:24,688 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:11:24,984 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:11:29,311 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 15:11:29,313 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 15:11:29,314 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 15:11:29,316 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 15:11:29,843 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 15:11:30,121 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:11:30,318 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:11:30,849 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:11:31,853 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:11:34,168 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:11:35,122 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:11:36,170 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:11:40,639 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:11:41,839 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:11:41,920 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:11:46,841 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:11:46,992 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:11:49,968 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:11:51,993 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:11:52,205 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:11:57,206 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:11:57,812 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:11:59,046 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:12:02,813 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:12:03,034 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:12:04,169 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:12:08,407 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:12:08,407 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:12:08,913 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:12:09,108 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:12:13,408 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:12:13,914 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:12:19,222 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:12:19,343 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:12:19,469 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:12:24,471 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:12:25,705 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:12:29,286 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:12:30,070 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:12:30,706 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:12:34,171 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:12:35,173 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:12:36,157 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:12:36,158 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:12:37,433 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:12:40,437 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:12:41,158 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:12:45,438 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:12:46,671 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:12:47,505 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:12:50,971 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:12:51,672 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:12:56,628 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:12:57,266 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:12:57,569 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:13:01,629 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:13:02,267 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:13:04,172 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:13:06,747 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:13:07,441 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:13:07,640 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:13:11,748 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:13:13,913 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:13:17,696 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:13:17,711 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:13:18,914 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:13:22,698 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:13:24,016 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:13:24,017 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:13:27,773 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:13:28,337 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:13:29,017 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:13:33,338 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:13:34,172 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:13:34,783 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:13:37,847 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:13:39,143 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:13:39,786 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:13:44,145 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:13:45,123 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:13:47,913 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:13:49,320 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:13:50,123 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:13:54,322 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:13:55,535 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:13:55,983 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:13:59,902 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:14:00,536 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:14:04,173 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:14:05,255 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:14:06,038 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:14:06,042 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:14:10,256 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:14:11,041 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:14:15,920 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:14:16,103 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:14:16,649 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:14:20,921 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:14:22,645 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:14:22,646 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:14:25,921 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:14:26,165 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:14:27,647 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:14:30,923 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:14:33,069 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:14:34,173 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:14:35,238 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:14:36,180 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:14:38,070 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:14:41,182 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:14:43,037 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 15:14:43,039 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 15:14:43,039 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 15:14:43,042 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 15:14:43,293 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:14:43,294 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 15:14:43,891 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:14:45,302 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:14:47,044 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:14:48,892 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:14:52,045 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:14:53,355 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:14:54,533 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:14:57,908 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:14:59,534 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:15:03,425 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:15:03,441 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:15:04,174 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:15:05,149 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:15:09,175 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:15:10,554 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:15:13,490 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:15:14,681 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:15:15,555 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:15:19,683 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:15:20,908 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:15:23,562 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:15:25,416 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:15:25,910 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:15:30,417 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:15:31,603 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:15:31,609 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:15:34,174 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:15:36,175 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:15:36,604 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:15:41,177 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:15:41,843 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:15:43,691 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:15:46,459 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:15:46,844 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:15:51,933 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:15:52,181 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:15:53,848 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:15:56,933 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:15:57,182 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:16:01,898 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:16:02,447 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:16:02,823 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:16:04,175 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:16:07,448 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:16:08,176 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:16:11,963 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:16:13,173 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:16:13,646 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:16:18,647 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:16:19,052 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:16:22,028 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:16:24,054 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:16:24,386 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:16:29,387 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:16:30,006 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:16:32,098 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:16:34,175 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:16:35,007 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:16:35,176 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:16:40,177 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:16:40,673 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:16:40,674 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:16:44,176 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:16:45,274 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:16:45,674 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:16:50,275 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:16:51,105 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:16:54,245 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:16:55,480 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:16:56,107 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:17:00,481 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:17:01,108 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:17:04,177 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:17:05,675 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:17:06,209 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:17:08,320 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:17:10,676 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:17:11,210 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:17:15,677 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:17:17,138 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:17:18,388 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:17:20,867 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:17:22,187 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:17:25,868 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:17:27,358 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:17:27,453 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:17:31,075 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:17:32,361 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:17:34,178 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:17:36,824 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:17:37,521 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:17:37,534 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:17:41,825 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:17:44,648 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:17:46,862 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:17:47,584 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:17:49,649 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:17:51,862 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:17:55,161 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:17:55,162 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:17:57,420 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:17:57,650 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:18:00,162 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:18:02,421 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:18:04,179 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:18:05,197 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:18:05,461 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 15:18:05,463 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 15:18:05,463 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 15:18:05,465 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 15:18:05,700 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 15:18:07,467 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:18:07,712 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:18:10,198 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:18:12,468 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:18:15,757 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:18:15,769 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:18:17,983 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:18:20,758 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:18:22,985 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:18:25,836 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:18:26,559 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:18:28,906 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:18:31,560 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:18:33,907 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:18:34,179 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:18:36,027 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:18:37,212 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:18:39,019 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:18:43,106 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:18:44,020 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:18:46,099 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:18:48,107 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:18:49,201 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:18:53,259 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:18:53,260 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:18:54,154 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:18:54,433 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:18:58,260 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:18:59,434 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:19:03,655 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:19:04,179 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:19:04,209 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:19:05,180 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:19:08,656 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:19:10,181 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:19:14,267 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:19:14,271 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:19:15,497 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:19:19,270 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:19:20,499 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:19:24,338 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:19:25,213 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:19:26,444 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:19:30,970 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:19:31,445 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:19:34,180 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:19:34,400 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:19:35,971 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:19:37,181 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:19:41,468 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:19:41,469 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:19:42,741 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:19:44,463 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:19:46,469 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:19:47,763 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:19:51,957 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:19:52,515 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:19:53,227 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:19:56,958 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:19:58,228 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:20:02,581 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:20:02,598 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:20:03,913 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:20:04,180 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:20:07,600 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:20:09,182 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:20:11,635 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:20:13,255 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:20:14,537 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:20:19,270 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:20:19,539 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:20:21,693 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:20:24,271 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:20:25,504 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:20:29,757 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:20:31,081 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:20:31,764 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:20:34,181 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:20:34,758 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:20:36,182 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:20:40,445 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:20:41,725 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:20:41,830 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:20:45,446 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:20:46,726 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:20:49,885 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:20:50,895 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:20:52,113 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:20:55,897 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:20:57,115 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:20:59,953 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:21:01,625 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:21:02,830 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:21:04,181 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:21:07,229 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:21:08,185 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:21:10,013 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:21:12,230 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:21:13,382 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:21:17,589 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:21:17,856 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 15:21:17,858 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 15:21:17,858 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 15:21:17,861 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 15:21:18,053 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 15:21:18,863 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:21:20,063 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:21:22,590 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:21:23,864 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:21:27,958 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:21:28,120 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:21:29,165 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:21:32,959 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:21:34,167 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:21:34,183 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:21:38,187 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:21:38,446 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:21:39,745 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:21:43,476 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:21:44,747 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:21:48,257 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:21:49,015 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:21:50,206 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:21:54,436 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:21:54,436 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:21:55,207 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:21:58,321 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:21:59,437 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:22:00,689 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:22:04,184 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:22:04,567 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:22:05,850 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:22:08,380 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:22:09,568 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:22:10,851 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:22:15,374 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:22:16,436 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:22:16,480 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:22:20,375 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:22:21,482 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:22:25,437 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:22:26,504 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:22:26,736 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:22:30,568 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:22:31,737 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:22:34,184 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:22:35,806 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:22:36,575 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:22:37,185 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:22:40,808 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:22:42,186 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:22:45,829 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:22:46,645 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:22:47,255 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:22:50,830 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:22:52,257 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:22:55,931 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:22:56,713 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:22:57,342 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:23:02,344 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:23:02,379 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:23:04,184 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:23:05,806 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:23:07,381 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:23:08,185 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:23:12,966 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:23:12,967 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:23:13,187 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:23:15,868 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:23:17,968 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:23:18,417 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:23:23,674 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:23:23,931 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:23:23,935 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:23:28,675 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:23:28,931 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:23:33,999 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:23:34,005 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:23:34,185 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:23:34,185 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:23:39,187 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:23:40,535 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:23:40,535 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:23:40,536 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:23:44,075 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:23:44,821 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:23:45,536 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:23:49,823 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:23:50,694 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:23:54,137 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:23:54,980 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:23:55,695 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:23:59,982 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:24:01,318 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:24:04,185 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:24:04,204 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:24:05,186 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:24:06,319 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:24:10,187 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:24:11,536 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:24:12,259 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:24:15,836 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:24:16,678 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:24:21,449 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:24:22,189 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:24:22,331 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:24:26,450 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:24:28,607 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:24:28,608 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:24:29,903 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 15:24:29,904 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 15:24:29,905 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 15:24:29,907 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 15:24:30,390 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 15:24:31,918 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:24:32,401 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:24:33,609 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:24:34,186 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:24:37,187 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:24:38,828 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:24:42,250 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:24:42,461 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:24:43,829 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:24:47,251 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:24:49,334 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:24:52,483 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:24:52,535 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:24:54,335 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:24:57,485 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:24:59,622 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:25:00,591 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:25:02,821 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:25:04,186 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:25:04,622 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:25:08,188 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:25:10,132 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:25:10,653 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:25:13,410 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:25:15,133 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:25:19,391 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:25:20,206 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:25:20,720 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:25:24,392 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:25:25,208 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:25:30,160 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:25:30,788 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:25:30,878 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:25:34,187 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:25:35,189 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:25:36,361 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:25:40,620 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:25:40,848 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:25:41,362 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:25:45,622 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:25:46,875 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:25:46,876 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:25:48,897 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:25:51,059 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:25:51,877 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:25:56,060 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:25:56,878 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:25:58,960 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:26:01,504 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:26:02,250 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:26:04,189 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:26:06,790 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:26:07,610 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:26:08,032 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:26:11,791 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:26:12,612 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:26:16,859 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:26:18,149 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:26:18,595 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:26:21,860 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:26:24,018 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:26:26,213 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:26:27,242 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:26:29,019 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:26:32,244 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:26:34,189 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:26:34,555 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:26:36,276 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:26:37,828 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:26:39,556 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:26:42,830 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:26:44,950 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:26:44,951 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:26:46,340 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:26:48,148 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:26:49,952 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:26:53,149 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:26:55,365 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:26:56,400 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:26:58,659 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:27:01,860 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:27:01,860 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:27:04,131 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:27:04,189 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:27:04,449 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:27:06,861 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:27:09,191 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:27:12,338 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:27:14,521 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:27:14,615 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:27:17,339 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:27:19,616 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:27:23,113 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:27:24,577 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:27:25,286 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:27:28,114 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:27:30,288 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:27:33,634 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:27:34,191 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:27:34,654 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:27:36,192 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:27:40,102 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:27:41,193 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:27:41,286 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 15:27:41,289 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 15:27:41,290 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 15:27:41,292 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 15:27:41,703 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 15:27:42,707 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:27:45,104 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:27:46,295 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:27:50,578 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:27:51,822 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:27:52,822 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:27:55,580 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:27:56,823 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:28:01,470 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:28:01,470 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:28:01,824 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:28:04,190 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:28:04,913 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:28:06,471 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:28:07,192 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:28:11,605 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:28:12,864 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:28:14,979 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:28:16,606 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:28:17,865 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:28:22,000 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:28:23,035 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:28:23,129 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:28:27,000 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:28:28,130 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:28:32,306 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:28:33,099 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:28:33,559 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:28:34,191 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:28:38,641 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:28:39,192 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:28:43,164 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:28:43,642 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:28:44,821 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:28:49,558 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:28:50,803 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:28:52,216 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:28:54,559 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:28:55,804 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:29:00,306 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:29:00,832 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:29:02,273 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:29:04,192 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:29:05,428 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:29:06,193 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:29:10,329 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:29:11,070 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:29:11,353 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:29:16,355 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:29:17,437 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:29:17,437 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:29:20,388 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:29:21,541 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:29:22,438 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:29:26,543 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:29:27,709 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:29:30,449 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:29:31,974 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:29:32,710 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:29:34,194 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:29:37,194 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:29:38,025 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:29:38,507 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:29:42,195 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:29:43,026 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:29:47,994 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:29:48,571 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:29:48,775 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:29:53,000 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:29:53,776 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:29:58,001 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:29:59,221 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:30:03,001 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:30:04,193 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:30:04,237 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:30:08,195 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:30:09,526 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:30:10,682 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:30:13,388 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:30:14,527 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:30:18,389 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:30:20,029 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:30:20,743 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:30:23,665 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:30:25,030 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:30:28,666 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:30:30,775 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:30:30,809 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:30:34,194 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:30:34,195 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:30:36,000 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:30:39,995 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:30:40,897 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:30:41,670 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:30:44,996 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:30:47,898 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:30:50,553 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:30:50,966 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:30:52,899 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:30:55,554 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:30:57,919 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:31:01,039 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:31:01,233 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:31:02,920 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:31:04,194 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:31:07,196 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:31:08,504 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:31:08,813 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 15:31:08,814 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 15:31:08,815 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 15:31:08,817 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 15:31:09,087 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 15:31:10,093 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:31:11,097 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:31:12,819 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:31:13,505 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:31:17,820 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:31:18,951 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:31:18,951 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:31:21,160 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:31:23,117 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:31:23,951 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:31:29,110 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:31:29,858 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:31:31,226 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:31:34,110 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:31:34,195 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:31:34,859 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:31:39,209 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:31:40,702 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:31:45,080 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:31:45,304 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:31:45,707 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:31:50,082 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:31:51,313 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:31:53,354 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:31:55,535 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:31:56,314 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:32:00,536 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:32:01,844 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:32:02,410 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:32:04,196 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:32:06,197 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:32:06,847 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:32:11,787 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:32:12,495 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:32:12,512 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:32:16,788 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:32:17,513 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:32:22,691 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:32:22,717 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:32:23,427 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:32:27,692 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:32:29,705 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:32:32,794 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:32:32,874 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:32:34,196 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:32:34,706 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:32:38,198 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:32:40,148 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:32:42,851 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:32:43,396 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:32:45,149 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:32:48,398 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:32:50,618 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:32:50,911 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:32:53,878 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:32:55,619 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:32:58,880 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:33:00,971 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:33:01,048 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:33:04,197 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:33:04,198 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:33:07,545 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:33:09,797 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:33:11,041 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:33:12,546 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:33:14,850 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:33:17,551 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:33:17,552 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:33:19,896 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:33:21,109 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:33:22,553 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:33:24,897 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:33:28,481 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:33:30,648 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:33:31,173 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:33:33,482 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:33:34,197 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:33:36,199 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:33:38,965 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:33:41,201 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:33:41,239 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:33:43,966 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:33:46,202 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:33:49,291 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:33:49,376 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:33:51,601 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:33:55,773 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:33:56,602 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:33:59,352 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:34:00,774 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:34:01,937 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:34:04,200 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:34:06,287 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:34:06,287 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:34:07,695 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:34:09,470 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:34:11,288 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:34:12,695 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:34:16,755 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:34:18,010 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:34:19,529 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:34:21,756 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:34:23,011 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:34:27,227 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 15:34:27,228 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 15:34:27,229 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 15:34:27,231 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 15:34:27,382 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:34:27,594 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 15:34:28,233 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:34:28,600 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:34:32,383 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:34:33,234 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:34:34,200 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:34:37,580 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:34:38,683 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:34:38,862 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:34:42,581 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:34:43,863 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:34:48,032 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:34:48,746 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:34:49,301 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:34:53,037 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:34:54,303 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:34:56,800 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:34:58,636 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:34:59,902 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:35:04,202 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:35:04,470 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:35:05,203 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:35:06,865 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:35:09,472 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:35:10,666 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:35:14,589 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:35:15,668 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:35:16,922 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:35:19,590 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:35:20,894 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:35:24,900 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:35:26,374 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:35:26,998 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:35:29,909 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:35:31,374 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:35:34,203 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:35:35,038 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:35:35,038 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:35:37,074 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:35:37,271 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:35:40,039 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:35:42,273 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:35:45,705 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:35:47,141 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:35:47,992 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:35:50,705 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:35:52,994 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:35:55,215 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:35:56,117 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:35:58,267 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:36:02,646 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:36:03,269 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:36:04,203 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:36:05,289 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:36:07,691 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:36:09,204 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:36:13,673 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:36:14,205 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:36:15,350 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:36:18,675 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:36:19,249 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:36:23,820 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:36:23,820 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:36:24,250 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:36:25,417 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:36:28,821 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:36:29,330 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:36:34,038 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:36:34,043 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:36:34,203 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:36:35,204 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:36:35,479 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:36:39,623 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:36:40,205 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:36:45,401 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:36:45,552 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:36:45,680 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:36:50,681 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:36:51,930 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:36:55,623 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:36:56,561 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:36:56,931 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:37:01,563 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:37:02,162 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:37:04,204 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:37:05,680 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:37:07,163 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:37:07,205 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:37:12,213 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:37:12,400 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:37:15,745 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:37:17,401 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:37:17,624 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:37:22,716 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:37:22,717 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:37:22,804 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:37:22,898 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:37:27,717 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:37:27,899 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:37:32,873 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:37:33,042 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:37:33,218 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:37:34,205 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:37:39,207 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:37:39,270 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:37:40,423 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 15:37:40,425 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 15:37:40,425 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 15:37:40,428 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 15:37:40,923 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:37:40,924 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 15:37:44,271 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:37:44,430 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:37:49,432 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:37:50,371 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:37:50,371 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:37:50,372 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:37:53,000 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:37:54,694 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:37:55,372 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:37:59,696 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:38:00,747 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:38:03,082 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:38:04,206 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:38:05,207 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:38:05,748 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:38:10,967 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:38:11,119 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:38:11,144 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:38:15,968 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:38:16,119 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:38:21,202 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:38:21,372 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:38:21,909 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:38:26,910 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:38:28,265 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:38:31,262 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:38:32,665 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:38:33,266 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:38:34,206 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:38:38,208 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:38:38,749 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:38:38,750 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:38:41,324 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:38:43,751 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:38:43,969 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:38:49,159 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:38:49,386 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:38:49,405 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:38:54,160 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:38:54,406 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:38:59,447 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:38:59,566 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:38:59,763 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:39:04,206 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:39:04,567 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:39:05,208 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:39:09,509 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:39:09,696 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:39:10,994 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:39:15,995 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:39:16,207 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:39:19,572 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:39:21,208 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:39:21,527 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:39:26,529 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:39:26,657 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:39:26,657 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:39:29,645 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:39:31,658 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:39:31,786 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:39:34,209 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:39:36,696 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:39:37,497 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:39:37,701 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:39:42,198 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:39:42,497 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:39:47,644 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:39:47,779 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:39:47,969 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:39:52,645 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:39:52,970 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:39:57,842 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:39:57,872 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:39:58,050 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:40:02,882 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:40:03,051 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:40:04,209 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:40:07,914 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:40:08,539 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:40:08,850 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:40:13,851 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:40:14,164 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:40:16,965 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:40:19,166 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:40:19,452 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:40:24,453 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:40:24,911 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:40:24,911 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:40:27,030 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:40:29,912 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:40:30,179 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:40:34,210 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:40:35,537 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:40:35,849 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:40:37,094 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:40:40,538 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:40:40,850 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:40:45,670 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:40:46,016 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:40:47,166 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:40:50,675 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:40:51,017 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:40:54,695 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 15:40:54,697 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 15:40:54,698 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 15:40:54,700 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 15:40:55,233 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:40:55,234 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 15:40:56,197 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:40:56,197 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:40:56,702 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:40:57,242 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:41:01,199 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:41:01,703 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:41:04,210 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:41:05,305 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:41:07,093 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:41:07,255 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:41:12,257 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:41:12,455 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:41:12,456 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:41:15,370 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:41:17,467 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:41:17,859 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:41:22,860 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:41:23,197 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:41:23,198 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:41:25,439 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:41:28,334 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:41:28,529 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:41:33,478 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:41:33,531 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:41:34,210 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:41:35,515 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:41:38,479 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:41:39,212 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:41:43,973 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:41:44,338 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:41:45,577 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:41:48,975 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:41:49,339 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:41:54,158 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:41:54,565 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:41:55,643 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:41:59,159 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:41:59,566 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:42:04,211 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:42:04,954 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:42:05,212 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:42:05,708 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:42:09,956 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:42:10,213 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:42:15,689 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:42:15,802 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:42:16,001 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:42:21,003 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:42:21,937 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:42:25,867 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:42:26,088 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:42:26,939 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:42:31,089 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:42:32,348 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:42:34,235 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:42:35,934 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:42:36,236 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:42:37,358 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:42:41,238 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:42:42,421 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:42:42,422 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:42:45,994 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:42:46,893 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:42:47,422 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:42:51,894 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:42:53,161 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:42:54,046 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:42:57,410 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:42:58,162 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:43:02,986 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:43:03,721 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:43:04,118 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:43:04,224 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:43:08,225 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:43:08,770 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:43:13,234 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:43:14,043 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:43:14,767 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:43:19,044 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:43:19,768 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:43:24,045 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:43:24,769 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:43:29,046 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:43:34,047 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:43:34,225 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:43:34,485 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:43:39,226 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:43:39,486 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:43:44,227 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:43:44,487 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:43:49,228 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:43:49,491 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:43:54,229 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:43:54,496 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:43:59,229 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:43:59,504 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:44:04,225 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:44:04,507 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:44:05,226 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:44:10,163 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:44:10,164 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:44:10,228 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:44:13,684 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:44:15,164 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:44:15,951 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:44:20,952 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:44:21,202 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:44:23,750 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:44:26,203 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:44:26,470 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:44:31,472 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:44:31,711 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:44:33,869 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:44:34,226 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:44:36,712 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:44:37,227 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:44:42,226 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:44:42,315 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:44:43,939 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:44:47,227 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:44:47,316 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:44:52,410 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:44:52,992 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:44:54,003 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:44:57,411 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:44:57,994 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:45:01,048 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 15:45:01,049 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 15:45:01,050 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 15:45:01,053 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 15:45:01,060 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 15:45:02,071 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:45:03,055 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:45:03,080 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:45:03,652 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:45:04,226 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:45:08,228 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:45:09,307 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:45:09,307 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:45:09,308 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:45:12,137 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:45:13,562 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:45:14,308 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:45:18,564 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:45:19,820 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:45:22,201 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:45:24,061 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:45:24,821 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:45:29,062 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:45:30,572 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:45:32,267 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:45:34,227 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:45:34,228 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:45:35,573 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:45:39,229 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:45:40,963 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:45:41,331 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:45:44,539 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:45:45,987 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:45:50,183 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:45:51,393 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:45:51,889 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:45:55,183 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:45:57,346 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:45:57,347 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:45:59,456 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:46:00,581 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:46:02,348 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:46:04,227 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:46:06,228 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:46:07,970 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:46:09,521 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:46:12,167 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:46:12,971 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:46:17,169 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:46:18,608 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:46:19,576 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:46:22,904 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:46:23,683 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:46:28,846 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:46:29,556 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:46:29,656 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:46:33,847 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:46:34,228 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:46:34,558 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:46:39,472 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:46:39,734 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:46:40,308 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:46:44,473 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:46:45,384 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:46:49,475 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:46:51,039 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:46:53,811 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:46:55,352 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:46:56,040 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:47:00,354 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:47:01,838 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:47:01,866 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:47:04,228 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:47:06,229 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:47:06,839 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:47:11,932 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:47:12,140 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:47:12,656 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:47:17,141 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:47:17,664 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:47:22,000 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:47:22,715 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:47:22,940 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:47:27,717 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:47:27,942 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:47:32,064 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:47:33,472 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:47:33,641 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:47:34,229 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:47:39,230 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:47:39,651 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:47:42,130 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:47:44,652 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:47:44,833 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:47:49,835 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:47:50,140 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:47:52,200 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:47:55,140 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:47:55,448 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:48:00,804 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:48:01,123 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:48:02,274 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:48:04,230 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:48:05,805 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:48:06,231 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:48:11,520 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:48:11,593 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:48:12,333 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:48:16,596 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:48:17,981 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:48:19,313 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 15:48:19,315 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 15:48:19,315 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 15:48:19,317 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 15:48:19,382 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 15:48:21,391 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:48:22,319 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:48:22,396 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:48:22,983 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:48:27,320 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:48:28,417 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:48:28,417 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:48:32,463 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:48:32,755 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:48:33,418 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:48:34,230 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:48:38,232 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:48:39,341 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:48:41,532 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:48:43,759 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:48:44,400 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:48:48,761 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:48:49,439 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:48:51,654 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:48:54,440 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:48:54,721 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:48:59,708 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:48:59,839 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:49:00,041 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:49:04,231 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:49:04,840 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:49:05,232 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:49:09,774 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:49:10,344 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:49:10,529 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:49:15,345 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:49:15,530 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:49:19,839 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:49:20,536 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:49:20,751 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:49:25,537 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:49:25,752 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:49:29,906 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:49:31,199 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:49:31,353 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:49:34,231 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:49:36,772 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:49:37,233 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:49:39,981 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:49:41,773 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:49:42,974 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:49:47,273 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:49:48,556 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:49:50,040 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:49:52,274 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:49:53,572 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:49:57,991 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:49:57,991 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:49:58,090 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:49:59,234 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:50:02,992 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:50:04,234 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:50:04,236 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:50:08,160 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:50:08,883 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:50:10,089 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:50:15,091 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:50:15,206 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:50:18,219 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:50:20,207 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:50:20,540 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:50:25,541 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:50:25,926 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:50:28,281 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:50:30,927 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:50:31,195 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:50:34,235 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:50:36,590 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:50:36,936 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:50:38,368 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:50:41,591 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:50:41,937 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:50:46,446 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:50:47,163 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:50:47,408 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:50:52,164 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:50:52,410 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:50:56,509 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:50:57,774 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:50:58,050 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:51:03,051 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:51:03,947 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:51:04,236 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:51:06,573 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:51:08,332 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:51:08,949 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:51:13,333 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:51:14,576 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:51:14,577 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:51:16,632 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:51:18,772 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:51:19,577 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:51:23,774 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:51:25,011 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:51:26,693 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:51:29,256 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:51:30,012 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:51:33,949 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 15:51:33,950 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 15:51:33,951 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 15:51:33,953 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 15:51:34,236 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:51:34,747 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 15:51:35,237 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:51:35,578 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:51:35,752 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:51:40,238 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:51:40,580 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:51:43,800 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:51:45,411 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:51:46,208 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:51:50,412 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:51:51,822 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:51:53,867 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:51:55,965 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:51:56,823 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:52:00,966 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:52:01,904 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:52:03,941 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:52:04,237 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:52:06,238 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:52:06,905 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:52:11,239 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:52:12,208 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:52:12,209 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:52:14,011 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:52:16,581 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:52:17,434 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:52:21,583 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:52:22,908 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:52:24,079 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:52:27,231 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:52:27,909 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:52:32,776 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:52:33,637 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:52:34,143 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:52:34,237 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:52:38,238 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:52:38,639 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:52:42,199 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:52:43,452 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:52:44,215 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:52:48,454 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:52:50,095 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:52:52,274 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:52:54,322 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:52:55,096 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:52:59,336 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:53:00,840 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:53:00,840 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:53:02,347 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:53:04,237 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:53:05,238 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:53:05,841 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:53:10,240 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:53:11,295 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:53:12,420 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:53:15,446 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:53:16,296 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:53:20,474 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:53:20,684 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:53:21,490 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:53:25,685 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:53:27,802 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:53:27,802 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:53:27,803 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:53:30,538 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:53:31,049 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:53:32,803 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:53:34,238 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:53:36,239 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:53:37,804 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:53:41,241 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:53:42,620 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:53:43,039 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:53:46,422 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:53:48,041 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:53:52,137 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:53:52,677 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:53:53,957 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:53:57,138 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:53:59,570 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:53:59,570 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:54:02,756 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:54:02,774 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:54:04,239 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:54:04,571 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:54:08,240 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:54:09,884 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:54:12,836 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:54:14,063 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:54:14,884 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:54:19,064 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:54:20,273 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:54:20,897 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:54:24,715 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:54:25,274 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:54:29,717 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:54:30,478 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:54:30,479 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:54:32,003 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:54:34,239 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:54:35,240 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:54:35,526 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:54:40,895 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:54:41,520 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:54:42,086 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:54:45,895 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:54:46,522 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:54:49,498 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 15:54:49,499 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 15:54:49,500 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 15:54:49,502 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 15:54:50,150 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:54:50,150 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 15:54:51,504 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:54:52,195 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:54:56,505 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:54:57,681 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:55:00,218 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:55:01,943 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:55:02,683 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:55:04,240 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:55:07,241 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:55:08,195 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:55:10,298 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:55:12,450 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:55:13,244 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:55:17,451 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:55:18,322 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:55:18,322 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:55:20,359 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:55:22,542 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:55:23,323 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:55:27,543 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:55:28,478 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:55:30,419 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:55:33,293 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:55:33,981 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:55:34,240 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:55:38,475 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:55:38,749 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:55:39,489 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:55:43,750 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:55:44,492 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:55:48,538 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:55:49,302 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:55:50,177 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:55:54,304 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:55:55,631 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:55:58,607 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:55:59,958 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:56:00,633 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:56:04,242 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:56:05,245 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:56:06,275 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:56:08,675 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:56:10,551 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:56:11,276 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:56:15,552 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:56:16,396 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:56:16,396 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:56:18,748 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:56:21,000 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:56:21,410 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:56:26,001 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:56:27,338 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:56:28,808 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:56:31,596 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:56:32,339 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:56:34,244 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:56:37,288 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:56:37,909 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:56:38,870 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:56:42,289 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:56:42,911 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:56:46,932 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:56:47,960 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:56:48,845 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:56:52,961 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:56:54,988 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:56:58,296 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:56:59,010 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:56:59,989 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:57:03,298 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:57:04,245 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:57:05,618 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:57:07,063 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:57:08,905 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:57:10,619 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:57:13,907 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:57:15,845 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:57:16,122 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:57:19,286 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:57:21,150 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:57:24,287 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:57:26,185 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:57:26,617 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:57:29,868 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:57:31,619 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:57:34,245 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:57:35,411 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:57:36,243 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:57:37,265 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:57:40,412 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:57:43,395 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:57:45,467 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:57:46,308 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:57:48,396 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:57:50,468 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:57:53,845 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:57:56,203 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:57:56,370 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:57:58,846 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:58:01,204 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:58:03,822 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 15:58:03,824 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 15:58:03,824 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 15:58:03,827 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 15:58:04,245 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:58:04,423 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:58:04,423 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 15:58:04,524 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:58:04,525 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:58:06,246 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:58:06,432 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:58:09,525 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:58:11,248 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:58:14,483 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:58:14,845 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:58:16,598 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:58:20,146 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:58:21,600 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:58:24,549 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:58:25,926 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:58:27,198 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:58:31,388 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:58:31,388 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:58:31,389 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:58:32,199 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:58:34,246 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:58:34,617 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:58:36,432 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:58:37,247 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:58:42,248 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:58:42,381 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:58:44,690 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:58:47,382 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:58:47,677 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:58:52,942 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:58:53,211 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:58:54,757 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:58:57,953 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:58:58,212 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:59:02,817 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:59:03,389 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:59:03,867 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:59:04,246 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:59:09,248 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:59:10,024 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:59:13,144 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:59:15,025 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:59:15,229 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:59:20,230 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:59:20,827 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:59:20,827 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:59:23,216 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:59:25,837 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:59:26,005 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:59:31,388 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:59:31,389 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:59:31,736 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:59:32,277 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:59:34,247 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 15:59:36,462 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:59:37,248 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:59:41,690 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:59:42,338 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:59:42,878 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:59:46,691 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:59:47,879 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:59:52,015 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:59:52,406 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 15:59:53,866 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 15:59:57,016 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 15:59:58,867 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:00:02,199 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:00:02,466 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:00:04,247 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:00:04,248 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:00:08,711 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:00:09,249 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:00:10,525 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:00:13,712 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:00:15,040 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:00:19,116 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:00:19,116 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:00:20,237 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:00:20,595 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:00:24,117 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:00:25,237 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:00:29,544 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:00:30,665 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:00:30,825 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:00:34,248 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:00:34,600 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:00:36,249 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:00:39,657 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:00:40,736 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:00:42,096 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:00:44,659 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:00:47,098 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:00:50,472 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:00:50,472 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:00:50,804 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:00:52,707 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:00:55,473 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:00:57,708 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:01:01,532 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:01:02,710 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:01:04,248 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:01:04,905 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:01:06,534 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:01:08,249 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:01:12,138 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:01:13,393 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:01:14,970 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:01:17,139 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:01:18,394 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:01:21,625 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 16:01:21,626 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 16:01:21,627 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 16:01:21,629 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 16:01:22,020 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 16:01:22,441 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:01:22,441 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:01:23,025 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:01:23,631 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:01:24,029 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:01:27,442 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:01:28,632 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:01:32,472 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:01:33,095 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:01:33,862 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:01:34,249 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:01:37,696 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:01:39,250 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:01:43,157 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:01:43,624 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:01:44,898 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:01:49,346 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:01:49,347 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:01:49,900 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:01:53,223 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:01:54,348 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:01:55,475 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:01:59,919 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:02:01,144 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:02:03,296 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:02:04,249 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:02:04,920 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:02:06,250 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:02:10,715 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:02:11,350 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:02:11,947 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:02:15,716 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:02:16,948 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:02:21,340 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:02:21,415 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:02:22,547 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:02:26,342 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:02:27,548 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:02:30,483 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:02:32,137 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:02:33,362 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:02:34,250 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:02:37,592 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:02:37,592 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:02:38,847 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:02:40,551 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:02:42,593 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:02:43,848 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:02:47,837 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:02:48,605 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:02:48,917 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:02:52,838 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:02:53,918 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:02:58,133 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:02:58,655 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:02:59,331 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:03:03,134 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:03:04,252 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:03:05,254 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:03:08,578 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:03:08,761 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:03:10,855 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:03:13,580 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:03:15,856 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:03:18,822 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:03:19,340 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:03:21,537 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:03:24,885 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:03:27,208 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:03:28,885 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:03:29,887 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:03:32,209 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:03:34,253 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:03:35,427 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:03:35,427 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:03:36,938 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:03:37,667 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:03:40,428 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:03:42,668 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:03:45,867 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:03:47,017 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:03:48,124 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:03:50,868 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:03:53,125 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:03:56,372 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:03:57,082 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:03:58,618 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:04:02,850 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:04:03,619 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:04:04,253 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:04:05,134 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:04:07,878 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:04:09,254 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:04:13,768 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:04:15,056 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:04:15,215 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:04:18,769 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:04:20,057 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:04:24,085 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:04:24,085 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:04:25,266 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:04:25,287 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:04:29,086 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:04:30,267 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:04:32,809 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 16:04:32,811 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 16:04:32,811 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 16:04:32,813 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 16:04:33,338 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 16:04:34,253 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:04:34,344 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:04:34,427 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:04:34,427 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:04:35,346 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:04:36,254 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:04:40,909 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:04:41,263 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:04:45,409 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:04:45,911 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:04:46,379 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:04:51,380 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:04:51,561 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:04:51,562 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:04:53,462 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:04:56,562 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:04:56,727 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:05:01,892 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:05:02,141 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:05:03,526 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:05:04,254 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:05:06,893 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:05:07,255 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:05:12,824 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:05:13,102 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:05:13,588 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:05:18,103 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:05:19,188 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:05:22,652 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:05:23,288 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:05:24,189 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:05:28,290 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:05:29,661 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:05:30,705 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:05:33,807 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:05:34,254 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:05:34,665 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:05:39,256 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:05:40,296 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:05:40,297 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:05:40,766 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:05:44,488 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:05:45,298 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:05:49,671 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:05:50,561 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:05:50,561 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:05:50,832 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:05:54,672 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:05:56,890 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:05:58,879 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:06:00,173 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:06:01,899 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:06:04,255 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:06:05,256 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:06:07,763 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:06:07,764 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:06:08,943 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:06:10,974 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:06:12,764 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:06:15,976 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:06:18,196 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:06:19,012 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:06:21,550 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:06:23,197 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:06:26,552 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:06:28,769 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:06:29,086 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:06:32,376 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:06:33,822 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:06:34,256 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:06:37,924 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:06:39,153 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:06:39,630 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:06:42,925 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:06:45,164 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:06:48,499 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:06:49,233 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:06:50,165 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:06:53,501 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:06:56,100 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:06:56,100 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:06:57,296 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:06:59,281 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:07:01,101 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:07:04,259 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:07:05,261 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:07:06,392 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:07:07,370 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:07:10,568 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:07:11,393 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:07:15,707 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:07:16,565 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:07:17,436 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:07:20,708 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:07:23,055 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:07:23,055 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:07:23,056 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:07:25,494 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:07:26,312 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:07:28,057 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:07:31,314 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:07:33,361 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:07:34,258 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:07:35,566 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:07:37,259 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:07:38,362 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:07:42,261 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:07:43,280 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 16:07:43,281 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 16:07:43,282 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 16:07:43,284 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 16:07:43,620 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 16:07:43,924 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:07:44,626 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:07:45,630 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:07:47,286 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:07:48,925 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:07:52,288 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:07:54,056 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:07:55,695 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:07:57,920 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:07:59,615 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:08:02,921 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:08:04,259 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:08:04,809 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:08:05,758 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:08:08,260 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:08:09,811 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:08:13,741 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:08:15,458 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:08:15,815 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:08:18,743 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:08:20,949 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:08:23,870 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:08:24,158 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:08:25,951 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:08:29,160 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:08:31,480 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:08:32,931 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:08:34,259 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:08:34,260 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:08:36,481 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:08:39,261 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:08:42,046 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:08:42,046 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:08:43,178 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:08:44,316 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:08:47,047 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:08:49,318 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:08:52,725 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:08:53,239 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:08:54,809 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:08:57,726 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:08:59,811 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:09:03,300 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:09:03,430 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:09:04,260 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:09:05,260 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:09:08,808 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:09:08,809 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:09:08,809 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:09:11,011 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:09:11,350 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:09:13,810 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:09:16,012 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:09:19,256 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:09:21,415 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:09:21,452 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:09:24,257 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:09:26,453 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:09:29,742 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:09:31,477 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:09:32,154 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:09:34,260 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:09:34,743 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:09:37,262 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:09:39,583 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:09:39,808 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:09:42,397 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:09:46,625 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:09:47,398 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:09:49,644 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:09:51,626 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:09:52,846 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:09:57,264 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:09:58,401 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:09:59,708 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:10:02,265 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:10:03,401 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:10:04,264 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:10:07,549 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:10:08,766 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:10:09,777 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:10:12,550 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:10:13,767 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:10:17,835 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:10:17,897 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:10:19,101 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:10:24,102 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:10:24,214 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:10:24,215 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:10:24,215 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:10:27,890 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:10:29,216 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:10:29,371 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:10:34,262 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:10:34,732 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:10:35,027 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:10:37,957 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:10:39,733 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:10:40,028 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:10:45,690 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:10:45,906 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:10:48,028 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:10:50,691 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:10:50,907 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:10:54,486 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 16:10:54,488 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 16:10:54,489 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 16:10:54,491 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 16:10:55,077 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:10:55,077 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 16:10:56,215 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:10:56,493 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:10:57,087 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:11:01,494 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:11:02,788 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:11:04,262 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:11:05,145 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:11:07,263 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:11:07,790 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:11:12,279 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:11:13,409 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:11:13,410 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:11:15,209 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:11:17,920 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:11:18,410 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:11:22,921 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:11:24,215 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:11:25,272 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:11:27,933 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:11:29,270 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:11:32,934 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:11:34,278 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:11:35,025 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:11:35,336 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:11:38,279 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:11:40,026 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:11:43,280 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:11:45,274 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:11:45,406 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:11:48,664 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:11:50,275 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:11:53,666 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:11:55,467 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:11:55,593 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:11:59,279 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:12:00,594 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:12:04,263 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:12:04,308 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:12:05,533 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:12:05,926 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:12:09,309 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:12:10,926 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:12:14,311 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:12:15,587 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:12:15,941 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:12:19,337 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:12:20,942 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:12:25,304 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:12:25,653 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:12:26,216 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:12:30,305 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:12:31,577 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:12:34,264 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:12:35,697 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:12:35,718 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:12:37,332 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:12:40,698 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:12:43,177 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:12:43,177 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:12:45,784 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:12:46,436 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:12:48,178 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:12:51,437 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:12:53,627 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:12:55,891 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:12:56,829 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:12:58,628 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:13:01,831 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:13:03,950 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:13:04,265 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:13:05,954 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:13:07,265 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:13:08,951 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:13:12,267 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:13:14,483 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:13:16,017 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:13:17,619 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:13:19,483 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:13:22,620 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:13:24,484 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:13:27,621 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:13:29,485 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:13:32,622 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:13:34,115 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:13:34,265 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:13:34,614 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:13:38,266 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:13:41,208 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:13:41,208 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:13:43,542 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:13:44,184 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:13:46,209 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:13:48,543 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:13:51,986 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:13:54,254 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:13:54,344 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:13:56,987 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:13:59,345 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:14:02,542 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:14:04,265 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:14:04,316 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:14:05,266 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:14:07,543 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:14:10,267 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:14:12,861 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:14:13,382 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:14:16,060 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:14:17,863 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:14:21,101 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:14:21,187 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 16:14:21,188 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 16:14:21,189 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 16:14:21,191 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 16:14:21,443 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 16:14:23,457 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:14:23,774 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:14:26,193 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:14:29,510 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:14:31,508 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:14:31,825 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:14:34,266 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:14:34,511 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:14:37,268 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:14:39,862 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:14:39,862 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:14:41,578 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:14:42,460 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:14:45,246 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:14:47,461 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:14:50,789 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:14:51,663 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:14:53,111 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:14:55,790 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:14:58,113 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:15:01,477 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:15:01,736 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:15:03,970 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:15:04,266 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:15:06,478 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:15:09,297 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:15:12,304 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:15:13,805 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:15:14,728 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:15:17,305 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:15:19,730 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:15:22,910 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:15:23,867 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:15:25,204 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:15:27,910 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:15:30,204 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:15:33,838 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:15:34,269 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:15:36,181 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:15:37,959 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:15:38,839 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:15:41,181 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:15:44,537 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:15:44,538 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:15:46,020 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:15:46,838 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:15:49,538 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:15:51,840 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:15:55,175 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:15:56,127 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:15:57,556 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:16:00,176 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:16:02,557 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:16:04,270 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:16:05,288 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:16:06,187 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:16:07,573 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:16:10,289 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:16:12,575 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:16:15,753 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:16:16,256 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:16:18,002 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:16:22,392 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:16:23,003 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:16:26,328 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:16:27,393 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:16:28,514 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:16:32,788 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:16:34,046 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:16:34,270 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:16:34,375 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:16:37,791 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:16:39,272 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:16:43,721 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:16:43,722 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:16:44,437 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:16:44,995 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:16:48,722 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:16:49,997 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:16:54,374 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:16:54,504 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:16:55,552 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:16:59,376 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:17:00,553 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:17:03,571 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:17:04,271 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:17:05,115 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:17:06,271 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:17:10,517 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:17:11,273 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:17:13,649 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:17:15,518 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:17:16,754 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:17:21,198 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:17:22,364 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:17:23,715 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:17:26,199 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:17:27,365 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:17:31,490 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:17:31,490 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:17:31,772 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:17:32,666 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:17:34,271 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:17:36,491 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:17:38,273 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:17:40,077 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 16:17:40,078 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 16:17:40,079 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 16:17:40,081 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 16:17:40,834 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 16:17:41,721 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:17:41,840 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:17:44,083 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:17:48,311 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:17:49,084 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:17:51,911 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:17:53,312 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:17:54,548 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:17:59,061 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:17:59,062 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:18:00,195 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:18:01,979 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:18:04,062 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:18:04,272 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:18:05,273 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:18:09,493 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:18:10,031 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:18:10,743 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:18:14,494 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:18:15,745 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:18:19,943 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:18:20,103 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:18:21,223 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:18:24,944 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:18:26,224 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:18:30,061 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:18:30,163 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:18:31,878 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:18:34,273 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:18:36,533 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:18:37,274 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:18:40,231 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:18:41,534 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:18:42,852 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:18:47,018 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:18:47,018 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:18:48,299 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:18:50,302 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:18:52,019 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:18:53,300 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:18:57,533 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:18:58,356 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:18:58,675 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:19:02,534 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:19:03,677 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:19:04,275 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:19:07,949 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:19:08,424 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:19:09,236 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:19:12,958 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:19:14,237 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:19:18,494 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:19:18,543 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:19:18,543 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:19:19,694 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:19:23,916 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:19:24,695 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:19:28,562 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:19:28,918 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:19:30,452 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:19:34,114 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:19:34,114 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:19:34,277 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:19:36,371 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:19:38,622 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:19:39,112 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:19:41,372 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:19:44,543 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:19:46,677 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:19:47,002 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:19:49,643 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:19:52,004 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:19:55,344 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:19:55,736 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:19:57,605 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:20:00,345 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:20:02,606 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:20:04,276 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:20:05,798 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:20:05,882 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:20:08,277 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:20:10,883 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:20:13,279 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:20:15,879 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:20:16,543 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:20:19,007 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:20:22,198 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:20:24,564 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:20:25,946 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:20:27,199 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:20:29,565 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:20:33,067 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:20:33,067 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:20:33,997 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:20:34,277 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:20:35,277 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:20:38,068 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:20:40,280 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:20:43,597 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:20:44,083 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:20:45,929 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:20:48,598 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:20:50,930 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:20:53,599 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:20:55,931 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:20:58,600 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:21:00,932 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:21:04,256 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:21:04,257 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:21:04,277 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:21:06,278 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:21:09,257 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:21:11,279 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:21:14,579 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:21:14,579 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:21:16,280 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:21:16,910 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 16:21:16,911 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 16:21:16,911 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 16:21:16,915 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 16:21:17,268 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 16:21:18,275 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:21:19,580 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:21:21,923 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:21:25,271 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:21:27,900 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:21:28,351 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:21:30,272 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:21:32,901 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:21:34,278 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:21:35,496 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:21:38,023 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:21:38,411 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:21:40,520 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:21:43,024 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:21:46,395 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:21:46,396 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:21:48,487 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:21:48,776 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:21:51,396 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:21:53,778 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:21:56,772 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:21:58,548 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:21:59,324 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:22:01,775 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:22:04,279 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:22:05,281 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:22:06,846 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:22:08,612 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:22:10,401 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:22:11,847 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:22:15,401 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:22:17,045 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:22:18,673 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:22:20,456 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:22:22,046 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:22:25,458 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:22:27,920 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:22:28,741 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:22:31,284 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:22:32,921 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:22:34,284 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:22:36,285 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:22:38,626 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:22:38,807 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:22:41,960 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:22:43,627 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:22:46,962 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:22:48,865 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:22:49,230 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:22:49,230 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:22:52,543 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:22:54,231 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:22:58,318 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:22:58,933 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:23:00,105 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:23:03,319 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:23:04,281 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:23:05,493 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:23:05,493 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:23:07,989 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:23:08,726 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:23:10,494 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:23:13,727 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:23:16,230 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:23:16,231 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:23:18,051 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:23:19,550 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:23:21,315 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:23:24,551 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:23:26,479 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:23:28,120 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:23:29,639 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:23:31,480 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:23:34,285 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:23:35,288 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:23:37,000 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:23:38,194 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:23:41,200 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:23:42,001 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:23:46,699 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:23:47,230 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:23:48,255 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:23:51,700 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:23:54,027 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:23:56,344 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:23:57,360 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:23:59,028 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:24:02,361 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:24:04,284 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:24:04,861 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:24:04,861 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:24:06,409 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:24:08,045 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:24:09,862 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:24:13,047 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:24:15,455 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:24:16,479 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:24:18,610 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:24:20,456 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:24:23,611 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:24:26,012 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:24:26,541 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:24:29,163 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:24:32,456 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:24:32,456 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:24:32,457 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:24:33,722 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 16:24:33,723 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 16:24:33,724 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 16:24:33,727 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 16:24:34,285 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:24:34,285 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:24:34,602 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:24:34,603 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 16:24:35,609 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:24:37,457 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:24:39,286 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:24:42,758 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:24:44,670 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:24:44,914 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:24:47,759 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:24:49,915 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:24:53,546 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:24:54,748 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:24:55,776 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:24:58,547 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:25:00,778 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:25:03,956 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:25:04,285 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:25:04,823 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:25:06,286 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:25:08,958 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:25:11,287 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:25:12,879 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:25:14,487 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:25:16,744 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:25:20,032 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:25:20,032 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:25:22,274 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:25:22,946 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:25:25,034 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:25:27,274 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:25:30,555 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:25:32,817 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:25:33,010 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:25:34,286 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:25:35,556 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:25:38,288 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:25:40,996 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:25:43,074 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:25:43,595 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:25:45,997 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:25:48,596 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:25:50,998 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:25:53,597 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:25:55,999 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:25:58,598 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:26:00,176 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:26:01,775 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:26:04,136 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:26:04,286 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:26:08,032 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:26:09,288 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:26:10,243 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:26:13,033 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:26:14,541 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:26:18,832 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:26:18,833 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:26:20,170 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:26:20,303 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:26:23,834 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:26:25,171 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:26:29,396 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:26:30,368 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:26:30,779 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:26:34,287 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:26:34,397 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:26:36,291 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:26:40,017 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:26:40,429 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:26:41,322 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:26:46,324 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:26:46,506 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:26:48,472 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:26:51,507 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:26:51,783 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:26:56,785 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:26:56,939 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:27:00,542 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:27:01,940 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:27:02,333 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:27:04,288 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:27:07,747 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:27:07,748 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:27:07,930 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:27:08,594 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:27:12,748 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:27:12,931 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:27:17,833 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:27:18,443 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:27:18,654 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:27:23,197 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:27:23,444 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:27:28,814 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:27:28,864 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:27:29,025 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:27:34,026 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:27:34,290 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:27:35,416 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:27:38,876 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:27:39,760 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:27:40,417 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:27:44,762 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:27:45,592 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:27:48,942 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:27:49,899 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:27:50,593 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:27:54,901 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:27:56,317 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:27:56,501 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 16:27:56,502 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 16:27:56,503 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 16:27:56,505 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 16:27:57,001 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 16:27:58,009 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:27:59,012 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:28:00,507 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:28:01,318 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:28:04,291 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:28:06,045 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:28:06,780 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:28:09,084 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:28:11,046 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:28:11,781 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:28:16,341 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:28:17,133 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:28:17,145 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:28:21,342 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:28:22,147 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:28:27,146 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:28:27,200 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:28:27,940 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:28:32,148 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:28:33,415 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:28:33,416 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:28:33,416 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:28:34,291 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:28:37,260 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:28:37,693 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:28:38,418 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:28:42,695 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:28:44,133 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:28:47,327 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:28:48,271 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:28:49,134 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:28:53,272 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:28:54,375 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:28:54,397 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:28:58,639 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:28:59,398 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:29:04,294 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:29:04,294 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:29:04,417 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:29:04,441 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:29:09,295 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:29:10,167 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:29:14,481 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:29:14,514 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:29:15,254 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:29:19,482 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:29:21,847 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:29:21,847 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:29:24,574 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:29:25,088 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:29:26,848 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:29:30,089 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:29:32,144 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:29:34,292 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:29:34,640 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:29:35,293 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:29:37,145 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:29:40,294 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:29:42,555 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:29:42,730 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:29:45,727 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:29:47,556 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:29:50,729 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:29:52,788 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:29:52,873 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:29:52,873 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:29:56,068 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:29:57,874 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:30:01,070 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:30:02,839 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:30:03,059 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:30:04,295 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:30:06,295 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:30:08,060 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:30:12,077 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:30:12,905 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:30:13,661 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:30:17,078 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:30:19,983 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:30:19,983 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:30:19,983 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:30:22,299 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:30:22,963 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:30:24,984 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:30:27,300 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:30:29,985 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:30:32,302 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:30:34,294 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:30:35,220 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:30:37,040 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:30:37,708 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:30:40,226 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:30:42,749 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:30:45,270 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:30:47,108 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:30:48,356 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:30:50,271 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:30:53,358 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:30:55,663 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:30:57,171 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:30:58,980 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:31:00,702 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:31:03,981 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:31:04,305 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:31:05,983 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:31:07,242 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:31:09,780 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:31:11,596 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:31:14,355 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 16:31:14,356 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 16:31:14,357 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 16:31:14,359 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 16:31:15,305 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 16:31:15,361 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:31:16,311 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:31:17,225 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:31:17,317 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:31:20,362 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:31:22,627 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:31:22,627 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:31:25,373 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:31:25,879 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:31:27,629 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:31:31,184 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:31:31,414 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:31:32,421 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:31:32,630 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:31:34,295 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:31:36,253 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:31:37,448 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:31:37,631 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:31:38,454 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:31:41,260 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:31:42,632 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:31:46,933 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:31:47,506 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:31:47,797 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:31:51,934 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:31:54,271 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:31:54,271 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:31:56,572 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:31:57,553 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:31:59,272 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:32:02,554 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:32:04,296 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:32:04,738 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:32:06,634 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:32:08,023 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:32:09,739 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:32:13,024 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:32:15,371 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:32:16,703 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:32:18,749 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:32:20,372 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:32:23,750 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:32:24,757 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:32:26,000 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:32:29,318 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:32:32,424 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:32:34,296 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:32:34,821 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:32:35,297 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:32:37,425 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:32:40,298 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:32:42,535 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:32:44,885 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:32:45,825 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:32:47,554 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:32:50,826 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:32:53,038 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:32:53,038 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:32:54,941 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:32:56,076 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:32:58,039 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:33:01,077 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:33:03,695 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:33:04,296 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:33:05,005 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:33:06,297 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:33:08,697 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:33:11,299 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:33:14,100 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:33:15,060 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:33:16,346 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:33:19,101 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:33:21,348 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:33:24,620 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:33:24,620 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:33:25,123 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:33:26,922 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:33:29,622 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:33:31,923 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:33:34,297 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:33:34,985 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:33:35,188 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:33:37,299 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:33:41,407 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:33:42,300 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:33:45,252 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:33:46,408 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:33:47,655 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:33:51,453 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:33:51,453 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:33:52,657 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:33:55,314 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:33:56,454 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:33:57,755 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:34:01,996 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:34:03,245 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:34:03,375 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:34:04,300 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:34:06,997 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:34:08,301 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:34:12,629 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:34:13,443 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:34:13,907 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:34:17,630 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:34:18,908 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:34:21,359 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 16:34:21,360 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 16:34:21,360 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 16:34:21,363 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 16:34:21,503 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 16:34:22,507 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:34:23,081 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:34:23,513 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:34:24,365 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:34:28,082 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:34:29,367 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:34:33,245 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:34:33,572 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:34:34,298 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:34:35,299 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:34:39,618 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:34:40,300 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:34:43,638 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:34:44,619 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:34:45,797 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:34:49,903 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:34:51,060 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:34:51,696 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:34:54,904 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:34:56,061 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:35:00,511 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:35:01,681 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:35:01,766 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:35:04,299 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:35:05,512 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:35:07,301 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:35:10,822 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:35:11,028 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:35:12,394 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:35:17,395 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:35:17,507 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:35:18,872 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:35:22,508 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:35:22,593 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:35:27,594 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:35:28,366 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:35:28,934 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:35:32,660 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:35:33,367 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:35:34,300 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:35:38,301 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:35:38,998 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:35:39,028 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:35:43,357 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:35:44,164 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:35:48,847 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:35:49,057 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:35:49,630 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:35:53,848 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:35:54,632 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:35:59,114 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:35:59,428 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:36:00,265 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:36:04,302 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:36:05,305 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:36:05,991 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:36:09,181 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:36:10,992 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:36:11,174 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:36:16,175 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:36:16,389 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:36:19,247 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:36:21,390 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:36:21,830 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:36:27,276 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:36:27,276 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:36:27,309 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:36:27,442 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:36:32,277 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:36:32,443 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:36:34,303 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:36:37,366 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:36:37,623 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:36:37,865 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:36:42,866 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:36:44,134 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:36:47,441 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:36:48,419 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:36:49,135 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:36:53,421 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:36:54,629 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:36:54,629 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:36:57,504 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:36:58,870 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:36:59,642 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:37:03,872 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:37:04,304 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:37:05,590 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:37:07,569 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:37:09,856 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:37:10,591 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:37:15,447 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:37:15,619 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:37:16,134 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:37:20,458 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:37:22,553 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:37:25,687 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:37:25,777 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:37:27,554 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:37:30,778 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:37:32,797 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:37:32,988 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 16:37:32,990 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 16:37:32,990 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 16:37:32,993 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 16:37:33,736 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 16:37:34,304 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:37:34,743 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:37:35,747 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:37:36,305 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:37:37,798 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:37:41,307 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:37:43,210 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:37:43,210 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:37:43,801 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:37:46,471 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:37:48,226 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:37:51,472 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:37:53,629 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:37:53,629 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:37:53,865 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:37:57,222 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:38:00,470 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:38:02,758 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:38:03,974 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:38:04,310 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:38:05,472 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:38:08,312 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:38:11,044 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:38:11,045 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:38:13,032 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:38:14,264 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:38:16,045 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:38:19,266 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:38:21,090 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:38:21,294 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:38:24,431 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:38:26,294 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:38:29,432 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:38:31,160 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:38:31,984 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:38:34,311 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:38:35,311 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:38:36,986 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:38:41,018 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:38:41,239 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:38:42,044 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:38:46,019 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:38:48,845 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:38:51,304 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:38:51,993 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:38:53,846 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:38:56,994 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:38:59,329 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:38:59,329 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:39:01,374 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:39:02,614 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:39:04,311 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:39:04,330 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:39:08,315 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:39:09,982 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:39:11,440 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:39:14,281 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:39:14,983 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:39:19,498 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:39:19,937 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:39:20,653 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:39:24,938 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:39:25,655 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:39:29,573 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:39:30,781 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:39:31,548 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:39:34,312 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:39:36,313 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:39:37,607 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:39:39,638 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:39:41,899 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:39:42,609 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:39:46,901 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:39:48,084 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:39:49,698 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:39:52,835 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:39:53,085 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:39:57,837 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:39:58,750 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:39:58,751 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:39:59,763 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:40:02,962 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:40:03,764 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:40:04,312 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:40:08,313 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:40:09,475 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:40:09,833 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:40:13,681 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:40:14,476 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:40:19,295 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:40:19,899 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:40:19,995 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:40:24,296 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:40:24,996 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:40:29,926 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:40:29,965 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:40:30,744 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:40:34,315 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:40:35,316 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:40:36,915 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:40:40,048 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:40:41,153 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:40:41,916 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:40:46,155 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:40:47,738 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:40:47,981 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 16:40:47,982 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 16:40:47,983 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 16:40:47,985 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 16:40:48,106 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 16:40:49,110 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:40:50,115 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:40:51,987 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:40:52,739 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:40:56,988 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:40:58,265 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:40:59,178 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:41:02,556 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:41:03,267 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:41:04,317 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:41:08,357 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:41:09,153 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:41:09,271 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:41:13,361 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:41:15,715 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:41:15,715 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:41:17,331 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:41:19,005 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:41:20,745 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:41:24,006 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:41:27,009 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:41:27,009 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:41:27,009 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:41:29,264 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:41:29,409 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:41:32,010 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:41:34,265 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:41:34,317 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:41:37,836 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:41:39,477 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:41:40,245 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:41:42,837 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:41:45,247 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:41:47,874 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:41:49,547 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:41:50,384 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:41:52,874 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:41:55,385 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:41:57,932 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:41:59,604 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:42:01,155 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:42:02,933 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:42:04,318 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:42:06,319 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:42:08,158 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:42:09,675 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:42:11,493 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:42:13,159 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:42:16,495 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:42:18,759 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:42:19,809 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:42:22,130 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:42:23,760 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:42:27,132 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:42:29,020 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:42:29,878 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:42:32,749 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:42:34,321 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:42:35,965 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:42:38,251 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:42:39,945 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:42:40,966 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:42:43,252 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:42:46,515 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:42:46,516 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:42:47,995 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:42:48,795 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:42:51,517 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:42:53,796 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:42:57,010 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:42:58,053 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:42:59,418 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:43:02,236 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:43:04,320 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:43:05,322 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:43:07,238 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:43:08,123 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:43:10,486 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:43:12,239 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:43:15,487 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:43:17,709 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:43:18,183 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:43:21,089 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:43:24,228 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:43:26,406 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:43:28,240 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:43:29,229 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:43:31,407 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:43:34,321 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:43:34,618 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:43:36,292 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:43:36,912 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:43:39,619 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:43:41,914 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:43:44,657 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:43:44,658 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:43:46,352 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:43:47,913 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:43:49,658 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:43:52,915 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:43:55,101 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:43:56,425 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:43:58,287 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:44:00,102 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:44:03,289 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:44:03,881 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 16:44:03,883 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 16:44:03,883 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 16:44:03,885 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 16:44:04,321 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:44:04,481 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 16:44:05,486 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:44:05,612 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:44:08,322 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:44:10,614 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:44:13,324 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:44:15,558 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:44:16,321 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:44:18,607 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:44:21,861 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:44:23,612 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:44:24,021 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:44:26,862 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:44:29,022 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:44:32,769 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:44:33,675 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:44:34,321 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:44:34,322 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:44:37,770 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:44:39,323 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:44:42,824 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:44:44,324 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:44:47,753 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:44:48,607 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:44:50,063 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:44:53,608 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:44:55,064 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:44:57,815 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:44:59,263 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:45:00,562 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:45:04,323 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:45:04,988 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:45:06,223 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:45:07,892 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:45:09,989 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:45:11,223 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:45:15,388 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:45:15,388 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:45:16,625 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:45:17,956 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:45:20,452 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:45:21,626 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:45:26,310 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:45:27,526 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:45:28,025 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:45:31,311 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:45:32,527 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:45:34,324 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:45:36,920 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:45:38,090 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:45:38,092 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:45:41,920 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:45:43,091 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:45:46,922 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:45:48,092 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:45:52,883 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:45:53,093 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:45:57,884 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:45:58,094 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:46:02,885 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:46:03,094 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:46:04,324 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:46:08,046 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:46:08,326 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:46:09,948 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:46:13,046 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:46:13,849 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:46:18,274 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:46:19,694 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:46:20,011 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:46:23,275 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:46:24,695 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:46:28,334 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:46:29,783 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:46:30,073 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:46:34,080 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:46:34,325 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:46:35,326 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:46:39,296 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:46:40,143 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:46:40,547 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:46:44,297 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:46:45,547 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:46:49,603 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:46:50,204 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:46:51,105 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:46:54,604 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:46:56,107 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:47:00,268 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:47:00,406 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:47:02,060 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:47:04,325 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:47:05,426 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:47:07,327 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:47:10,323 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:47:11,271 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:47:12,573 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:47:16,759 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:47:16,759 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:47:17,575 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:47:20,390 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:47:21,762 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:47:23,296 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:47:26,833 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:47:29,181 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:47:30,458 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:47:31,834 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:47:34,182 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:47:34,325 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:47:37,399 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:47:39,818 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:47:40,529 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:47:42,401 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:47:44,819 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:47:48,279 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:47:48,489 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 16:47:48,490 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 16:47:48,491 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 16:47:48,494 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 16:47:48,581 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 16:47:49,586 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:47:50,495 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:47:50,591 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:47:53,280 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:47:55,496 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:47:58,407 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:48:00,656 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:48:01,318 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:48:03,772 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:48:04,326 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:48:06,327 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:48:09,760 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:48:10,714 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:48:11,985 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:48:16,214 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:48:16,986 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:48:19,772 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:48:21,215 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:48:22,455 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:48:26,511 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:48:27,456 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:48:29,838 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:48:31,512 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:48:33,115 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:48:34,326 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:48:36,682 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:48:38,887 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:48:39,903 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:48:41,683 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:48:43,888 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:48:47,271 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:48:47,272 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:48:49,539 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:48:49,970 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:48:52,323 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:48:54,540 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:48:57,408 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:48:57,408 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:48:59,719 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:49:00,029 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:49:02,443 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:49:04,329 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:49:05,332 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:49:08,242 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:49:10,091 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:49:10,407 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:49:13,243 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:49:15,408 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:49:19,053 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:49:20,165 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:49:21,355 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:49:25,624 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:49:26,360 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:49:28,219 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:49:30,625 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:49:31,886 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:49:34,330 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:49:36,020 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:49:37,170 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:49:38,435 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:49:41,021 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:49:42,171 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:49:46,621 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:49:46,622 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:49:47,830 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:49:48,502 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:49:51,623 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:49:52,831 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:49:57,170 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:49:58,466 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:49:58,570 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:50:02,171 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:50:03,468 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:50:04,345 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:50:06,621 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:50:08,028 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:50:09,232 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:50:13,477 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:50:13,478 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:50:13,478 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:50:14,233 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:50:16,689 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:50:18,479 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:50:19,711 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:50:23,826 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:50:25,028 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:50:26,753 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:50:28,827 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:50:30,029 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:50:34,115 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:50:34,332 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:50:34,811 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:50:35,333 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:50:39,116 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:50:40,334 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:50:44,478 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:50:44,880 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:50:46,057 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:50:49,685 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:50:51,059 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:50:53,943 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:50:55,444 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:50:56,696 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:51:00,902 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:51:00,902 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:51:01,697 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:51:02,060 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 16:51:02,061 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 16:51:02,062 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 16:51:02,066 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 16:51:03,011 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 16:51:04,018 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:51:04,333 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:51:05,906 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:51:07,334 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:51:10,961 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:51:13,259 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:51:14,087 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:51:15,977 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:51:18,260 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:51:21,441 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:51:23,672 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:51:24,155 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:51:26,442 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:51:28,674 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:51:31,892 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:51:32,221 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:51:34,091 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:51:34,333 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:51:36,893 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:51:39,335 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:51:42,279 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:51:42,437 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:51:45,099 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:51:47,438 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:51:50,101 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:51:52,340 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:51:52,695 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:51:55,843 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:51:59,039 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:52:01,476 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:52:02,399 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:52:04,040 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:52:04,333 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:52:07,335 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:52:09,187 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:52:12,463 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:52:12,510 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:52:14,188 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:52:17,512 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:52:19,872 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:52:19,872 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:52:22,539 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:52:23,108 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:52:24,873 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:52:28,109 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:52:29,892 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:52:29,892 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:52:32,597 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:52:33,805 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:52:34,334 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:52:35,452 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:52:39,336 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:52:40,664 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:52:41,324 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:52:44,533 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:52:46,325 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:52:49,534 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:52:51,518 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:52:52,737 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:52:54,694 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:52:57,900 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:53:00,118 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:53:00,790 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:53:02,901 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:53:04,335 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:53:05,336 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:53:08,810 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:53:10,863 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:53:11,059 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:53:13,811 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:53:16,061 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:53:19,484 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:53:19,485 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:53:20,934 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:53:21,696 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:53:24,485 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:53:26,698 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:53:29,892 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:53:31,000 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:53:32,278 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:53:34,335 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:53:36,546 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:53:37,336 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:53:41,080 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:53:41,547 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:53:42,765 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:53:47,209 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:53:47,210 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:53:48,126 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:53:48,407 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:53:52,211 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:53:53,408 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:53:57,246 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:53:58,190 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:53:58,563 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:54:02,280 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:54:03,563 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:54:04,337 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:54:07,927 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:54:08,300 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:54:09,144 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:54:12,928 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:54:14,145 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:54:16,599 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 16:54:16,601 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 16:54:16,601 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 16:54:16,604 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 16:54:17,366 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 16:54:18,210 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:54:18,373 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:54:19,605 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:54:24,607 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:54:24,756 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:54:28,440 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:54:29,757 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:54:29,992 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:54:34,338 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:54:35,250 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:54:35,250 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:54:35,340 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:54:36,506 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:54:40,251 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:54:40,387 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:54:45,719 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:54:45,970 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:54:46,571 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:54:50,721 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:54:50,971 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:54:56,136 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:54:56,638 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:54:56,716 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:55:01,137 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:55:01,718 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:55:04,339 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:55:06,335 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:55:06,703 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:55:06,791 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:55:11,336 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:55:11,792 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:55:16,721 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:55:16,762 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:55:17,211 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:55:21,722 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:55:22,239 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:55:26,834 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:55:27,369 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:55:27,682 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:55:32,370 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:55:32,683 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:55:34,340 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:55:36,901 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:55:38,068 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:55:38,311 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:55:43,312 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:55:43,627 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:55:46,973 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:55:48,628 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:55:48,941 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:55:53,778 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:55:53,942 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:55:57,044 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:55:58,780 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:55:59,257 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:56:03,968 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:56:03,969 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:56:04,258 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:56:04,342 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:56:07,120 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:56:08,970 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:56:09,559 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:56:14,070 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:56:14,561 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:56:17,193 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:56:19,072 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:56:20,322 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:56:24,517 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:56:25,741 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:56:27,254 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:56:29,518 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:56:30,742 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:56:34,342 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:56:35,044 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:56:35,045 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:56:36,322 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:56:36,344 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:56:40,045 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:56:41,344 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:56:45,470 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:56:46,385 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:56:46,700 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:56:50,471 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:56:51,702 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:56:55,957 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:56:56,447 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:56:57,222 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:57:00,959 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:57:02,224 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:57:04,343 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:57:04,526 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:57:06,679 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:57:06,679 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:57:07,344 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:57:11,911 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:57:12,345 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:57:14,598 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:57:16,912 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:57:18,077 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:57:22,495 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:57:22,496 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:57:23,714 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:57:24,663 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:57:27,496 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:57:28,715 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:57:32,324 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 16:57:32,325 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 16:57:32,326 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 16:57:32,329 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 16:57:32,678 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:57:32,714 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 16:57:34,331 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:57:34,343 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:57:34,724 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:57:38,040 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:57:39,344 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:57:43,240 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:57:44,414 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:57:44,788 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:57:48,241 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:57:49,416 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:57:52,843 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:57:53,625 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:57:54,848 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:57:59,849 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:57:59,971 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:58:02,908 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:58:04,343 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:58:04,972 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:58:05,344 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:58:10,345 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:58:10,630 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:58:12,974 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:58:15,632 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:58:15,894 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:58:21,242 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:58:21,243 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:58:21,515 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:58:23,043 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:58:26,243 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:58:26,516 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:58:31,888 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:58:32,144 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:58:33,107 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:58:34,344 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:58:36,912 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:58:37,345 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:58:41,162 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:58:42,805 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:58:42,958 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:58:47,959 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:58:48,306 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:58:48,306 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:58:48,307 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:58:51,231 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:58:53,308 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:58:53,627 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:58:58,628 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:58:59,073 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:59:01,298 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:59:04,074 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:59:04,228 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:59:04,344 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:59:09,362 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:59:09,469 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:59:11,363 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:59:14,363 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:59:14,470 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:59:19,365 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:59:19,471 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:59:24,714 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:59:25,209 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:59:25,449 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:59:29,715 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:59:31,498 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:59:34,356 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 16:59:34,521 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:59:35,357 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:59:36,499 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:59:40,358 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:59:41,943 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:59:44,590 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:59:46,139 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:59:46,944 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:59:51,140 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:59:52,421 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:59:52,421 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 16:59:52,657 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 16:59:56,747 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 16:59:57,422 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:00:02,239 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:00:02,713 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:00:02,953 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:00:04,345 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:00:07,347 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:00:07,955 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:00:12,746 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:00:12,791 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:00:13,550 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:00:17,747 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:00:18,871 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:00:18,871 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:00:18,872 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:00:20,850 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:00:23,090 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:00:23,872 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:00:28,092 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:00:29,126 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:00:32,922 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:00:33,337 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:00:34,128 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:00:34,346 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:00:38,347 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:00:39,678 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:00:40,983 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:00:43,869 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:00:44,679 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:00:48,871 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:00:49,522 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 17:00:49,523 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 17:00:49,524 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 17:00:49,527 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 17:00:49,871 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:00:50,044 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:00:50,045 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 17:00:51,048 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:00:54,529 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:00:55,208 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:00:59,811 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:01:00,587 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:01:01,105 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:01:04,346 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:01:05,347 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:01:07,153 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:01:07,154 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:01:10,473 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:01:11,172 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:01:12,155 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:01:15,475 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:01:17,200 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:01:21,235 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:01:21,451 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:01:22,201 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:01:26,453 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:01:27,512 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:01:29,287 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:01:31,747 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:01:32,513 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:01:34,347 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:01:37,068 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:01:37,849 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:01:39,349 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:01:42,069 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:01:42,850 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:01:47,423 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:01:48,207 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:01:49,424 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:01:52,424 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:01:54,544 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:01:57,476 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:01:57,781 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:01:59,545 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:02:02,783 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:02:04,351 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:02:04,556 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:02:04,556 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:02:04,557 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:02:07,552 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:02:07,966 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:02:09,557 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:02:12,968 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:02:15,069 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:02:17,615 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:02:18,272 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:02:20,070 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:02:23,274 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:02:25,634 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:02:27,685 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:02:28,936 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:02:30,636 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:02:33,937 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:02:34,350 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:02:36,049 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:02:36,745 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:02:39,351 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:02:41,051 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:02:44,799 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:02:44,843 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:02:46,652 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:02:49,843 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:02:51,980 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:02:51,980 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:02:54,868 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:02:55,209 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:02:56,981 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:03:00,211 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:03:02,168 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:03:04,351 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:03:04,935 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:03:05,352 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:03:07,169 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:03:10,353 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:03:12,565 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:03:15,005 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:03:15,737 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:03:17,566 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:03:20,738 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:03:22,939 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:03:22,939 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:03:23,069 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:03:26,195 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:03:27,940 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:03:31,196 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:03:33,124 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:03:33,490 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:03:34,351 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:03:36,352 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:03:38,492 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:03:42,270 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:03:43,202 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:03:44,122 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:03:47,271 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:03:49,710 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:03:52,893 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:03:53,261 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:03:54,711 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:03:57,895 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:04:00,491 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:04:00,682 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 17:04:00,683 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 17:04:00,684 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 17:04:00,686 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 17:04:01,316 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 17:04:02,321 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:04:03,326 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:04:03,688 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:04:04,351 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:04:05,492 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:04:09,353 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:04:10,791 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:04:11,379 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:04:14,941 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:04:15,792 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:04:20,092 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:04:20,939 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:04:20,939 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:04:21,442 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:04:25,093 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:04:27,278 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:04:30,602 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:04:31,517 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:04:32,279 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:04:34,352 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:04:36,353 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:04:37,783 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:04:37,783 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:04:39,569 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:04:42,230 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:04:42,784 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:04:47,232 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:04:48,476 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:04:49,646 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:04:52,810 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:04:53,477 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:04:57,812 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:04:58,766 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:04:59,704 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:05:02,982 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:05:03,776 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:05:04,355 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:05:08,496 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:05:08,783 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:05:09,759 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:05:13,497 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:05:15,685 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:05:18,887 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:05:19,830 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:05:20,686 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:05:23,888 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:05:25,969 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:05:25,969 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:05:27,884 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:05:29,211 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:05:30,970 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:05:34,213 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:05:34,355 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:05:36,374 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:05:36,945 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:05:39,607 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:05:41,375 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:05:44,608 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:05:46,670 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:05:47,017 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:05:49,811 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:05:51,672 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:05:55,690 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:05:57,076 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:05:57,451 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:05:57,451 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:06:00,691 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:06:02,960 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:06:04,355 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:06:05,137 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:06:06,356 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:06:07,960 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:06:11,357 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:06:13,733 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:06:15,197 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:06:16,898 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:06:18,734 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:06:21,900 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:06:24,211 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:06:25,267 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:06:27,421 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:06:29,212 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:06:32,422 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:06:34,356 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:06:34,499 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:06:35,327 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:06:38,357 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:06:40,925 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:06:40,925 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:06:43,378 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:06:44,109 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:06:45,926 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:06:49,110 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:06:51,608 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:06:53,440 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:06:54,793 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:06:56,609 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:06:59,794 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:07:01,753 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:07:03,509 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:07:04,356 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:07:05,357 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:07:06,754 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:07:10,359 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:07:11,650 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 17:07:11,651 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 17:07:11,651 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 17:07:11,654 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 17:07:12,220 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:07:12,220 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:07:12,579 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:07:12,580 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 17:07:13,583 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:07:15,659 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:07:17,221 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:07:20,660 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:07:22,297 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:07:23,667 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:07:26,486 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:07:27,298 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:07:31,798 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:07:32,610 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:07:33,743 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:07:34,357 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:07:37,358 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:07:38,907 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:07:38,907 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:07:38,909 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:07:41,802 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:07:43,119 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:07:43,908 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:07:48,120 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:07:49,267 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:07:51,869 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:07:53,430 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:07:54,268 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:07:58,431 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:07:59,303 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:08:01,926 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:08:03,550 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:08:04,304 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:08:04,357 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:08:09,161 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:08:09,818 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:08:11,998 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:08:14,162 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:08:14,819 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:08:19,483 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:08:20,056 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:08:20,261 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:08:24,484 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:08:25,264 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:08:29,557 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:08:30,122 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:08:30,300 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:08:34,358 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:08:35,359 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:08:36,671 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:08:39,188 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:08:41,069 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:08:41,675 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:08:46,071 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:08:47,458 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:08:49,257 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:08:51,674 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:08:52,462 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:08:56,675 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:08:58,358 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:08:58,359 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:08:59,322 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:09:01,703 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:09:03,359 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:09:04,359 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:09:06,894 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:09:08,733 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:09:09,381 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:09:11,895 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:09:15,056 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:09:17,180 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:09:17,443 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:09:20,057 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:09:22,180 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:09:25,413 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:09:25,414 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:09:27,513 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:09:27,658 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:09:30,414 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:09:32,659 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:09:34,360 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:09:35,936 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:09:37,580 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:09:38,206 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:09:40,937 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:09:43,208 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:09:46,357 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:09:47,647 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:09:48,464 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:09:51,358 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:09:53,466 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:09:56,414 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:09:57,703 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:09:59,118 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:10:01,827 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:10:04,119 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:10:04,360 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:10:05,763 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:10:07,437 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:10:09,699 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:10:12,878 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:10:12,879 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:10:15,104 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:10:15,824 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:10:17,880 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:10:20,105 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:10:23,244 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:10:23,464 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 17:10:23,465 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 17:10:23,466 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 17:10:23,468 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 17:10:23,876 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 17:10:24,881 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:10:25,470 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:10:25,893 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:10:28,244 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:10:30,472 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:10:33,597 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:10:33,955 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:10:34,361 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:10:36,362 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:10:38,598 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:10:41,363 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:10:43,642 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:10:43,643 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:10:44,024 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:10:46,967 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:10:48,655 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:10:51,968 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:10:54,080 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:10:54,083 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:10:57,275 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:11:00,471 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:11:02,776 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:11:04,141 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:11:04,361 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:11:05,472 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:11:08,363 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:11:11,070 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:11:14,218 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:11:14,273 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:11:16,072 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:11:19,275 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:11:21,657 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:11:22,276 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:11:24,917 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:11:26,658 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:11:29,918 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:11:31,350 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:11:32,264 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:11:34,362 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:11:35,363 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:11:37,266 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:11:40,364 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:11:41,420 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:11:43,071 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:11:45,464 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:11:48,592 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:11:50,794 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:11:51,491 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:11:53,593 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:11:55,795 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:11:58,975 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:11:58,976 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:12:01,217 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:12:01,555 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:12:03,976 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:12:04,363 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:12:06,382 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:12:09,614 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:12:09,621 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:12:11,873 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:12:14,615 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:12:16,875 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:12:19,694 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:12:19,919 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:12:22,030 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:12:24,921 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:12:27,031 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:12:29,763 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:12:30,633 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:12:32,818 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:12:34,366 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:12:35,999 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:12:38,233 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:12:39,827 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:12:41,000 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:12:43,234 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:12:46,549 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:12:47,885 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:12:48,730 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:12:51,550 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:12:53,732 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:12:57,200 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:12:57,200 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:12:57,950 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:12:59,375 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:13:02,201 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:13:04,367 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:13:05,369 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:13:07,562 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:13:08,018 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:13:10,788 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:13:12,564 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:13:15,789 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:13:17,862 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:13:18,093 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:13:21,195 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:13:22,864 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:13:26,953 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:13:28,155 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:13:28,201 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:13:31,954 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:13:33,711 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:13:34,367 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:13:37,368 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:13:38,952 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:13:40,422 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 17:13:40,424 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 17:13:40,424 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 17:13:40,427 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 17:13:41,260 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:13:41,261 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 17:13:42,264 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:13:42,429 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:13:43,953 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:13:47,431 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:13:49,620 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:13:52,331 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:13:52,870 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:13:54,621 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:13:57,871 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:14:00,144 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:14:00,145 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:14:02,402 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:14:03,371 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:14:04,367 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:14:05,146 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:14:09,369 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:14:10,200 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:14:12,472 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:14:14,492 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:14:15,322 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:14:20,222 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:14:20,522 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:14:21,108 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:14:25,223 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:14:27,432 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:14:27,433 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:14:29,601 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:14:30,706 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:14:32,434 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:14:34,367 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:14:36,368 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:14:37,828 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:14:39,666 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:14:41,982 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:14:42,829 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:14:46,983 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:14:48,310 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:14:49,720 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:14:52,562 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:14:53,310 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:14:57,776 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:14:58,145 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:14:58,874 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:15:03,146 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:15:03,875 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:15:04,370 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:15:07,841 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:15:08,221 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:15:08,953 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:15:13,222 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:15:15,455 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:15:15,456 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:15:17,910 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:15:18,952 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:15:20,457 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:15:23,953 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:15:25,875 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:15:25,875 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:15:27,987 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:15:29,588 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:15:31,279 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:15:34,371 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:15:35,372 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:15:36,757 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:15:38,058 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:15:40,959 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:15:41,757 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:15:46,582 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:15:47,409 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:15:48,120 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:15:51,583 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:15:53,773 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:15:56,179 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:15:57,002 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:15:58,775 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:16:02,003 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:16:04,371 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:16:04,733 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:16:06,246 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:16:07,026 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:16:09,734 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:16:12,028 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:16:15,504 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:16:15,505 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:16:16,311 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:16:17,686 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:16:20,505 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:16:22,687 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:16:25,680 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:16:26,377 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:16:27,835 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:16:32,268 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:16:32,836 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:16:34,371 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:16:36,451 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:16:37,269 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:16:38,372 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:16:42,939 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:16:42,939 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:16:44,153 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:16:44,499 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:16:47,940 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:16:49,154 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:16:52,510 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 17:16:52,512 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 17:16:52,512 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 17:16:52,515 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 17:16:52,548 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 17:16:53,307 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:16:53,552 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:16:54,516 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:16:54,557 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:16:58,308 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:16:59,518 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:17:04,209 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:17:04,371 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:17:04,624 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:17:05,372 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:17:10,374 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:17:10,769 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:17:14,692 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:17:15,771 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:17:16,027 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:17:21,029 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:17:21,309 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:17:23,752 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:17:26,310 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:17:26,537 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:17:31,770 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:17:31,770 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:17:31,840 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:17:31,998 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:17:34,372 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:17:36,771 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:17:37,381 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:17:41,902 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:17:41,939 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:17:41,939 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:17:42,458 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:17:47,459 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:17:48,646 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:17:52,046 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:17:52,757 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:17:53,647 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:17:57,759 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:17:59,297 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:17:59,298 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:18:02,105 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:18:03,529 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:18:04,298 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:18:04,373 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:18:08,934 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:18:09,698 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:18:10,164 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:18:13,935 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:18:14,699 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:18:19,710 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:18:20,229 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:18:20,265 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:18:24,711 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:18:25,267 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:18:30,126 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:18:30,294 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:18:30,299 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:18:34,394 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:18:35,396 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:18:36,388 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:18:40,360 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:18:40,649 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:18:41,392 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:18:45,650 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:18:47,251 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:18:47,251 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:18:48,427 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:18:51,438 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:18:52,252 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:18:56,440 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:18:57,558 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:19:00,499 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:19:01,788 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:19:02,559 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:19:04,376 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:19:07,255 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:19:07,974 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:19:08,554 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:19:12,256 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:19:12,975 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:19:17,553 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:19:18,373 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:19:18,623 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:19:22,554 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:19:24,620 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:19:27,802 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:19:28,680 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:19:29,621 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:19:32,803 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:19:34,378 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:19:35,323 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:19:36,733 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:19:38,553 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:19:40,324 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:19:43,555 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:19:45,780 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:19:46,803 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:19:49,055 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:19:50,791 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:19:54,057 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:19:56,633 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:19:56,862 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:19:59,797 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:20:01,635 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:20:04,379 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:20:04,715 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 17:20:04,716 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 17:20:04,717 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 17:20:04,720 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 17:20:04,932 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 17:20:05,721 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:20:05,940 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:20:06,945 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:20:07,435 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:20:10,722 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:20:12,436 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:20:15,723 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:20:17,437 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:20:20,725 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:20:22,564 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:20:25,914 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:20:26,064 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:20:27,565 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:20:30,915 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:20:33,114 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:20:33,114 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:20:34,118 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:20:34,379 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:20:36,380 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:20:38,115 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:20:41,381 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:20:43,584 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:20:44,184 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:20:46,892 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:20:48,585 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:20:51,894 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:20:54,141 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:20:54,254 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:20:57,314 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:21:00,568 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:21:00,569 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:21:02,860 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:21:04,323 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:21:04,379 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:21:05,569 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:21:08,381 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:21:10,616 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:21:13,858 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:21:14,390 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:21:15,618 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:21:18,859 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:21:20,928 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:21:22,451 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:21:24,066 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:21:25,929 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:21:29,067 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:21:31,208 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:21:32,519 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:21:34,380 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:21:34,381 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:21:36,210 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:21:39,382 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:21:41,707 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:21:42,584 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:21:45,009 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:21:46,709 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:21:50,597 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:21:50,658 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:21:52,286 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:21:55,597 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:21:58,197 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:22:00,731 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:22:01,465 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:22:03,198 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:22:04,381 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:22:07,382 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:22:08,655 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:22:10,795 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:22:13,013 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:22:13,656 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:22:18,015 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:22:19,287 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:22:20,862 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:22:23,555 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:22:24,288 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:22:28,796 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:22:28,928 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:22:29,622 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:22:33,797 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:22:34,382 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:22:35,912 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:22:38,999 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:22:39,170 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:22:40,914 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:22:44,172 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:22:46,033 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:22:46,034 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:22:49,079 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:22:49,326 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:22:51,034 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:22:54,328 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:22:56,291 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:22:59,147 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:22:59,428 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:23:01,292 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:23:04,383 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:23:05,385 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:23:06,718 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:23:08,203 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:23:10,963 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:23:11,718 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:23:16,258 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:23:16,498 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:23:17,216 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:23:21,499 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:23:23,754 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:23:24,985 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 17:23:24,986 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 17:23:24,987 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 17:23:24,989 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 17:23:25,313 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 17:23:26,318 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:23:26,991 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:23:28,755 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:23:31,992 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:23:34,268 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:23:34,383 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:23:36,389 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:23:37,544 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:23:39,268 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:23:42,545 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:23:44,524 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:23:46,449 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:23:47,863 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:23:49,525 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:23:52,865 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:23:55,304 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:23:56,516 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:23:58,776 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:24:00,304 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:24:03,777 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:24:04,384 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:24:06,162 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:24:06,569 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:24:09,426 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:24:11,164 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:24:15,137 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:24:16,216 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:24:16,633 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:24:20,137 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:24:22,299 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:24:24,696 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:24:25,447 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:24:27,300 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:24:30,448 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:24:32,624 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:24:32,625 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:24:34,384 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:24:34,764 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:24:36,385 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:24:37,626 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:24:41,387 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:24:42,778 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:24:44,839 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:24:46,957 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:24:47,779 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:24:51,958 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:24:53,120 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:24:54,911 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:24:57,838 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:24:58,121 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:25:03,254 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:25:03,254 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:25:03,411 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:25:04,385 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:25:04,974 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:25:08,255 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:25:09,386 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:25:13,039 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:25:13,737 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:25:15,014 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:25:18,738 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:25:20,016 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:25:23,741 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:25:25,121 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:25:25,238 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:25:28,762 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:25:30,240 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:25:33,854 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:25:34,385 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:25:35,194 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:25:35,386 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:25:38,855 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:25:40,387 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:25:44,217 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:25:45,268 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:25:46,240 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:25:49,843 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:25:51,241 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:25:55,343 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:25:55,511 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:25:57,061 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:26:01,523 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:26:01,524 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:26:02,063 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:26:04,386 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:26:04,398 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:26:06,525 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:26:07,387 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:26:12,135 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:26:13,325 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:26:14,463 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:26:17,136 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:26:18,326 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:26:22,918 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:26:24,322 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:26:24,521 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:26:27,919 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:26:29,323 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:26:32,585 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:26:33,655 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:26:34,386 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:26:34,387 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:26:38,657 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:26:39,388 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:26:41,769 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 17:26:41,771 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 17:26:41,772 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 17:26:41,774 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 17:26:42,660 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:26:42,660 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 17:26:44,540 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:26:44,670 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:26:44,776 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:26:49,778 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:26:50,008 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:26:50,008 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:26:52,721 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:26:55,015 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:26:55,357 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:27:00,358 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:27:00,655 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:27:00,656 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:27:02,782 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:27:04,387 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:27:05,388 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:27:05,725 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:27:10,390 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:27:11,166 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:27:12,844 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:27:15,418 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:27:16,167 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:27:20,729 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:27:20,899 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:27:21,517 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:27:25,746 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:27:26,518 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:27:30,830 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:27:30,963 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:27:31,574 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:27:34,387 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:27:36,388 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:27:36,628 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:27:41,027 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:27:41,634 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:27:42,082 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:27:46,635 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:27:47,100 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:27:51,099 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:27:51,707 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:27:53,056 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:27:56,709 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:27:58,058 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:28:01,168 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:28:02,522 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:28:03,086 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:28:04,399 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:28:07,523 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:28:08,400 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:28:13,325 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:28:13,402 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:28:17,265 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:28:18,326 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:28:18,767 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:28:23,768 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:28:24,117 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:28:27,340 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:28:29,118 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:28:29,377 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:28:34,388 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:28:34,389 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:28:34,598 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:28:35,391 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:28:39,390 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:28:39,605 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:28:44,758 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:28:45,453 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:28:45,502 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:28:49,759 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:28:50,504 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:28:55,482 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:28:55,515 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:28:56,279 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:29:00,483 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:29:01,762 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:29:01,762 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:29:04,389 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:29:04,578 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:29:06,390 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:29:06,763 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:29:11,391 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:29:12,524 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:29:14,653 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:29:16,701 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:29:17,525 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:29:21,702 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:29:22,712 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:29:22,905 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:29:27,149 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:29:27,925 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:29:32,774 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:29:33,128 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:29:33,763 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:29:34,389 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:29:38,391 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:29:40,327 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:29:42,838 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:29:43,633 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:29:45,328 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:29:48,634 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:29:51,014 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:29:51,014 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:29:52,907 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:29:54,331 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:29:56,015 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:29:59,333 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:30:00,782 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 17:30:00,783 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 17:30:00,785 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 17:30:00,789 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 17:30:00,963 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 17:30:01,571 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:30:01,967 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:30:02,972 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:30:04,390 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:30:04,390 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:30:06,572 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:30:09,391 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:30:11,572 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:30:14,392 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:30:16,573 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:30:19,393 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:30:21,574 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:30:24,393 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:30:26,575 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:30:29,394 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:30:31,576 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:30:34,394 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:30:35,394 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:30:36,577 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:30:40,395 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:30:41,578 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:30:45,396 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:30:46,578 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:30:50,397 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:30:51,580 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:30:55,397 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:30:56,580 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:31:00,398 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:31:01,581 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:31:04,393 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:31:06,394 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:31:06,582 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:31:11,395 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:31:11,583 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:31:16,396 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:31:16,584 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:31:21,396 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:31:21,584 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:31:26,397 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:31:26,585 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:31:31,398 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:31:31,586 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:31:34,394 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:31:36,587 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:31:37,395 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:31:41,588 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:31:42,395 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:31:46,589 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:31:47,396 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:31:51,589 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:31:52,397 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:31:56,590 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:31:57,398 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:32:01,591 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:32:02,398 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:32:04,395 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:32:06,592 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:32:08,396 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:32:11,592 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:32:13,397 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:32:16,593 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:32:18,397 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:32:21,594 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:32:23,398 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:32:26,595 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:32:28,399 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:32:31,596 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:32:33,399 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:32:34,397 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:32:36,596 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:32:39,399 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:32:41,599 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:32:44,399 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:32:46,598 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:32:49,400 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:32:51,599 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:32:54,401 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:32:56,600 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:32:59,402 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:33:01,601 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:33:04,398 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:33:05,399 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:33:06,601 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:33:10,400 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:33:11,602 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:33:15,401 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:33:16,603 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:33:20,401 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:33:21,604 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:33:25,402 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:33:26,604 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:33:30,403 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:33:31,605 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:33:34,399 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:33:36,399 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:33:36,606 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:33:41,400 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:33:41,607 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:33:46,401 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:33:46,608 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:33:51,402 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:33:51,609 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:33:56,402 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:33:56,610 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:34:01,403 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:34:01,610 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:34:04,399 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:34:06,611 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:34:07,400 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:34:11,612 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:34:12,401 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:34:16,613 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:34:17,401 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:34:21,614 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:34:22,402 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:34:26,615 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:34:27,408 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:34:31,616 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:34:32,409 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:34:34,399 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:34:36,617 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:34:38,400 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:34:41,617 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:34:43,401 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:34:46,618 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:34:48,402 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:34:51,619 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:34:53,402 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:34:56,620 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:34:58,403 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:35:01,621 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:35:03,404 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:35:04,401 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:35:06,622 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:35:09,403 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:35:11,622 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:35:14,403 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:35:16,623 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:35:19,404 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:35:21,624 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:35:24,405 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:35:26,625 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:35:29,406 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:35:31,626 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:35:34,402 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:35:35,403 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:35:36,626 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:35:40,404 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:35:41,627 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:35:45,404 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:35:46,628 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:35:50,405 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:35:51,629 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:35:55,406 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:35:56,630 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:36:00,407 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:36:01,630 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:36:04,402 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:36:06,403 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:36:06,631 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:36:11,404 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:36:11,632 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:36:16,405 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:36:16,633 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:36:21,405 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:36:21,634 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:36:26,406 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:36:26,635 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:36:31,407 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:36:31,635 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:36:34,402 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:36:36,636 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:36:37,403 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:36:41,637 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:36:42,404 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:36:46,638 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:36:47,405 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:36:51,639 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:36:52,406 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:36:56,640 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:36:57,406 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:37:01,640 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:37:02,407 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:37:04,403 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:37:06,641 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:37:08,404 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:37:11,642 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:37:13,405 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:37:16,643 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:37:18,405 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:37:21,644 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:37:23,406 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:37:26,644 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:37:28,407 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:37:31,645 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:37:33,408 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:37:34,404 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:37:36,646 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:37:39,405 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:37:41,647 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:37:44,406 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:37:46,648 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:37:49,406 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:37:51,648 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:37:54,407 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:37:56,649 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:37:59,408 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:38:01,650 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:38:04,404 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:38:05,405 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:38:06,651 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:38:10,406 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:38:11,652 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:38:15,407 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:38:16,653 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:38:20,407 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:38:21,654 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:38:25,408 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:38:26,654 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:38:30,409 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:38:31,655 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:38:34,404 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:38:36,405 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:38:36,656 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:38:41,406 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:38:41,657 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:38:46,406 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:38:46,657 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:38:51,407 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:38:51,658 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:38:56,408 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:38:56,659 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:39:01,408 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:39:01,660 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:39:04,405 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:39:06,661 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:39:07,406 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:39:11,661 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:39:12,407 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:39:16,662 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:39:17,407 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:39:21,663 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:39:22,408 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:39:26,664 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:39:27,409 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:39:31,665 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:39:32,410 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:39:34,405 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:39:36,666 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:39:38,406 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:39:41,667 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:39:43,407 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:39:46,668 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:39:48,408 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:39:51,669 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:39:53,408 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:39:56,670 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:39:58,409 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:40:01,671 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:40:03,410 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:40:04,406 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:40:06,672 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:40:09,407 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:40:11,673 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:40:11,788 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 17:40:11,793 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 17:40:11,794 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 17:40:11,797 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 17:40:12,725 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:40:12,726 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 17:40:13,730 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:40:14,799 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:40:16,674 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:40:19,800 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:40:21,675 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:40:24,801 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:40:26,675 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:40:29,801 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:40:31,676 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:40:34,417 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:40:35,418 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:40:36,677 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:40:40,419 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:40:41,679 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:40:43,892 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:40:46,301 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:40:46,780 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:40:47,022 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:40:51,302 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:40:52,189 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:40:56,302 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:40:57,824 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:41:01,303 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:41:03,467 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:41:04,407 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:41:06,408 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:41:09,424 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:41:11,408 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:41:15,507 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:41:16,409 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:41:21,410 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:41:23,175 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:41:24,121 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:41:25,125 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:41:27,243 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:41:29,475 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:41:31,159 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:41:32,498 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:41:34,408 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:41:36,886 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:41:38,409 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:41:43,410 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:41:48,410 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:41:50,445 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:41:50,445 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:41:50,446 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:41:53,262 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:41:53,461 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:41:55,446 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:41:58,462 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:42:00,447 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:42:03,462 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:42:04,408 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:42:05,447 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:42:09,409 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:42:10,448 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:42:14,410 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:42:15,449 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:42:19,410 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:42:20,450 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:42:24,411 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:42:25,450 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:42:29,412 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:42:30,451 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:42:34,409 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:42:35,409 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:42:35,452 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:42:40,410 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:42:40,453 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:42:45,411 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:42:45,453 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:42:50,454 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:42:50,477 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:42:50,522 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:42:55,524 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:42:56,437 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:42:56,437 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:43:00,544 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:43:00,931 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:43:01,455 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:43:04,409 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:43:06,410 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:43:06,744 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:43:09,596 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:43:11,782 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:43:12,355 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:43:17,356 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:43:17,868 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:43:21,672 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:43:22,594 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:43:22,870 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:43:27,595 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:43:28,137 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:43:29,737 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:43:33,138 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:43:33,385 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:43:34,410 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:43:38,411 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:43:38,941 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:43:38,941 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:43:39,802 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:43:43,943 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:43:44,285 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:43:48,973 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:43:49,286 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:43:49,866 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:43:53,974 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:43:54,397 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:43:59,583 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:43:59,931 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:44:00,294 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:44:04,411 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:44:04,584 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:44:05,425 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:44:09,790 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:44:10,004 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:44:11,140 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:44:14,791 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:44:16,141 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:44:20,069 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:44:20,077 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:44:21,511 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:44:25,078 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:44:26,512 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:44:30,136 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:44:30,914 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:44:32,405 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:44:34,411 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:44:35,915 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:44:37,412 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:44:41,492 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:44:42,207 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:44:42,995 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:44:46,513 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:44:47,997 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:44:51,635 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:44:52,269 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:44:53,022 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:44:56,637 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:44:58,023 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:45:02,331 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:45:02,561 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:45:03,642 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:45:04,411 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:45:09,022 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:45:09,024 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:45:09,024 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:45:09,413 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:45:12,397 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:45:14,024 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:45:15,404 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:45:19,102 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:45:20,406 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:45:22,454 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:45:24,103 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:45:25,434 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:45:29,792 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:45:31,396 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:45:32,521 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:45:34,412 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:45:34,793 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:45:36,413 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:45:39,894 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:45:42,153 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:45:42,580 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:45:44,895 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:45:47,154 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:45:50,463 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:45:52,652 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:45:52,927 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:45:55,464 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:45:57,929 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:46:00,714 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:46:01,322 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:46:03,590 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:46:04,412 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:46:06,323 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:46:09,414 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:46:09,740 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 17:46:09,741 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 17:46:09,741 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 17:46:09,744 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 17:46:09,783 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 17:46:11,340 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:46:11,793 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:46:14,746 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:46:16,341 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:46:19,748 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:46:21,864 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:46:21,990 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:46:25,398 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:46:26,991 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:46:30,399 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:46:31,930 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:46:32,794 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:46:34,413 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:46:35,414 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:46:37,795 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:46:40,415 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:46:41,994 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:46:42,958 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:46:42,959 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:46:46,400 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:46:47,960 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:46:52,061 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:46:52,120 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:46:53,857 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:46:57,121 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:46:59,581 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:47:02,133 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:47:02,979 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:47:04,413 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:47:04,583 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:47:08,415 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:47:09,820 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:47:12,282 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:47:14,158 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:47:14,821 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:47:19,160 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:47:20,011 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:47:22,400 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:47:24,479 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:47:25,012 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:47:29,481 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:47:30,538 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:47:32,455 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:47:34,414 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:47:35,415 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:47:35,539 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:47:40,958 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:47:41,379 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:47:42,526 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:47:46,130 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:47:46,380 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:47:51,429 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:47:51,798 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:47:54,593 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:47:56,799 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:47:57,360 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:48:02,797 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:48:03,044 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:48:04,414 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:48:04,665 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:48:07,798 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:48:08,416 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:48:12,728 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:48:13,719 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:48:14,349 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:48:14,355 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:48:18,720 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:48:19,356 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:48:24,316 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:48:24,793 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:48:24,974 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:48:29,317 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:48:29,976 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:48:34,415 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:48:34,416 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:48:34,879 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:48:35,652 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:48:39,417 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:48:41,888 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:48:41,889 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:48:41,890 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:48:43,952 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:48:45,237 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:48:46,890 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:48:50,238 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:48:52,683 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:48:54,020 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:48:55,915 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:48:57,684 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:49:00,917 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:49:03,153 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:49:04,167 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:49:04,416 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:49:06,417 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:49:08,160 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:49:11,418 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:49:14,045 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:49:14,236 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:49:17,248 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:49:19,047 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:49:22,250 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:49:24,299 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:49:24,886 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:49:28,198 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:49:31,398 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:49:31,398 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:49:32,665 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 17:49:32,666 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 17:49:32,667 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 17:49:32,669 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 17:49:33,357 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:49:33,358 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 17:49:33,670 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:49:34,363 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:49:34,416 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:49:36,399 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:49:39,417 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:49:41,580 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:49:44,431 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:49:44,734 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:49:46,581 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:49:49,735 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:49:52,274 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:49:54,510 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:49:54,747 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:49:57,275 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:49:59,749 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:50:03,266 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:50:04,417 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:50:04,571 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:50:05,417 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:50:08,267 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:50:10,419 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:50:14,046 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:50:14,637 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:50:16,020 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:50:19,163 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:50:21,022 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:50:24,713 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:50:24,812 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:50:26,267 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:50:29,812 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:50:31,269 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:50:34,419 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:50:34,772 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:50:35,053 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:50:36,421 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:50:40,054 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:50:41,423 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:50:44,836 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:50:45,204 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:50:46,973 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:50:50,205 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:50:51,974 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:50:55,866 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:50:56,910 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:50:57,181 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:51:00,868 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:51:02,182 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:51:04,418 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:51:06,689 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:51:06,987 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:51:07,419 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:51:12,420 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:51:12,501 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:51:12,502 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:51:12,502 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:51:15,038 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:51:17,503 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:51:17,754 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:51:22,756 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:51:22,859 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:51:27,130 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:51:27,860 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:51:28,239 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:51:33,240 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:51:33,490 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:51:34,176 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:51:34,418 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:51:38,419 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:51:38,497 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:51:43,421 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:51:43,502 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:51:44,239 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:51:48,461 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:51:49,327 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:51:54,109 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:51:54,316 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:51:54,789 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:51:59,110 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:51:59,791 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:52:04,384 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:52:04,419 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:52:04,420 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:52:05,572 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:52:09,420 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:52:14,915 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:52:14,916 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:52:16,824 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:52:16,933 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:52:19,919 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:52:21,964 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:52:24,987 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:52:25,903 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:52:26,965 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:52:30,904 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:52:32,336 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:52:32,337 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:52:34,419 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:52:35,049 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:52:36,420 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:52:37,338 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:52:41,421 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:52:42,503 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:52:42,504 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:52:45,121 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:52:47,417 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:52:48,053 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:52:52,419 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:52:53,787 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: partial_history 2023-11-23 17:52:53,789 DEBUG SenderThread:3236 [sender.py:send():380] send: history 2023-11-23 17:52:53,789 DEBUG SenderThread:3236 [sender.py:send_request():407] send_request: summary_record 2023-11-23 17:52:53,797 INFO SenderThread:3236 [sender.py:_save_file():1380] saving file wandb-summary.json with policy end 2023-11-23 17:52:53,956 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:52:54,185 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/wandb-summary.json 2023-11-23 17:52:55,192 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:52:57,800 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:52:58,957 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:53:02,802 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:53:04,240 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:53:04,420 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:53:05,368 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:53:08,430 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:53:09,241 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:53:13,432 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:53:14,477 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:53:15,426 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:53:18,863 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:53:19,478 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:53:24,363 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:53:25,146 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:53:25,484 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:53:29,364 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:53:30,148 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:53:34,420 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:53:34,421 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:53:35,550 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:53:35,875 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:53:39,422 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:53:41,646 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:53:45,016 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:53:45,611 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:53:46,647 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:53:50,017 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:53:52,358 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:53:55,633 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:53:55,708 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:53:57,359 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:54:00,634 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:54:02,828 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:54:04,421 DEBUG SenderThread:3236 [sender.py:send():380] send: stats 2023-11-23 17:54:05,770 INFO Thread-12 :3236 [dir_watcher.py:_on_file_modified():288] file/dir modified: /mnt/e/wandb/run-20231122_234140-blvl5h7j/files/output.log 2023-11-23 17:54:06,423 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:54:07,832 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive 2023-11-23 17:54:11,424 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: status_report 2023-11-23 17:54:13,526 DEBUG HandlerThread:3236 [handler.py:handle_request():146] handle_request: keepalive