File size: 43,124 Bytes
966d24c 6a026ce 966d24c 35c352c |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 |
2024-05-26 14:13:05,028 INFO StreamThr :225 [internal.py:wandb_internal():86] W&B internal server running at pid: 225, started at: 2024-05-26 14:13:05.028130
2024-05-26 14:13:05,030 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status
2024-05-26 14:13:06,044 INFO WriterThread:225 [datastore.py:open_for_write():87] open: /kaggle/working/wandb/run-20240526_141305-wfyhopk0/run-wfyhopk0.wandb
2024-05-26 14:13:06,044 DEBUG SenderThread:225 [sender.py:send():379] send: header
2024-05-26 14:13:06,047 DEBUG SenderThread:225 [sender.py:send():379] send: run
2024-05-26 14:13:06,468 INFO SenderThread:225 [dir_watcher.py:__init__():211] watching files in: /kaggle/working/wandb/run-20240526_141305-wfyhopk0/files
2024-05-26 14:13:06,469 INFO SenderThread:225 [sender.py:_start_run_threads():1124] run started: wfyhopk0 with start time 1716732785.02907
2024-05-26 14:13:06,477 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: check_version
2024-05-26 14:13:06,477 DEBUG SenderThread:225 [sender.py:send_request():406] send_request: check_version
2024-05-26 14:13:06,534 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: run_start
2024-05-26 14:13:06,545 DEBUG HandlerThread:225 [system_info.py:__init__():26] System info init
2024-05-26 14:13:06,545 DEBUG HandlerThread:225 [system_info.py:__init__():41] System info init done
2024-05-26 14:13:06,545 INFO HandlerThread:225 [system_monitor.py:start():194] Starting system monitor
2024-05-26 14:13:06,546 INFO SystemMonitor:225 [system_monitor.py:_start():158] Starting system asset monitoring threads
2024-05-26 14:13:06,546 INFO HandlerThread:225 [system_monitor.py:probe():214] Collecting system info
2024-05-26 14:13:06,546 INFO SystemMonitor:225 [interfaces.py:start():190] Started cpu monitoring
2024-05-26 14:13:06,547 INFO SystemMonitor:225 [interfaces.py:start():190] Started disk monitoring
2024-05-26 14:13:06,549 INFO SystemMonitor:225 [interfaces.py:start():190] Started gpu monitoring
2024-05-26 14:13:06,550 INFO SystemMonitor:225 [interfaces.py:start():190] Started memory monitoring
2024-05-26 14:13:06,550 INFO SystemMonitor:225 [interfaces.py:start():190] Started network monitoring
2024-05-26 14:13:06,561 DEBUG HandlerThread:225 [system_info.py:probe():150] Probing system
2024-05-26 14:13:06,563 DEBUG HandlerThread:225 [gitlib.py:_init_repo():56] git repository is invalid
2024-05-26 14:13:06,563 DEBUG HandlerThread:225 [system_info.py:probe():198] Probing system done
2024-05-26 14:13:06,563 DEBUG HandlerThread:225 [system_monitor.py:probe():223] {'os': 'Linux-5.15.133+-x86_64-with-glibc2.31', 'python': '3.10.13', 'heartbeatAt': '2024-05-26T14:13:06.561396', 'startedAt': '2024-05-26T14:13:05.022165', 'docker': None, 'cuda': None, 'args': (), 'state': 'running', 'program': 'kaggle.ipynb', 'codePathLocal': None, 'root': '/kaggle/working', 'host': '142d81d78b72', 'username': 'root', 'executable': '/opt/conda/bin/python3.10', 'cpu_count': 2, 'cpu_count_logical': 4, 'cpu_freq': {'current': 2000.164, 'min': 0.0, 'max': 0.0}, 'cpu_freq_per_core': [{'current': 2000.164, 'min': 0.0, 'max': 0.0}, {'current': 2000.164, 'min': 0.0, 'max': 0.0}, {'current': 2000.164, 'min': 0.0, 'max': 0.0}, {'current': 2000.164, 'min': 0.0, 'max': 0.0}], 'disk': {'/': {'total': 8062.387607574463, 'used': 5597.954666137695}}, 'gpu': 'Tesla P100-PCIE-16GB', 'gpu_count': 1, 'gpu_devices': [{'name': 'Tesla P100-PCIE-16GB', 'memory_total': 17179869184}], 'memory': {'total': 31.357559204101562}}
2024-05-26 14:13:06,563 INFO HandlerThread:225 [system_monitor.py:probe():224] Finished collecting system info
2024-05-26 14:13:06,563 INFO HandlerThread:225 [system_monitor.py:probe():227] Publishing system info
2024-05-26 14:13:06,563 DEBUG HandlerThread:225 [system_info.py:_save_conda():207] Saving list of conda packages installed into the current environment
2024-05-26 14:13:07,470 INFO Thread-12 :225 [dir_watcher.py:_on_file_created():271] file/dir created: /kaggle/working/wandb/run-20240526_141305-wfyhopk0/files/conda-environment.yaml
2024-05-26 14:13:21,580 ERROR HandlerThread:225 [system_info.py:_save_conda():221] Error saving conda packages: Command '['conda', 'env', 'export']' timed out after 15 seconds
Traceback (most recent call last):
File "/opt/conda/lib/python3.10/site-packages/wandb/sdk/internal/system/system_info.py", line 214, in _save_conda
subprocess.call(
File "/opt/conda/lib/python3.10/subprocess.py", line 347, in call
return p.wait(timeout=timeout)
File "/opt/conda/lib/python3.10/subprocess.py", line 1209, in wait
return self._wait(timeout=timeout)
File "/opt/conda/lib/python3.10/subprocess.py", line 1951, in _wait
raise TimeoutExpired(self.args, timeout)
subprocess.TimeoutExpired: Command '['conda', 'env', 'export']' timed out after 15 seconds
2024-05-26 14:13:21,583 DEBUG HandlerThread:225 [system_info.py:_save_conda():222] Saving conda packages done
2024-05-26 14:13:21,583 INFO HandlerThread:225 [system_monitor.py:probe():229] Finished publishing system info
2024-05-26 14:13:21,591 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:13:21,592 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: keepalive
2024-05-26 14:13:21,592 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:13:21,592 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: keepalive
2024-05-26 14:13:21,592 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:13:21,592 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: keepalive
2024-05-26 14:13:21,592 DEBUG SenderThread:225 [sender.py:send():379] send: files
2024-05-26 14:13:21,593 INFO SenderThread:225 [sender.py:_save_file():1390] saving file wandb-metadata.json with policy now
2024-05-26 14:13:22,239 INFO wandb-upload_0:225 [upload_job.py:push():131] Uploaded file /tmp/tmpfxjhfb1zwandb/k20mz4b6-wandb-metadata.json
2024-05-26 14:13:22,474 INFO Thread-12 :225 [dir_watcher.py:_on_file_created():271] file/dir created: /kaggle/working/wandb/run-20240526_141305-wfyhopk0/files/wandb-metadata.json
2024-05-26 14:13:22,557 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: python_packages
2024-05-26 14:13:22,558 DEBUG SenderThread:225 [sender.py:send_request():406] send_request: python_packages
2024-05-26 14:13:22,560 DEBUG SenderThread:225 [sender.py:send():379] send: telemetry
2024-05-26 14:13:22,573 DEBUG SenderThread:225 [sender.py:send():379] send: config
2024-05-26 14:13:22,575 DEBUG SenderThread:225 [sender.py:send():379] send: metric
2024-05-26 14:13:22,575 DEBUG SenderThread:225 [sender.py:send():379] send: telemetry
2024-05-26 14:13:22,575 DEBUG SenderThread:225 [sender.py:send():379] send: metric
2024-05-26 14:13:22,575 WARNING SenderThread:225 [sender.py:send_metric():1341] Seen metric with glob (shouldn't happen)
2024-05-26 14:13:22,575 DEBUG SenderThread:225 [sender.py:send():379] send: telemetry
2024-05-26 14:13:22,576 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: stop_status
2024-05-26 14:13:22,577 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: internal_messages
2024-05-26 14:13:22,577 DEBUG SenderThread:225 [sender.py:send_request():406] send_request: stop_status
2024-05-26 14:13:23,474 INFO Thread-12 :225 [dir_watcher.py:_on_file_created():271] file/dir created: /kaggle/working/wandb/run-20240526_141305-wfyhopk0/files/requirements.txt
2024-05-26 14:13:23,475 INFO Thread-12 :225 [dir_watcher.py:_on_file_created():271] file/dir created: /kaggle/working/wandb/run-20240526_141305-wfyhopk0/files/output.log
2024-05-26 14:13:25,475 INFO Thread-12 :225 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240526_141305-wfyhopk0/files/output.log
2024-05-26 14:13:25,533 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:13:30,534 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:13:35,540 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:13:36,479 INFO Thread-12 :225 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240526_141305-wfyhopk0/files/config.yaml
2024-05-26 14:13:37,729 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: stop_status
2024-05-26 14:13:37,730 DEBUG SenderThread:225 [sender.py:send_request():406] send_request: stop_status
2024-05-26 14:13:37,732 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: internal_messages
2024-05-26 14:13:40,943 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:13:45,944 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:13:50,945 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:13:52,677 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: stop_status
2024-05-26 14:13:52,677 DEBUG SenderThread:225 [sender.py:send_request():406] send_request: stop_status
2024-05-26 14:13:52,717 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: internal_messages
2024-05-26 14:13:56,897 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:14:01,898 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:14:06,550 DEBUG SystemMonitor:225 [system_monitor.py:_start():172] Starting system metrics aggregation loop
2024-05-26 14:14:06,552 DEBUG SenderThread:225 [sender.py:send():379] send: stats
2024-05-26 14:14:07,553 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:14:07,975 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: internal_messages
2024-05-26 14:14:07,976 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: stop_status
2024-05-26 14:14:07,977 DEBUG SenderThread:225 [sender.py:send_request():406] send_request: stop_status
2024-05-26 14:14:13,194 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:14:18,196 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:14:22,965 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: internal_messages
2024-05-26 14:14:22,974 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: stop_status
2024-05-26 14:14:22,975 DEBUG SenderThread:225 [sender.py:send_request():406] send_request: stop_status
2024-05-26 14:14:24,158 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:14:29,159 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:14:34,160 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:14:36,553 DEBUG SenderThread:225 [sender.py:send():379] send: stats
2024-05-26 14:14:37,966 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: internal_messages
2024-05-26 14:14:37,975 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: stop_status
2024-05-26 14:14:37,975 DEBUG SenderThread:225 [sender.py:send_request():406] send_request: stop_status
2024-05-26 14:14:39,207 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:14:44,208 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:14:49,210 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:14:53,045 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: internal_messages
2024-05-26 14:14:53,066 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: stop_status
2024-05-26 14:14:53,067 DEBUG SenderThread:225 [sender.py:send_request():406] send_request: stop_status
2024-05-26 14:14:54,265 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:14:59,266 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:15:04,267 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:15:06,554 DEBUG SenderThread:225 [sender.py:send():379] send: stats
2024-05-26 14:15:08,034 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: stop_status
2024-05-26 14:15:08,034 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: internal_messages
2024-05-26 14:15:08,035 DEBUG SenderThread:225 [sender.py:send_request():406] send_request: stop_status
2024-05-26 14:15:10,176 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:15:15,177 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:15:20,179 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:15:23,224 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: stop_status
2024-05-26 14:15:23,225 DEBUG SenderThread:225 [sender.py:send_request():406] send_request: stop_status
2024-05-26 14:15:23,228 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: internal_messages
2024-05-26 14:15:25,427 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:15:30,428 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:15:35,428 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:15:36,555 DEBUG SenderThread:225 [sender.py:send():379] send: stats
2024-05-26 14:15:38,194 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: internal_messages
2024-05-26 14:15:38,195 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: stop_status
2024-05-26 14:15:38,195 DEBUG SenderThread:225 [sender.py:send_request():406] send_request: stop_status
2024-05-26 14:15:41,356 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:15:46,356 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:15:51,357 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:15:53,194 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: stop_status
2024-05-26 14:15:53,195 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: internal_messages
2024-05-26 14:15:53,195 DEBUG SenderThread:225 [sender.py:send_request():406] send_request: stop_status
2024-05-26 14:15:56,402 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:16:01,403 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:16:06,404 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:16:06,555 DEBUG SenderThread:225 [sender.py:send():379] send: stats
2024-05-26 14:16:08,194 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: internal_messages
2024-05-26 14:16:08,195 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: stop_status
2024-05-26 14:16:08,195 DEBUG SenderThread:225 [sender.py:send_request():406] send_request: stop_status
2024-05-26 14:16:11,441 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:16:16,442 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:16:21,443 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:16:23,331 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: stop_status
2024-05-26 14:16:23,332 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: internal_messages
2024-05-26 14:16:23,332 DEBUG SenderThread:225 [sender.py:send_request():406] send_request: stop_status
2024-05-26 14:16:26,538 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:16:31,539 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:16:36,540 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:16:36,556 DEBUG SenderThread:225 [sender.py:send():379] send: stats
2024-05-26 14:16:38,437 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: internal_messages
2024-05-26 14:16:38,438 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: stop_status
2024-05-26 14:16:38,439 DEBUG SenderThread:225 [sender.py:send_request():406] send_request: stop_status
2024-05-26 14:16:41,579 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:16:46,580 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:16:51,581 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:16:53,400 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: internal_messages
2024-05-26 14:16:53,401 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: stop_status
2024-05-26 14:16:53,401 DEBUG SenderThread:225 [sender.py:send_request():406] send_request: stop_status
2024-05-26 14:16:56,646 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:17:01,647 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:17:06,557 DEBUG SenderThread:225 [sender.py:send():379] send: stats
2024-05-26 14:17:07,558 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:17:08,401 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: internal_messages
2024-05-26 14:17:08,401 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: stop_status
2024-05-26 14:17:08,401 DEBUG SenderThread:225 [sender.py:send_request():406] send_request: stop_status
2024-05-26 14:17:12,573 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:17:17,574 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:17:22,575 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:17:23,457 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: stop_status
2024-05-26 14:17:23,457 DEBUG SenderThread:225 [sender.py:send_request():406] send_request: stop_status
2024-05-26 14:17:23,461 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: internal_messages
2024-05-26 14:17:27,603 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:17:32,604 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:17:36,558 DEBUG SenderThread:225 [sender.py:send():379] send: stats
2024-05-26 14:17:38,511 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:17:38,512 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: internal_messages
2024-05-26 14:17:38,512 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: stop_status
2024-05-26 14:17:38,513 DEBUG SenderThread:225 [sender.py:send_request():406] send_request: stop_status
2024-05-26 14:17:43,750 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:17:48,751 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:17:53,602 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: stop_status
2024-05-26 14:17:53,602 DEBUG SenderThread:225 [sender.py:send_request():406] send_request: stop_status
2024-05-26 14:17:53,643 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: internal_messages
2024-05-26 14:17:53,851 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:17:58,853 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:18:03,854 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:18:06,559 DEBUG SenderThread:225 [sender.py:send():379] send: stats
2024-05-26 14:18:08,729 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: stop_status
2024-05-26 14:18:08,730 DEBUG SenderThread:225 [sender.py:send_request():406] send_request: stop_status
2024-05-26 14:18:08,769 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: internal_messages
2024-05-26 14:18:08,886 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:18:13,886 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:18:18,887 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:18:23,707 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: stop_status
2024-05-26 14:18:23,708 DEBUG SenderThread:225 [sender.py:send_request():406] send_request: stop_status
2024-05-26 14:18:23,748 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: internal_messages
2024-05-26 14:18:23,935 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:18:28,935 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:18:33,937 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:18:34,444 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: partial_history
2024-05-26 14:18:34,446 DEBUG SenderThread:225 [sender.py:send():379] send: metric
2024-05-26 14:18:34,446 DEBUG SenderThread:225 [sender.py:send():379] send: metric
2024-05-26 14:18:34,447 DEBUG SenderThread:225 [sender.py:send():379] send: metric
2024-05-26 14:18:34,447 DEBUG SenderThread:225 [sender.py:send():379] send: metric
2024-05-26 14:18:34,447 DEBUG SenderThread:225 [sender.py:send():379] send: history
2024-05-26 14:18:34,447 DEBUG SenderThread:225 [sender.py:send_request():406] send_request: summary_record
2024-05-26 14:18:34,447 INFO SenderThread:225 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
2024-05-26 14:18:34,595 INFO Thread-12 :225 [dir_watcher.py:_on_file_created():271] file/dir created: /kaggle/working/wandb/run-20240526_141305-wfyhopk0/files/wandb-summary.json
2024-05-26 14:18:36,560 DEBUG SenderThread:225 [sender.py:send():379] send: stats
2024-05-26 14:18:38,725 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: stop_status
2024-05-26 14:18:38,726 DEBUG SenderThread:225 [sender.py:send_request():406] send_request: stop_status
2024-05-26 14:18:38,728 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: internal_messages
2024-05-26 14:18:38,980 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:18:43,989 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:18:44,541 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: partial_history
2024-05-26 14:18:44,543 DEBUG SenderThread:225 [sender.py:send():379] send: metric
2024-05-26 14:18:44,543 DEBUG SenderThread:225 [sender.py:send():379] send: metric
2024-05-26 14:18:44,544 DEBUG SenderThread:225 [sender.py:send():379] send: metric
2024-05-26 14:18:44,544 DEBUG SenderThread:225 [sender.py:send():379] send: metric
2024-05-26 14:18:44,544 DEBUG SenderThread:225 [sender.py:send():379] send: metric
2024-05-26 14:18:44,544 DEBUG SenderThread:225 [sender.py:send():379] send: history
2024-05-26 14:18:44,544 DEBUG SenderThread:225 [sender.py:send_request():406] send_request: summary_record
2024-05-26 14:18:44,545 INFO SenderThread:225 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
2024-05-26 14:18:44,599 INFO Thread-12 :225 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240526_141305-wfyhopk0/files/config.yaml
2024-05-26 14:18:44,599 INFO Thread-12 :225 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240526_141305-wfyhopk0/files/wandb-summary.json
2024-05-26 14:18:47,600 INFO Thread-12 :225 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240526_141305-wfyhopk0/files/output.log
2024-05-26 14:18:49,772 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:18:53,717 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: stop_status
2024-05-26 14:18:53,717 DEBUG SenderThread:225 [sender.py:send_request():406] send_request: stop_status
2024-05-26 14:18:53,728 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: internal_messages
2024-05-26 14:18:54,981 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:18:59,983 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:19:04,983 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:19:06,561 DEBUG SenderThread:225 [sender.py:send():379] send: stats
2024-05-26 14:19:08,998 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: internal_messages
2024-05-26 14:19:08,999 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: stop_status
2024-05-26 14:19:08,999 DEBUG SenderThread:225 [sender.py:send_request():406] send_request: stop_status
2024-05-26 14:19:10,173 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:19:15,182 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:19:15,611 INFO Thread-12 :225 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240526_141305-wfyhopk0/files/config.yaml
2024-05-26 14:19:20,444 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:19:24,110 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: internal_messages
2024-05-26 14:19:24,111 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: stop_status
2024-05-26 14:19:24,112 DEBUG SenderThread:225 [sender.py:send_request():406] send_request: stop_status
2024-05-26 14:19:26,358 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:19:31,359 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:19:36,360 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:19:36,561 DEBUG SenderThread:225 [sender.py:send():379] send: stats
2024-05-26 14:19:39,080 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: internal_messages
2024-05-26 14:19:39,094 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: stop_status
2024-05-26 14:19:39,095 DEBUG SenderThread:225 [sender.py:send_request():406] send_request: stop_status
2024-05-26 14:19:42,342 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:19:47,343 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:19:52,344 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:19:54,133 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: stop_status
2024-05-26 14:19:54,134 DEBUG SenderThread:225 [sender.py:send_request():406] send_request: stop_status
2024-05-26 14:19:54,134 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: internal_messages
2024-05-26 14:19:57,378 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:20:02,379 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:20:06,562 DEBUG SenderThread:225 [sender.py:send():379] send: stats
2024-05-26 14:20:07,563 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:20:09,161 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: stop_status
2024-05-26 14:20:09,162 DEBUG SenderThread:225 [sender.py:send_request():406] send_request: stop_status
2024-05-26 14:20:09,202 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: internal_messages
2024-05-26 14:20:13,403 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:20:18,404 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:20:23,405 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:20:24,256 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: internal_messages
2024-05-26 14:20:24,257 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: stop_status
2024-05-26 14:20:24,258 DEBUG SenderThread:225 [sender.py:send_request():406] send_request: stop_status
2024-05-26 14:20:28,448 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:20:33,449 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:20:36,563 DEBUG SenderThread:225 [sender.py:send():379] send: stats
2024-05-26 14:20:38,564 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:20:39,217 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: internal_messages
2024-05-26 14:20:39,229 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: stop_status
2024-05-26 14:20:39,229 DEBUG SenderThread:225 [sender.py:send_request():406] send_request: stop_status
2024-05-26 14:20:44,376 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:20:49,378 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:20:54,323 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: stop_status
2024-05-26 14:20:54,324 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: internal_messages
2024-05-26 14:20:54,325 DEBUG SenderThread:225 [sender.py:send_request():406] send_request: stop_status
2024-05-26 14:20:54,536 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:20:59,537 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:21:04,539 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:21:06,564 DEBUG SenderThread:225 [sender.py:send():379] send: stats
2024-05-26 14:21:09,282 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: internal_messages
2024-05-26 14:21:09,282 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: stop_status
2024-05-26 14:21:09,283 DEBUG SenderThread:225 [sender.py:send_request():406] send_request: stop_status
2024-05-26 14:21:10,414 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:21:15,415 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:21:20,416 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:21:24,408 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: stop_status
2024-05-26 14:21:24,408 DEBUG SenderThread:225 [sender.py:send_request():406] send_request: stop_status
2024-05-26 14:21:24,448 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: internal_messages
2024-05-26 14:21:25,636 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:21:30,637 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:21:35,638 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:21:36,565 DEBUG SenderThread:225 [sender.py:send():379] send: stats
2024-05-26 14:21:39,424 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: stop_status
2024-05-26 14:21:39,425 DEBUG SenderThread:225 [sender.py:send_request():406] send_request: stop_status
2024-05-26 14:21:39,465 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: internal_messages
2024-05-26 14:21:41,638 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:21:46,639 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:21:51,640 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:21:54,587 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: internal_messages
2024-05-26 14:21:54,588 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: stop_status
2024-05-26 14:21:54,588 DEBUG SenderThread:225 [sender.py:send_request():406] send_request: stop_status
2024-05-26 14:21:56,779 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:22:01,780 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:22:06,566 DEBUG SenderThread:225 [sender.py:send():379] send: stats
2024-05-26 14:22:07,567 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:22:09,559 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: stop_status
2024-05-26 14:22:09,560 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: internal_messages
2024-05-26 14:22:09,560 DEBUG SenderThread:225 [sender.py:send_request():406] send_request: stop_status
2024-05-26 14:22:12,730 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:22:17,730 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:22:22,731 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:22:24,792 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: stop_status
2024-05-26 14:22:24,794 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: internal_messages
2024-05-26 14:22:24,794 DEBUG SenderThread:225 [sender.py:send_request():406] send_request: stop_status
2024-05-26 14:22:28,053 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:22:33,054 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:22:36,567 DEBUG SenderThread:225 [sender.py:send():379] send: stats
2024-05-26 14:22:38,568 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:22:39,741 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: stop_status
2024-05-26 14:22:39,742 DEBUG SenderThread:225 [sender.py:send_request():406] send_request: stop_status
2024-05-26 14:22:39,782 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: internal_messages
2024-05-26 14:22:43,915 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:22:48,916 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:22:53,917 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:22:54,741 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: stop_status
2024-05-26 14:22:54,741 DEBUG SenderThread:225 [sender.py:send_request():406] send_request: stop_status
2024-05-26 14:22:54,782 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: internal_messages
2024-05-26 14:22:59,906 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:23:04,907 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:23:06,568 DEBUG SenderThread:225 [sender.py:send():379] send: stats
2024-05-26 14:23:09,808 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: stop_status
2024-05-26 14:23:09,809 DEBUG SenderThread:225 [sender.py:send_request():406] send_request: stop_status
2024-05-26 14:23:09,849 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: internal_messages
2024-05-26 14:23:10,024 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:23:15,025 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:23:20,026 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:23:24,784 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: stop_status
2024-05-26 14:23:24,785 DEBUG SenderThread:225 [sender.py:send_request():406] send_request: stop_status
2024-05-26 14:23:24,826 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: internal_messages
2024-05-26 14:23:26,014 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:23:31,015 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:23:36,016 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:23:36,569 DEBUG SenderThread:225 [sender.py:send():379] send: stats
2024-05-26 14:23:39,780 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: internal_messages
2024-05-26 14:23:39,784 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: stop_status
2024-05-26 14:23:39,784 DEBUG SenderThread:225 [sender.py:send_request():406] send_request: stop_status
2024-05-26 14:23:41,930 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:23:46,931 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:23:51,933 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:23:55,089 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: stop_status
2024-05-26 14:23:55,090 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: internal_messages
2024-05-26 14:23:55,091 DEBUG SenderThread:225 [sender.py:send_request():406] send_request: stop_status
2024-05-26 14:23:55,427 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: partial_history
2024-05-26 14:23:55,428 DEBUG SenderThread:225 [sender.py:send():379] send: history
2024-05-26 14:23:55,428 DEBUG SenderThread:225 [sender.py:send_request():406] send_request: summary_record
2024-05-26 14:23:55,429 INFO SenderThread:225 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
2024-05-26 14:23:55,720 INFO Thread-12 :225 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240526_141305-wfyhopk0/files/wandb-summary.json
2024-05-26 14:23:57,430 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:24:02,431 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:24:05,670 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: partial_history
2024-05-26 14:24:05,672 DEBUG SenderThread:225 [sender.py:send():379] send: history
2024-05-26 14:24:05,672 DEBUG SenderThread:225 [sender.py:send_request():406] send_request: summary_record
2024-05-26 14:24:05,672 INFO SenderThread:225 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
2024-05-26 14:24:05,725 INFO Thread-12 :225 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240526_141305-wfyhopk0/files/wandb-summary.json
2024-05-26 14:24:06,569 DEBUG SenderThread:225 [sender.py:send():379] send: stats
2024-05-26 14:24:07,570 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:24:09,929 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: internal_messages
2024-05-26 14:24:09,930 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: stop_status
2024-05-26 14:24:09,931 DEBUG SenderThread:225 [sender.py:send_request():406] send_request: stop_status
2024-05-26 14:24:10,727 INFO Thread-12 :225 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240526_141305-wfyhopk0/files/output.log
2024-05-26 14:24:13,110 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:24:18,111 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
2024-05-26 14:24:23,113 DEBUG HandlerThread:225 [handler.py:handle_request():146] handle_request: status_report
|