Hugging Face's logo

sanchit-gandhi
/
distil-zephyr-1.5b-dpo-ultrafeedback-200k

Text Generation
Transformers
TensorBoard
mistral
conversational
Inference Endpoints
text-generation-inference
Model card Files Metrics Community
sanchit-gandhi HF staff commited on
Commit
b555414
•
1 Parent(s): 718d1ae

Training in progress, step 200

Browse files
Files changed (6) hide show
  1. runs/Apr26_16-38-17_ip-26-0-161-178/events.out.tfevents.1714149975.ip-26-0-161-178.306341.0 +2 -2
  2. wandb/debug-internal.log +162 -0
  3. wandb/run-20240426_164617-71zld9et/files/output.log +80 -0
  4. wandb/run-20240426_164617-71zld9et/files/wandb-summary.json +1 -1
  5. wandb/run-20240426_164617-71zld9et/logs/debug-internal.log +162 -0
  6. wandb/run-20240426_164617-71zld9et/run-71zld9et.wandb +0 -0
runs/Apr26_16-38-17_ip-26-0-161-178/events.out.tfevents.1714149975.ip-26-0-161-178.306341.0 CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:1965c5ca22ae4f3f8b62fc21e894bc3ef67848a9d40a53379b8db44b2fdb9d04
3
- size 8887
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d149f5606ddf497f2a3d347d4b402fb75a76cdc684b60787dbe6918afcce0acf
3
+ size 12367
wandb/debug-internal.log CHANGED
@@ -241,3 +241,165 @@
241
  2024-04-26 16:48:54,535 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/config.yaml
242
  2024-04-26 16:48:58,655 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: status_report
243
  2024-04-26 16:49:03,655 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: status_report
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
241
  2024-04-26 16:48:54,535 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/config.yaml
242
  2024-04-26 16:48:58,655 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: status_report
243
  2024-04-26 16:49:03,655 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: status_report
244
+ 2024-04-26 16:49:06,546 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
245
+ 2024-04-26 16:49:07,059 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: stop_status
246
+ 2024-04-26 16:49:07,059 DEBUG SenderThread:307541 [sender.py:send_request():406] send_request: stop_status
247
+ 2024-04-26 16:49:07,059 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: internal_messages
248
+ 2024-04-26 16:49:08,548 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
249
+ 2024-04-26 16:49:08,661 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: status_report
250
+ 2024-04-26 16:49:10,549 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
251
+ 2024-04-26 16:49:12,551 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
252
+ 2024-04-26 16:49:13,870 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: status_report
253
+ 2024-04-26 16:49:14,553 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
254
+ 2024-04-26 16:49:16,555 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
255
+ 2024-04-26 16:49:17,536 DEBUG SenderThread:307541 [sender.py:send():379] send: stats
256
+ 2024-04-26 16:49:18,557 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
257
+ 2024-04-26 16:49:18,920 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: status_report
258
+ 2024-04-26 16:49:20,559 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
259
+ 2024-04-26 16:49:22,059 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: stop_status
260
+ 2024-04-26 16:49:22,059 DEBUG SenderThread:307541 [sender.py:send_request():406] send_request: stop_status
261
+ 2024-04-26 16:49:22,061 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: internal_messages
262
+ 2024-04-26 16:49:22,560 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
263
+ 2024-04-26 16:49:24,563 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
264
+ 2024-04-26 16:49:24,867 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: status_report
265
+ 2024-04-26 16:49:26,565 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
266
+ 2024-04-26 16:49:28,566 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
267
+ 2024-04-26 16:49:29,924 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: status_report
268
+ 2024-04-26 16:49:30,568 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
269
+ 2024-04-26 16:49:32,570 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
270
+ 2024-04-26 16:49:34,572 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
271
+ 2024-04-26 16:49:35,006 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: status_report
272
+ 2024-04-26 16:49:35,325 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: partial_history
273
+ 2024-04-26 16:49:35,327 DEBUG SenderThread:307541 [sender.py:send():379] send: history
274
+ 2024-04-26 16:49:35,327 DEBUG SenderThread:307541 [sender.py:send_request():406] send_request: summary_record
275
+ 2024-04-26 16:49:35,329 INFO SenderThread:307541 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
276
+ 2024-04-26 16:49:35,574 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/wandb-summary.json
277
+ 2024-04-26 16:49:36,575 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
278
+ 2024-04-26 16:49:37,059 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: stop_status
279
+ 2024-04-26 16:49:37,059 DEBUG SenderThread:307541 [sender.py:send_request():406] send_request: stop_status
280
+ 2024-04-26 16:49:37,060 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: internal_messages
281
+ 2024-04-26 16:49:38,577 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
282
+ 2024-04-26 16:49:40,048 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: status_report
283
+ 2024-04-26 16:49:40,578 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
284
+ 2024-04-26 16:49:42,580 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
285
+ 2024-04-26 16:49:44,582 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
286
+ 2024-04-26 16:49:45,171 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: status_report
287
+ 2024-04-26 16:49:46,584 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
288
+ 2024-04-26 16:49:47,496 DEBUG SenderThread:307541 [sender.py:send():379] send: stats
289
+ 2024-04-26 16:49:48,586 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
290
+ 2024-04-26 16:49:50,412 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: status_report
291
+ 2024-04-26 16:49:50,588 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
292
+ 2024-04-26 16:49:52,059 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: stop_status
293
+ 2024-04-26 16:49:52,059 DEBUG SenderThread:307541 [sender.py:send_request():406] send_request: stop_status
294
+ 2024-04-26 16:49:52,061 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: internal_messages
295
+ 2024-04-26 16:49:52,590 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
296
+ 2024-04-26 16:49:54,592 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
297
+ 2024-04-26 16:49:55,562 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: status_report
298
+ 2024-04-26 16:49:56,594 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
299
+ 2024-04-26 16:49:58,595 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
300
+ 2024-04-26 16:50:00,597 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
301
+ 2024-04-26 16:50:00,720 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: status_report
302
+ 2024-04-26 16:50:02,599 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
303
+ 2024-04-26 16:50:04,601 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
304
+ 2024-04-26 16:50:06,603 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
305
+ 2024-04-26 16:50:06,663 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: status_report
306
+ 2024-04-26 16:50:07,032 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: partial_history
307
+ 2024-04-26 16:50:07,034 DEBUG SenderThread:307541 [sender.py:send():379] send: history
308
+ 2024-04-26 16:50:07,035 DEBUG SenderThread:307541 [sender.py:send_request():406] send_request: summary_record
309
+ 2024-04-26 16:50:07,037 INFO SenderThread:307541 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
310
+ 2024-04-26 16:50:07,059 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: stop_status
311
+ 2024-04-26 16:50:07,059 DEBUG SenderThread:307541 [sender.py:send_request():406] send_request: stop_status
312
+ 2024-04-26 16:50:07,061 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: internal_messages
313
+ 2024-04-26 16:50:07,604 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/wandb-summary.json
314
+ 2024-04-26 16:50:08,605 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
315
+ 2024-04-26 16:50:10,607 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
316
+ 2024-04-26 16:50:11,799 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: status_report
317
+ 2024-04-26 16:50:12,609 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
318
+ 2024-04-26 16:50:14,611 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
319
+ 2024-04-26 16:50:16,613 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
320
+ 2024-04-26 16:50:16,896 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: status_report
321
+ 2024-04-26 16:50:17,498 DEBUG SenderThread:307541 [sender.py:send():379] send: stats
322
+ 2024-04-26 16:50:18,615 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
323
+ 2024-04-26 16:50:20,617 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
324
+ 2024-04-26 16:50:21,993 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: status_report
325
+ 2024-04-26 16:50:22,059 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: stop_status
326
+ 2024-04-26 16:50:22,059 DEBUG SenderThread:307541 [sender.py:send_request():406] send_request: stop_status
327
+ 2024-04-26 16:50:22,061 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: internal_messages
328
+ 2024-04-26 16:50:22,619 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
329
+ 2024-04-26 16:50:24,621 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
330
+ 2024-04-26 16:50:26,622 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
331
+ 2024-04-26 16:50:27,117 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: status_report
332
+ 2024-04-26 16:50:28,624 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
333
+ 2024-04-26 16:50:30,626 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
334
+ 2024-04-26 16:50:32,252 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: status_report
335
+ 2024-04-26 16:50:32,628 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
336
+ 2024-04-26 16:50:34,630 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
337
+ 2024-04-26 16:50:36,632 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
338
+ 2024-04-26 16:50:37,059 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: stop_status
339
+ 2024-04-26 16:50:37,060 DEBUG SenderThread:307541 [sender.py:send_request():406] send_request: stop_status
340
+ 2024-04-26 16:50:37,061 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: internal_messages
341
+ 2024-04-26 16:50:37,937 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: status_report
342
+ 2024-04-26 16:50:38,633 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
343
+ 2024-04-26 16:50:38,930 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: partial_history
344
+ 2024-04-26 16:50:38,932 DEBUG SenderThread:307541 [sender.py:send():379] send: history
345
+ 2024-04-26 16:50:38,933 DEBUG SenderThread:307541 [sender.py:send_request():406] send_request: summary_record
346
+ 2024-04-26 16:50:38,934 INFO SenderThread:307541 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
347
+ 2024-04-26 16:50:39,635 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/wandb-summary.json
348
+ 2024-04-26 16:50:40,636 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
349
+ 2024-04-26 16:50:42,638 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
350
+ 2024-04-26 16:50:43,758 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: status_report
351
+ 2024-04-26 16:50:44,640 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
352
+ 2024-04-26 16:50:46,642 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
353
+ 2024-04-26 16:50:47,512 DEBUG SenderThread:307541 [sender.py:send():379] send: stats
354
+ 2024-04-26 16:50:48,643 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
355
+ 2024-04-26 16:50:48,938 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: status_report
356
+ 2024-04-26 16:50:50,645 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
357
+ 2024-04-26 16:50:52,059 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: stop_status
358
+ 2024-04-26 16:50:52,060 DEBUG SenderThread:307541 [sender.py:send_request():406] send_request: stop_status
359
+ 2024-04-26 16:50:52,061 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: internal_messages
360
+ 2024-04-26 16:50:52,647 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
361
+ 2024-04-26 16:50:54,023 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: status_report
362
+ 2024-04-26 16:50:54,649 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
363
+ 2024-04-26 16:50:56,651 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
364
+ 2024-04-26 16:50:58,653 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
365
+ 2024-04-26 16:50:59,059 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: status_report
366
+ 2024-04-26 16:51:00,654 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
367
+ 2024-04-26 16:51:02,656 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
368
+ 2024-04-26 16:51:04,062 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: status_report
369
+ 2024-04-26 16:51:04,658 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
370
+ 2024-04-26 16:51:06,660 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
371
+ 2024-04-26 16:51:07,059 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: stop_status
372
+ 2024-04-26 16:51:07,060 DEBUG SenderThread:307541 [sender.py:send_request():406] send_request: stop_status
373
+ 2024-04-26 16:51:07,061 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: internal_messages
374
+ 2024-04-26 16:51:08,661 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
375
+ 2024-04-26 16:51:09,248 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: status_report
376
+ 2024-04-26 16:51:10,663 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
377
+ 2024-04-26 16:51:10,829 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: partial_history
378
+ 2024-04-26 16:51:10,831 DEBUG SenderThread:307541 [sender.py:send():379] send: history
379
+ 2024-04-26 16:51:10,831 DEBUG SenderThread:307541 [sender.py:send_request():406] send_request: summary_record
380
+ 2024-04-26 16:51:10,833 INFO SenderThread:307541 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
381
+ 2024-04-26 16:51:11,665 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/wandb-summary.json
382
+ 2024-04-26 16:51:12,666 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
383
+ 2024-04-26 16:51:14,426 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: status_report
384
+ 2024-04-26 16:51:14,668 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
385
+ 2024-04-26 16:51:16,670 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
386
+ 2024-04-26 16:51:17,502 DEBUG SenderThread:307541 [sender.py:send():379] send: stats
387
+ 2024-04-26 16:51:18,671 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
388
+ 2024-04-26 16:51:19,100 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: partial_history
389
+ 2024-04-26 16:51:19,102 DEBUG SenderThread:307541 [sender.py:send():379] send: history
390
+ 2024-04-26 16:51:19,102 DEBUG SenderThread:307541 [sender.py:send_request():406] send_request: summary_record
391
+ 2024-04-26 16:51:19,105 INFO SenderThread:307541 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
392
+ 2024-04-26 16:51:19,673 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/wandb-summary.json
393
+ 2024-04-26 16:51:20,113 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: status_report
394
+ 2024-04-26 16:51:20,674 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
395
+ 2024-04-26 16:51:22,060 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: stop_status
396
+ 2024-04-26 16:51:22,060 DEBUG SenderThread:307541 [sender.py:send_request():406] send_request: stop_status
397
+ 2024-04-26 16:51:22,061 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: internal_messages
398
+ 2024-04-26 16:51:25,185 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: status_report
399
+ 2024-04-26 16:51:28,682 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
400
+ 2024-04-26 16:51:31,090 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: status_report
401
+ 2024-04-26 16:51:36,091 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: status_report
402
+ 2024-04-26 16:51:37,298 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: stop_status
403
+ 2024-04-26 16:51:37,299 DEBUG SenderThread:307541 [sender.py:send_request():406] send_request: stop_status
404
+ 2024-04-26 16:51:37,578 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: internal_messages
405
+ 2024-04-26 16:51:41,466 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: status_report
wandb/run-20240426_164617-71zld9et/files/output.log CHANGED
@@ -78,3 +78,83 @@
78
  [INFO|configuration_utils.py:697] 2024-04-26 16:48:38,655 >> Configuration saved in ./checkpoint-100/generation_config.json
79
  [INFO|modeling_utils.py:2598] 2024-04-26 16:48:46,786 >> The model is bigger than the maximum size per checkpoint (5GB) and is going to be split in 2 checkpoint shards. You can find where each parameters has been saved in the index located at ./checkpoint-100/model.safetensors.index.json.
80
  [INFO|tokenization_utils_base.py:2488] 2024-04-26 16:48:46,789 >> tokenizer config file saved in ./checkpoint-100/tokenizer_config.json
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
78
  [INFO|configuration_utils.py:697] 2024-04-26 16:48:38,655 >> Configuration saved in ./checkpoint-100/generation_config.json
79
  [INFO|modeling_utils.py:2598] 2024-04-26 16:48:46,786 >> The model is bigger than the maximum size per checkpoint (5GB) and is going to be split in 2 checkpoint shards. You can find where each parameters has been saved in the index located at ./checkpoint-100/model.safetensors.index.json.
80
  [INFO|tokenization_utils_base.py:2488] 2024-04-26 16:48:46,789 >> tokenizer config file saved in ./checkpoint-100/tokenizer_config.json
81
+ [INFO|tokenization_utils_base.py:2497] 2024-04-26 16:48:46,791 >> Special tokens file saved in ./checkpoint-100/special_tokens_map.json
82
+ [INFO|tokenization_utils_base.py:2488] 2024-04-26 16:49:03,914 >> tokenizer config file saved in ./tokenizer_config.json
83
+ [INFO|tokenization_utils_base.py:2497] 2024-04-26 16:49:03,915 >> Special tokens file saved in ./special_tokens_map.json
84
+
85
+
86
+
87
+
88
+
89
+
90
+
91
+
92
+
93
+
94
+
95
+
96
+
97
+
98
+ 26%|██▌ | 123/478 [03:10<07:30, 1.27s/it]
99
+
100
+
101
+
102
+
103
+
104
+
105
+
106
+
107
+
108
+
109
+
110
+
111
+
112
+
113
+
114
+
115
+ 31%|███ | 149/478 [03:43<06:51, 1.25s/it]
116
+
117
+
118
+
119
+
120
+
121
+
122
+
123
+
124
+
125
+
126
+
127
+
128
+
129
+
130
+
131
+
132
+ 36%|███▌ | 173/478 [04:14<06:26, 1.27s/it]
133
+
134
+
135
+
136
+
137
+
138
+
139
+
140
+
141
+
142
+
143
+
144
+
145
+
146
+
147
+
148
+
149
+ 42%|████▏ | 199/478 [04:47<05:55, 1.27s/it]
150
+ 42%|████▏ | 200/478 [04:48<05:53, 1.27s/it][INFO|trainer.py:3614] 2024-04-26 16:51:10,829 >> ***** Running Evaluation *****
151
+ [INFO|trainer.py:3616] 2024-04-26 16:51:10,830 >> Num examples = 2000
152
+ [INFO|trainer.py:3619] 2024-04-26 16:51:10,830 >> Batch size = 8
153
+
154
+
155
+
156
+ 84%|████████▍ | 27/32 [00:06<00:01, 3.92it/s]
157
+ [INFO|configuration_utils.py:471] 2024-04-26 16:51:19,111 >> Configuration saved in ./checkpoint-200/config.json
158
+ [INFO|configuration_utils.py:697] 2024-04-26 16:51:19,113 >> Configuration saved in ./checkpoint-200/generation_config.json
159
+ [INFO|modeling_utils.py:2598] 2024-04-26 16:51:27,046 >> The model is bigger than the maximum size per checkpoint (5GB) and is going to be split in 2 checkpoint shards. You can find where each parameters has been saved in the index located at ./checkpoint-200/model.safetensors.index.json.
160
+ [INFO|tokenization_utils_base.py:2488] 2024-04-26 16:51:27,048 >> tokenizer config file saved in ./checkpoint-200/tokenizer_config.json
wandb/run-20240426_164617-71zld9et/files/wandb-summary.json CHANGED
@@ -1 +1 @@
1
- {"train/loss": 0.6832, "train/grad_norm": 5.553352355957031, "train/learning_rate": 4.821741763807186e-07, "train/rewards/chosen": -0.023883424699306488, "train/rewards/rejected": -0.04398660734295845, "train/rewards/accuracies": 0.6175000071525574, "train/rewards/margins": 0.020103182643651962, "train/logps/rejected": -390.61370849609375, "train/logps/chosen": -430.1453857421875, "train/logits/rejected": -3.442944049835205, "train/logits/chosen": -3.469045400619507, "train/epoch": 0.20920502092050208, "train/global_step": 100, "_timestamp": 1714150118.6419067, "_runtime": 141.53320169448853, "_step": 5, "eval/loss": 0.6805875301361084, "eval/runtime": 8.2626, "eval/samples_per_second": 242.055, "eval/steps_per_second": 3.873, "eval/rewards/chosen": -0.03507506474852562, "eval/rewards/rejected": -0.06189027801156044, "eval/rewards/accuracies": 0.63671875, "eval/rewards/margins": 0.02681521140038967, "eval/logps/rejected": -402.3321533203125, "eval/logps/chosen": -424.9571838378906, "eval/logits/rejected": -3.494518756866455, "eval/logits/chosen": -3.529766798019409}
 
1
+ {"train/loss": 0.6599, "train/grad_norm": 5.549075126647949, "train/learning_rate": 3.610497133404795e-07, "train/rewards/chosen": -0.16135534644126892, "train/rewards/rejected": -0.24477005004882812, "train/rewards/accuracies": 0.637499988079071, "train/rewards/margins": 0.0834147036075592, "train/logps/rejected": -399.30889892578125, "train/logps/chosen": -437.3006591796875, "train/logits/rejected": -3.4351885318756104, "train/logits/chosen": -3.489590644836426, "train/epoch": 0.41841004184100417, "train/global_step": 200, "_timestamp": 1714150279.0996985, "_runtime": 301.99099349975586, "_step": 10, "eval/loss": 0.6574448943138123, "eval/runtime": 8.258, "eval/samples_per_second": 242.19, "eval/steps_per_second": 3.875, "eval/rewards/chosen": -0.1690133512020111, "eval/rewards/rejected": -0.25762197375297546, "eval/rewards/accuracies": 0.65234375, "eval/rewards/margins": 0.08860861510038376, "eval/logps/rejected": -421.90533447265625, "eval/logps/chosen": -438.3510437011719, "eval/logits/rejected": -3.49556827545166, "eval/logits/chosen": -3.5279171466827393}
wandb/run-20240426_164617-71zld9et/logs/debug-internal.log CHANGED
@@ -241,3 +241,165 @@
241
  2024-04-26 16:48:54,535 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/config.yaml
242
  2024-04-26 16:48:58,655 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: status_report
243
  2024-04-26 16:49:03,655 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: status_report
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
241
  2024-04-26 16:48:54,535 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/config.yaml
242
  2024-04-26 16:48:58,655 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: status_report
243
  2024-04-26 16:49:03,655 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: status_report
244
+ 2024-04-26 16:49:06,546 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
245
+ 2024-04-26 16:49:07,059 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: stop_status
246
+ 2024-04-26 16:49:07,059 DEBUG SenderThread:307541 [sender.py:send_request():406] send_request: stop_status
247
+ 2024-04-26 16:49:07,059 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: internal_messages
248
+ 2024-04-26 16:49:08,548 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
249
+ 2024-04-26 16:49:08,661 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: status_report
250
+ 2024-04-26 16:49:10,549 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
251
+ 2024-04-26 16:49:12,551 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
252
+ 2024-04-26 16:49:13,870 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: status_report
253
+ 2024-04-26 16:49:14,553 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
254
+ 2024-04-26 16:49:16,555 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
255
+ 2024-04-26 16:49:17,536 DEBUG SenderThread:307541 [sender.py:send():379] send: stats
256
+ 2024-04-26 16:49:18,557 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
257
+ 2024-04-26 16:49:18,920 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: status_report
258
+ 2024-04-26 16:49:20,559 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
259
+ 2024-04-26 16:49:22,059 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: stop_status
260
+ 2024-04-26 16:49:22,059 DEBUG SenderThread:307541 [sender.py:send_request():406] send_request: stop_status
261
+ 2024-04-26 16:49:22,061 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: internal_messages
262
+ 2024-04-26 16:49:22,560 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
263
+ 2024-04-26 16:49:24,563 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
264
+ 2024-04-26 16:49:24,867 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: status_report
265
+ 2024-04-26 16:49:26,565 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
266
+ 2024-04-26 16:49:28,566 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
267
+ 2024-04-26 16:49:29,924 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: status_report
268
+ 2024-04-26 16:49:30,568 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
269
+ 2024-04-26 16:49:32,570 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
270
+ 2024-04-26 16:49:34,572 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
271
+ 2024-04-26 16:49:35,006 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: status_report
272
+ 2024-04-26 16:49:35,325 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: partial_history
273
+ 2024-04-26 16:49:35,327 DEBUG SenderThread:307541 [sender.py:send():379] send: history
274
+ 2024-04-26 16:49:35,327 DEBUG SenderThread:307541 [sender.py:send_request():406] send_request: summary_record
275
+ 2024-04-26 16:49:35,329 INFO SenderThread:307541 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
276
+ 2024-04-26 16:49:35,574 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/wandb-summary.json
277
+ 2024-04-26 16:49:36,575 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
278
+ 2024-04-26 16:49:37,059 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: stop_status
279
+ 2024-04-26 16:49:37,059 DEBUG SenderThread:307541 [sender.py:send_request():406] send_request: stop_status
280
+ 2024-04-26 16:49:37,060 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: internal_messages
281
+ 2024-04-26 16:49:38,577 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
282
+ 2024-04-26 16:49:40,048 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: status_report
283
+ 2024-04-26 16:49:40,578 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
284
+ 2024-04-26 16:49:42,580 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
285
+ 2024-04-26 16:49:44,582 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
286
+ 2024-04-26 16:49:45,171 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: status_report
287
+ 2024-04-26 16:49:46,584 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
288
+ 2024-04-26 16:49:47,496 DEBUG SenderThread:307541 [sender.py:send():379] send: stats
289
+ 2024-04-26 16:49:48,586 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
290
+ 2024-04-26 16:49:50,412 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: status_report
291
+ 2024-04-26 16:49:50,588 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
292
+ 2024-04-26 16:49:52,059 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: stop_status
293
+ 2024-04-26 16:49:52,059 DEBUG SenderThread:307541 [sender.py:send_request():406] send_request: stop_status
294
+ 2024-04-26 16:49:52,061 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: internal_messages
295
+ 2024-04-26 16:49:52,590 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
296
+ 2024-04-26 16:49:54,592 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
297
+ 2024-04-26 16:49:55,562 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: status_report
298
+ 2024-04-26 16:49:56,594 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
299
+ 2024-04-26 16:49:58,595 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
300
+ 2024-04-26 16:50:00,597 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
301
+ 2024-04-26 16:50:00,720 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: status_report
302
+ 2024-04-26 16:50:02,599 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
303
+ 2024-04-26 16:50:04,601 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
304
+ 2024-04-26 16:50:06,603 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
305
+ 2024-04-26 16:50:06,663 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: status_report
306
+ 2024-04-26 16:50:07,032 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: partial_history
307
+ 2024-04-26 16:50:07,034 DEBUG SenderThread:307541 [sender.py:send():379] send: history
308
+ 2024-04-26 16:50:07,035 DEBUG SenderThread:307541 [sender.py:send_request():406] send_request: summary_record
309
+ 2024-04-26 16:50:07,037 INFO SenderThread:307541 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
310
+ 2024-04-26 16:50:07,059 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: stop_status
311
+ 2024-04-26 16:50:07,059 DEBUG SenderThread:307541 [sender.py:send_request():406] send_request: stop_status
312
+ 2024-04-26 16:50:07,061 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: internal_messages
313
+ 2024-04-26 16:50:07,604 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/wandb-summary.json
314
+ 2024-04-26 16:50:08,605 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
315
+ 2024-04-26 16:50:10,607 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
316
+ 2024-04-26 16:50:11,799 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: status_report
317
+ 2024-04-26 16:50:12,609 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
318
+ 2024-04-26 16:50:14,611 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
319
+ 2024-04-26 16:50:16,613 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
320
+ 2024-04-26 16:50:16,896 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: status_report
321
+ 2024-04-26 16:50:17,498 DEBUG SenderThread:307541 [sender.py:send():379] send: stats
322
+ 2024-04-26 16:50:18,615 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
323
+ 2024-04-26 16:50:20,617 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
324
+ 2024-04-26 16:50:21,993 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: status_report
325
+ 2024-04-26 16:50:22,059 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: stop_status
326
+ 2024-04-26 16:50:22,059 DEBUG SenderThread:307541 [sender.py:send_request():406] send_request: stop_status
327
+ 2024-04-26 16:50:22,061 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: internal_messages
328
+ 2024-04-26 16:50:22,619 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
329
+ 2024-04-26 16:50:24,621 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
330
+ 2024-04-26 16:50:26,622 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
331
+ 2024-04-26 16:50:27,117 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: status_report
332
+ 2024-04-26 16:50:28,624 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
333
+ 2024-04-26 16:50:30,626 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
334
+ 2024-04-26 16:50:32,252 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: status_report
335
+ 2024-04-26 16:50:32,628 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
336
+ 2024-04-26 16:50:34,630 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
337
+ 2024-04-26 16:50:36,632 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
338
+ 2024-04-26 16:50:37,059 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: stop_status
339
+ 2024-04-26 16:50:37,060 DEBUG SenderThread:307541 [sender.py:send_request():406] send_request: stop_status
340
+ 2024-04-26 16:50:37,061 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: internal_messages
341
+ 2024-04-26 16:50:37,937 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: status_report
342
+ 2024-04-26 16:50:38,633 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
343
+ 2024-04-26 16:50:38,930 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: partial_history
344
+ 2024-04-26 16:50:38,932 DEBUG SenderThread:307541 [sender.py:send():379] send: history
345
+ 2024-04-26 16:50:38,933 DEBUG SenderThread:307541 [sender.py:send_request():406] send_request: summary_record
346
+ 2024-04-26 16:50:38,934 INFO SenderThread:307541 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
347
+ 2024-04-26 16:50:39,635 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/wandb-summary.json
348
+ 2024-04-26 16:50:40,636 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
349
+ 2024-04-26 16:50:42,638 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
350
+ 2024-04-26 16:50:43,758 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: status_report
351
+ 2024-04-26 16:50:44,640 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
352
+ 2024-04-26 16:50:46,642 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
353
+ 2024-04-26 16:50:47,512 DEBUG SenderThread:307541 [sender.py:send():379] send: stats
354
+ 2024-04-26 16:50:48,643 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
355
+ 2024-04-26 16:50:48,938 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: status_report
356
+ 2024-04-26 16:50:50,645 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
357
+ 2024-04-26 16:50:52,059 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: stop_status
358
+ 2024-04-26 16:50:52,060 DEBUG SenderThread:307541 [sender.py:send_request():406] send_request: stop_status
359
+ 2024-04-26 16:50:52,061 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: internal_messages
360
+ 2024-04-26 16:50:52,647 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
361
+ 2024-04-26 16:50:54,023 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: status_report
362
+ 2024-04-26 16:50:54,649 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
363
+ 2024-04-26 16:50:56,651 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
364
+ 2024-04-26 16:50:58,653 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
365
+ 2024-04-26 16:50:59,059 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: status_report
366
+ 2024-04-26 16:51:00,654 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
367
+ 2024-04-26 16:51:02,656 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
368
+ 2024-04-26 16:51:04,062 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: status_report
369
+ 2024-04-26 16:51:04,658 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
370
+ 2024-04-26 16:51:06,660 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
371
+ 2024-04-26 16:51:07,059 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: stop_status
372
+ 2024-04-26 16:51:07,060 DEBUG SenderThread:307541 [sender.py:send_request():406] send_request: stop_status
373
+ 2024-04-26 16:51:07,061 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: internal_messages
374
+ 2024-04-26 16:51:08,661 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
375
+ 2024-04-26 16:51:09,248 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: status_report
376
+ 2024-04-26 16:51:10,663 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
377
+ 2024-04-26 16:51:10,829 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: partial_history
378
+ 2024-04-26 16:51:10,831 DEBUG SenderThread:307541 [sender.py:send():379] send: history
379
+ 2024-04-26 16:51:10,831 DEBUG SenderThread:307541 [sender.py:send_request():406] send_request: summary_record
380
+ 2024-04-26 16:51:10,833 INFO SenderThread:307541 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
381
+ 2024-04-26 16:51:11,665 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/wandb-summary.json
382
+ 2024-04-26 16:51:12,666 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
383
+ 2024-04-26 16:51:14,426 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: status_report
384
+ 2024-04-26 16:51:14,668 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
385
+ 2024-04-26 16:51:16,670 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
386
+ 2024-04-26 16:51:17,502 DEBUG SenderThread:307541 [sender.py:send():379] send: stats
387
+ 2024-04-26 16:51:18,671 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
388
+ 2024-04-26 16:51:19,100 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: partial_history
389
+ 2024-04-26 16:51:19,102 DEBUG SenderThread:307541 [sender.py:send():379] send: history
390
+ 2024-04-26 16:51:19,102 DEBUG SenderThread:307541 [sender.py:send_request():406] send_request: summary_record
391
+ 2024-04-26 16:51:19,105 INFO SenderThread:307541 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
392
+ 2024-04-26 16:51:19,673 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/wandb-summary.json
393
+ 2024-04-26 16:51:20,113 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: status_report
394
+ 2024-04-26 16:51:20,674 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
395
+ 2024-04-26 16:51:22,060 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: stop_status
396
+ 2024-04-26 16:51:22,060 DEBUG SenderThread:307541 [sender.py:send_request():406] send_request: stop_status
397
+ 2024-04-26 16:51:22,061 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: internal_messages
398
+ 2024-04-26 16:51:25,185 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: status_report
399
+ 2024-04-26 16:51:28,682 INFO Thread-12 :307541 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-dpo-ultrafeedback-200k/wandb/run-20240426_164617-71zld9et/files/output.log
400
+ 2024-04-26 16:51:31,090 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: status_report
401
+ 2024-04-26 16:51:36,091 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: status_report
402
+ 2024-04-26 16:51:37,298 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: stop_status
403
+ 2024-04-26 16:51:37,299 DEBUG SenderThread:307541 [sender.py:send_request():406] send_request: stop_status
404
+ 2024-04-26 16:51:37,578 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: internal_messages
405
+ 2024-04-26 16:51:41,466 DEBUG HandlerThread:307541 [handler.py:handle_request():146] handle_request: status_report
wandb/run-20240426_164617-71zld9et/run-71zld9et.wandb CHANGED
Binary files a/wandb/run-20240426_164617-71zld9et/run-71zld9et.wandb and b/wandb/run-20240426_164617-71zld9et/run-71zld9et.wandb differ