ahcene-ikram commited on
Commit
30fd732
1 Parent(s): 1be292e

Training in progress, step 1000

Browse files
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:8170ab916444959863e37609f0bb883afba57873f7c4c32d5180443e81e6347d
3
  size 485755120
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a0e04316e9f4d4dbb2fdd70789bbd560cb21829d1ae7e9ac2abe9ab4d16f5b83
3
  size 485755120
runs/Jun02_21-20-24_cfb55ca92bb5/events.out.tfevents.1717363382.cfb55ca92bb5.34.0 CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:25c50f283266b11b5c719ac8cfbc49194b893f4b86b908eedc48d71921aa53e8
3
- size 242959
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d5e0644b8330cf40f914aecf42615aa319c14954a0f5b21cbf82f72c6d388d34
3
+ size 243170
wandb/debug-internal.log CHANGED
@@ -982,3 +982,53 @@ subprocess.TimeoutExpired: Command '['conda', 'env', 'export']' timed out after
982
  2024-06-02 21:58:45,611 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: stop_status
983
  2024-06-02 21:58:45,612 DEBUG SenderThread:82 [sender.py:send_request():406] send_request: stop_status
984
  2024-06-02 21:58:45,685 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
982
  2024-06-02 21:58:45,611 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: stop_status
983
  2024-06-02 21:58:45,612 DEBUG SenderThread:82 [sender.py:send_request():406] send_request: stop_status
984
  2024-06-02 21:58:45,685 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
985
+ 2024-06-02 21:58:50,686 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
986
+ 2024-06-02 21:58:55,687 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
987
+ 2024-06-02 21:58:59,918 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: internal_messages
988
+ 2024-06-02 21:59:00,606 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: stop_status
989
+ 2024-06-02 21:59:00,607 DEBUG SenderThread:82 [sender.py:send_request():406] send_request: stop_status
990
+ 2024-06-02 21:59:00,733 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
991
+ 2024-06-02 21:59:05,734 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
992
+ 2024-06-02 21:59:10,735 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
993
+ 2024-06-02 21:59:13,593 DEBUG SenderThread:82 [sender.py:send():379] send: stats
994
+ 2024-06-02 21:59:14,918 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: internal_messages
995
+ 2024-06-02 21:59:15,606 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: stop_status
996
+ 2024-06-02 21:59:15,607 DEBUG SenderThread:82 [sender.py:send_request():406] send_request: stop_status
997
+ 2024-06-02 21:59:16,675 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
998
+ 2024-06-02 21:59:21,676 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
999
+ 2024-06-02 21:59:26,676 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
1000
+ 2024-06-02 21:59:29,920 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: internal_messages
1001
+ 2024-06-02 21:59:30,607 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: stop_status
1002
+ 2024-06-02 21:59:30,607 DEBUG SenderThread:82 [sender.py:send_request():406] send_request: stop_status
1003
+ 2024-06-02 21:59:31,682 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
1004
+ 2024-06-02 21:59:36,683 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
1005
+ 2024-06-02 21:59:41,684 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
1006
+ 2024-06-02 21:59:43,594 DEBUG SenderThread:82 [sender.py:send():379] send: stats
1007
+ 2024-06-02 21:59:44,921 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: internal_messages
1008
+ 2024-06-02 21:59:45,607 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: stop_status
1009
+ 2024-06-02 21:59:45,607 DEBUG SenderThread:82 [sender.py:send_request():406] send_request: stop_status
1010
+ 2024-06-02 21:59:47,639 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
1011
+ 2024-06-02 21:59:52,640 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
1012
+ 2024-06-02 21:59:57,641 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
1013
+ 2024-06-02 21:59:59,921 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: internal_messages
1014
+ 2024-06-02 22:00:00,607 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: stop_status
1015
+ 2024-06-02 22:00:00,607 DEBUG SenderThread:82 [sender.py:send_request():406] send_request: stop_status
1016
+ 2024-06-02 22:00:03,633 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
1017
+ 2024-06-02 22:00:08,634 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
1018
+ 2024-06-02 22:00:13,595 DEBUG SenderThread:82 [sender.py:send():379] send: stats
1019
+ 2024-06-02 22:00:14,596 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
1020
+ 2024-06-02 22:00:14,705 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: partial_history
1021
+ 2024-06-02 22:00:14,706 DEBUG SenderThread:82 [sender.py:send():379] send: history
1022
+ 2024-06-02 22:00:14,706 DEBUG SenderThread:82 [sender.py:send_request():406] send_request: summary_record
1023
+ 2024-06-02 22:00:14,706 INFO SenderThread:82 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
1024
+ 2024-06-02 22:00:14,921 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: internal_messages
1025
+ 2024-06-02 22:00:15,324 INFO Thread-12 :82 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240602_212342-ypv2rcj7/files/wandb-summary.json
1026
+ 2024-06-02 22:00:15,608 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: stop_status
1027
+ 2024-06-02 22:00:15,608 DEBUG SenderThread:82 [sender.py:send_request():406] send_request: stop_status
1028
+ 2024-06-02 22:00:20,134 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
1029
+ 2024-06-02 22:00:20,326 INFO Thread-12 :82 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240602_212342-ypv2rcj7/files/output.log
1030
+ 2024-06-02 22:00:25,135 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
1031
+ 2024-06-02 22:00:29,921 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: internal_messages
1032
+ 2024-06-02 22:00:30,607 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: stop_status
1033
+ 2024-06-02 22:00:30,608 DEBUG SenderThread:82 [sender.py:send_request():406] send_request: stop_status
1034
+ 2024-06-02 22:00:30,726 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
wandb/run-20240602_212342-ypv2rcj7/files/output.log CHANGED
@@ -38,4 +38,6 @@
38
  warnings.warn('Was asked to gather along dimension 0, but all '
39
  /opt/conda/lib/python3.10/site-packages/torch/nn/parallel/_functions.py:68: UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector.
40
  warnings.warn('Was asked to gather along dimension 0, but all '
 
 
41
  /opt/conda/lib/python3.10/site-packages/torch/nn/parallel/_functions.py:68: UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector.
 
38
  warnings.warn('Was asked to gather along dimension 0, but all '
39
  /opt/conda/lib/python3.10/site-packages/torch/nn/parallel/_functions.py:68: UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector.
40
  warnings.warn('Was asked to gather along dimension 0, but all '
41
+ /opt/conda/lib/python3.10/site-packages/torch/nn/parallel/_functions.py:68: UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector.
42
+ warnings.warn('Was asked to gather along dimension 0, but all '
43
  /opt/conda/lib/python3.10/site-packages/torch/nn/parallel/_functions.py:68: UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector.
wandb/run-20240602_212342-ypv2rcj7/files/wandb-summary.json CHANGED
@@ -1 +1 @@
1
- {"train/loss": 472.2275, "train/grad_norm": 5.150758266448975, "train/learning_rate": 3.5397196261682246e-05, "train/epoch": 0.58, "train/global_step": 500, "_timestamp": 1717364544.5038507, "_runtime": 1121.5761528015137, "_step": 0}
 
1
+ {"train/loss": 7.3887, "train/grad_norm": 5.3752336502075195, "train/learning_rate": 2.0794392523364487e-05, "train/epoch": 1.17, "train/global_step": 1000, "_timestamp": 1717365614.7046628, "_runtime": 2191.776964902878, "_step": 1}
wandb/run-20240602_212342-ypv2rcj7/logs/debug-internal.log CHANGED
@@ -982,3 +982,53 @@ subprocess.TimeoutExpired: Command '['conda', 'env', 'export']' timed out after
982
  2024-06-02 21:58:45,611 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: stop_status
983
  2024-06-02 21:58:45,612 DEBUG SenderThread:82 [sender.py:send_request():406] send_request: stop_status
984
  2024-06-02 21:58:45,685 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
982
  2024-06-02 21:58:45,611 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: stop_status
983
  2024-06-02 21:58:45,612 DEBUG SenderThread:82 [sender.py:send_request():406] send_request: stop_status
984
  2024-06-02 21:58:45,685 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
985
+ 2024-06-02 21:58:50,686 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
986
+ 2024-06-02 21:58:55,687 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
987
+ 2024-06-02 21:58:59,918 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: internal_messages
988
+ 2024-06-02 21:59:00,606 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: stop_status
989
+ 2024-06-02 21:59:00,607 DEBUG SenderThread:82 [sender.py:send_request():406] send_request: stop_status
990
+ 2024-06-02 21:59:00,733 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
991
+ 2024-06-02 21:59:05,734 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
992
+ 2024-06-02 21:59:10,735 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
993
+ 2024-06-02 21:59:13,593 DEBUG SenderThread:82 [sender.py:send():379] send: stats
994
+ 2024-06-02 21:59:14,918 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: internal_messages
995
+ 2024-06-02 21:59:15,606 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: stop_status
996
+ 2024-06-02 21:59:15,607 DEBUG SenderThread:82 [sender.py:send_request():406] send_request: stop_status
997
+ 2024-06-02 21:59:16,675 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
998
+ 2024-06-02 21:59:21,676 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
999
+ 2024-06-02 21:59:26,676 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
1000
+ 2024-06-02 21:59:29,920 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: internal_messages
1001
+ 2024-06-02 21:59:30,607 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: stop_status
1002
+ 2024-06-02 21:59:30,607 DEBUG SenderThread:82 [sender.py:send_request():406] send_request: stop_status
1003
+ 2024-06-02 21:59:31,682 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
1004
+ 2024-06-02 21:59:36,683 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
1005
+ 2024-06-02 21:59:41,684 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
1006
+ 2024-06-02 21:59:43,594 DEBUG SenderThread:82 [sender.py:send():379] send: stats
1007
+ 2024-06-02 21:59:44,921 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: internal_messages
1008
+ 2024-06-02 21:59:45,607 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: stop_status
1009
+ 2024-06-02 21:59:45,607 DEBUG SenderThread:82 [sender.py:send_request():406] send_request: stop_status
1010
+ 2024-06-02 21:59:47,639 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
1011
+ 2024-06-02 21:59:52,640 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
1012
+ 2024-06-02 21:59:57,641 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
1013
+ 2024-06-02 21:59:59,921 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: internal_messages
1014
+ 2024-06-02 22:00:00,607 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: stop_status
1015
+ 2024-06-02 22:00:00,607 DEBUG SenderThread:82 [sender.py:send_request():406] send_request: stop_status
1016
+ 2024-06-02 22:00:03,633 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
1017
+ 2024-06-02 22:00:08,634 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
1018
+ 2024-06-02 22:00:13,595 DEBUG SenderThread:82 [sender.py:send():379] send: stats
1019
+ 2024-06-02 22:00:14,596 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
1020
+ 2024-06-02 22:00:14,705 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: partial_history
1021
+ 2024-06-02 22:00:14,706 DEBUG SenderThread:82 [sender.py:send():379] send: history
1022
+ 2024-06-02 22:00:14,706 DEBUG SenderThread:82 [sender.py:send_request():406] send_request: summary_record
1023
+ 2024-06-02 22:00:14,706 INFO SenderThread:82 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
1024
+ 2024-06-02 22:00:14,921 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: internal_messages
1025
+ 2024-06-02 22:00:15,324 INFO Thread-12 :82 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240602_212342-ypv2rcj7/files/wandb-summary.json
1026
+ 2024-06-02 22:00:15,608 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: stop_status
1027
+ 2024-06-02 22:00:15,608 DEBUG SenderThread:82 [sender.py:send_request():406] send_request: stop_status
1028
+ 2024-06-02 22:00:20,134 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
1029
+ 2024-06-02 22:00:20,326 INFO Thread-12 :82 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240602_212342-ypv2rcj7/files/output.log
1030
+ 2024-06-02 22:00:25,135 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
1031
+ 2024-06-02 22:00:29,921 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: internal_messages
1032
+ 2024-06-02 22:00:30,607 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: stop_status
1033
+ 2024-06-02 22:00:30,608 DEBUG SenderThread:82 [sender.py:send_request():406] send_request: stop_status
1034
+ 2024-06-02 22:00:30,726 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
wandb/run-20240602_212342-ypv2rcj7/run-ypv2rcj7.wandb CHANGED
Binary files a/wandb/run-20240602_212342-ypv2rcj7/run-ypv2rcj7.wandb and b/wandb/run-20240602_212342-ypv2rcj7/run-ypv2rcj7.wandb differ