Training in progress, epoch 3

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:192d8baedf293ded26712a381040dbc7cb538e7107af62587c36575808683930
+oid sha256:9a46b31286363bfff34a61f058fdeb17c4332abbcbc9258b7dad384971cb0c20
 size 596114452

runs/Mar06_15-36-27_64d026e37c1e/events.out.tfevents.1709739388.64d026e37c1e.34.1

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:93a666353e9ba72eb52a33868e0f40f4492e231e7dc468334f42615a3f5b4336
-size 6627
+oid sha256:10be525aec7a18c3ee177737e4c8a6319142cbac826c19b085a9b63679820151
+size 7297

wandb/debug-internal.log

@@ -201,3 +201,29 @@
 2024-03-06 15:37:42,791 DEBUG   HandlerThread:135 [handler.py:handle_request():146] handle_request: keepalive
 2024-03-06 15:37:46,307 DEBUG   HandlerThread:135 [handler.py:handle_request():146] handle_request: status_report
 2024-03-06 15:37:47,792 DEBUG   HandlerThread:135 [handler.py:handle_request():146] handle_request: keepalive
+2024-03-06 15:37:51,308 DEBUG   HandlerThread:135 [handler.py:handle_request():146] handle_request: status_report
+2024-03-06 15:37:52,793 DEBUG   HandlerThread:135 [handler.py:handle_request():146] handle_request: keepalive
+2024-03-06 15:37:56,309 DEBUG   HandlerThread:135 [handler.py:handle_request():146] handle_request: status_report
+2024-03-06 15:37:56,952 DEBUG   SenderThread:135 [sender.py:send():382] send: stats
+2024-03-06 15:37:57,804 DEBUG   HandlerThread:135 [handler.py:handle_request():146] handle_request: keepalive
+2024-03-06 15:38:01,953 DEBUG   HandlerThread:135 [handler.py:handle_request():146] handle_request: status_report
+2024-03-06 15:38:02,813 DEBUG   HandlerThread:135 [handler.py:handle_request():146] handle_request: keepalive
+2024-03-06 15:38:03,715 DEBUG   HandlerThread:135 [handler.py:handle_request():146] handle_request: partial_history
+2024-03-06 15:38:03,716 DEBUG   SenderThread:135 [sender.py:send():382] send: history
+2024-03-06 15:38:03,716 DEBUG   SenderThread:135 [sender.py:send_request():409] send_request: summary_record
+2024-03-06 15:38:03,716 INFO    SenderThread:135 [sender.py:_save_file():1403] saving file wandb-summary.json with policy end
+2024-03-06 15:38:04,565 INFO    Thread-12 :135 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240306_153450-l1uc7yan/files/wandb-summary.json
+2024-03-06 15:38:05,732 DEBUG   HandlerThread:135 [handler.py:handle_request():146] handle_request: partial_history
+2024-03-06 15:38:05,734 DEBUG   SenderThread:135 [sender.py:send():382] send: history
+2024-03-06 15:38:05,734 DEBUG   SenderThread:135 [sender.py:send_request():409] send_request: summary_record
+2024-03-06 15:38:05,734 INFO    SenderThread:135 [sender.py:_save_file():1403] saving file wandb-summary.json with policy end
+2024-03-06 15:38:06,566 INFO    Thread-12 :135 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240306_153450-l1uc7yan/files/wandb-summary.json
+2024-03-06 15:38:07,567 INFO    Thread-12 :135 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240306_153450-l1uc7yan/files/output.log
+2024-03-06 15:38:07,735 DEBUG   HandlerThread:135 [handler.py:handle_request():146] handle_request: status_report
+2024-03-06 15:38:08,263 DEBUG   HandlerThread:135 [handler.py:handle_request():146] handle_request: keepalive
+2024-03-06 15:38:12,736 DEBUG   HandlerThread:135 [handler.py:handle_request():146] handle_request: status_report
+2024-03-06 15:38:13,277 DEBUG   HandlerThread:135 [handler.py:handle_request():146] handle_request: keepalive
+2024-03-06 15:38:17,737 DEBUG   HandlerThread:135 [handler.py:handle_request():146] handle_request: status_report
+2024-03-06 15:38:18,278 DEBUG   HandlerThread:135 [handler.py:handle_request():146] handle_request: keepalive
+2024-03-06 15:38:22,738 DEBUG   HandlerThread:135 [handler.py:handle_request():146] handle_request: status_report
+2024-03-06 15:38:23,279 DEBUG   HandlerThread:135 [handler.py:handle_request():146] handle_request: keepalive

wandb/run-20240306_153450-l1uc7yan/files/output.log

@@ -1,5 +1,7 @@
 Some weights of BertForSequenceClassification were not initialized from the model checkpoint at lanwuwei/BERTOverflow_stackoverflow_github and are newly initialized: ['classifier.bias', 'classifier.weight']
 You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
+/opt/conda/lib/python3.10/site-packages/sklearn/metrics/_classification.py:1344: UndefinedMetricWarning: Precision is ill-defined and being set to 0.0 in labels with no predicted samples. Use `zero_division` parameter to control this behavior.
+  _warn_prf(average, modifier, msg_start, len(result))
 /opt/conda/lib/python3.10/site-packages/sklearn/metrics/_classification.py:1344: UndefinedMetricWarning: Precision is ill-defined and being set to 0.0 in labels with no predicted samples. Use `zero_division` parameter to control this behavior.
   _warn_prf(average, modifier, msg_start, len(result))
 /opt/conda/lib/python3.10/site-packages/sklearn/metrics/_classification.py:1344: UndefinedMetricWarning: Precision is ill-defined and being set to 0.0 in labels with no predicted samples. Use `zero_division` parameter to control this behavior.

wandb/run-20240306_153450-l1uc7yan/files/wandb-summary.json

@@ -1 +1 @@
-{"train/loss": 1.8114, "train/grad_norm": 4.968899726867676, "train/learning_rate": 1.2e-05, "train/epoch": 2.0, "train/global_step": 62, "_timestamp": 1709739450.3015118, "_runtime": 159.38722586631775, "_step": 4, "eval/loss": 1.769439935684204, "eval/accuracy": 0.34545454545454546, "eval/precision": 0.11933884297520661, "eval/recall": 0.34545454545454546, "eval/f1": 0.17739557739557738, "eval/runtime": 2.0329, "eval/samples_per_second": 27.055, "eval/steps_per_second": 1.968}
+{"train/loss": 1.7988, "train/grad_norm": 3.493379592895508, "train/learning_rate": 8.000000000000001e-06, "train/epoch": 3.0, "train/global_step": 93, "_timestamp": 1709739485.7324972, "_runtime": 194.81821131706238, "_step": 6, "eval/loss": 1.7901660203933716, "eval/accuracy": 0.34545454545454546, "eval/precision": 0.11933884297520661, "eval/recall": 0.34545454545454546, "eval/f1": 0.17739557739557738, "eval/runtime": 2.0141, "eval/samples_per_second": 27.307, "eval/steps_per_second": 1.986}

wandb/run-20240306_153450-l1uc7yan/logs/debug-internal.log

@@ -201,3 +201,29 @@
 2024-03-06 15:37:42,791 DEBUG   HandlerThread:135 [handler.py:handle_request():146] handle_request: keepalive
 2024-03-06 15:37:46,307 DEBUG   HandlerThread:135 [handler.py:handle_request():146] handle_request: status_report
 2024-03-06 15:37:47,792 DEBUG   HandlerThread:135 [handler.py:handle_request():146] handle_request: keepalive
+2024-03-06 15:37:51,308 DEBUG   HandlerThread:135 [handler.py:handle_request():146] handle_request: status_report
+2024-03-06 15:37:52,793 DEBUG   HandlerThread:135 [handler.py:handle_request():146] handle_request: keepalive
+2024-03-06 15:37:56,309 DEBUG   HandlerThread:135 [handler.py:handle_request():146] handle_request: status_report
+2024-03-06 15:37:56,952 DEBUG   SenderThread:135 [sender.py:send():382] send: stats
+2024-03-06 15:37:57,804 DEBUG   HandlerThread:135 [handler.py:handle_request():146] handle_request: keepalive
+2024-03-06 15:38:01,953 DEBUG   HandlerThread:135 [handler.py:handle_request():146] handle_request: status_report
+2024-03-06 15:38:02,813 DEBUG   HandlerThread:135 [handler.py:handle_request():146] handle_request: keepalive
+2024-03-06 15:38:03,715 DEBUG   HandlerThread:135 [handler.py:handle_request():146] handle_request: partial_history
+2024-03-06 15:38:03,716 DEBUG   SenderThread:135 [sender.py:send():382] send: history
+2024-03-06 15:38:03,716 DEBUG   SenderThread:135 [sender.py:send_request():409] send_request: summary_record
+2024-03-06 15:38:03,716 INFO    SenderThread:135 [sender.py:_save_file():1403] saving file wandb-summary.json with policy end
+2024-03-06 15:38:04,565 INFO    Thread-12 :135 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240306_153450-l1uc7yan/files/wandb-summary.json
+2024-03-06 15:38:05,732 DEBUG   HandlerThread:135 [handler.py:handle_request():146] handle_request: partial_history
+2024-03-06 15:38:05,734 DEBUG   SenderThread:135 [sender.py:send():382] send: history
+2024-03-06 15:38:05,734 DEBUG   SenderThread:135 [sender.py:send_request():409] send_request: summary_record
+2024-03-06 15:38:05,734 INFO    SenderThread:135 [sender.py:_save_file():1403] saving file wandb-summary.json with policy end
+2024-03-06 15:38:06,566 INFO    Thread-12 :135 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240306_153450-l1uc7yan/files/wandb-summary.json
+2024-03-06 15:38:07,567 INFO    Thread-12 :135 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240306_153450-l1uc7yan/files/output.log
+2024-03-06 15:38:07,735 DEBUG   HandlerThread:135 [handler.py:handle_request():146] handle_request: status_report
+2024-03-06 15:38:08,263 DEBUG   HandlerThread:135 [handler.py:handle_request():146] handle_request: keepalive
+2024-03-06 15:38:12,736 DEBUG   HandlerThread:135 [handler.py:handle_request():146] handle_request: status_report
+2024-03-06 15:38:13,277 DEBUG   HandlerThread:135 [handler.py:handle_request():146] handle_request: keepalive
+2024-03-06 15:38:17,737 DEBUG   HandlerThread:135 [handler.py:handle_request():146] handle_request: status_report
+2024-03-06 15:38:18,278 DEBUG   HandlerThread:135 [handler.py:handle_request():146] handle_request: keepalive
+2024-03-06 15:38:22,738 DEBUG   HandlerThread:135 [handler.py:handle_request():146] handle_request: status_report
+2024-03-06 15:38:23,279 DEBUG   HandlerThread:135 [handler.py:handle_request():146] handle_request: keepalive