kertob commited on
Commit
9ffaa91
1 Parent(s): fc44d77

Training in progress, step 10

Browse files
adapter_model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:fb52f073e891001b2508663ade9a076e5dc6b56ce073582ab0cf1eff244a3bdc
3
  size 261131840
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c64d2c2ebef88764f7ab7aca8520fb32afeec469a809ed0b3699fab4eb532e83
3
  size 261131840
runs/Feb14_11-53-30_f4b889916ade/events.out.tfevents.1707911621.f4b889916ade.4105.0 CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:0cd91697083b9a3efd30baa8c7e4c7eb8022f5dfd6a90ec07f8175b3b861a5db
3
- size 8256
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:73ad222f44694a26574c7a27e96b9399d962fb69d6bf78c325203ee2ab07cd47
3
+ size 8610
runs/Feb14_12-49-31_f4b889916ade/events.out.tfevents.1707914978.f4b889916ade.4105.1 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6354ee91de6833da7bfeba8b6a123628ddc4040fdab17cba286eb70cd22c1064
3
+ size 5620
tokenizer.json CHANGED
@@ -2,7 +2,7 @@
2
  "version": "1.0",
3
  "truncation": {
4
  "direction": "Right",
5
- "max_length": 256,
6
  "strategy": "LongestFirst",
7
  "stride": 0
8
  },
 
2
  "version": "1.0",
3
  "truncation": {
4
  "direction": "Right",
5
+ "max_length": 128,
6
  "strategy": "LongestFirst",
7
  "stride": 0
8
  },
training_args.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:c5492c812c65a4e0748ed682ee465b62ea7a97959164728c6ca1bc801859beeb
3
  size 4664
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b3ad917fee89e2d2ef8743a8c2e5730c727035657cfd0953306633c6dafc920e
3
  size 4664
wandb/debug-internal.log CHANGED
@@ -879,3 +879,770 @@
879
  2024-02-14 12:22:46,049 INFO SenderThread:19614 [sender.py:_save_file():1403] saving file wandb-summary.json with policy end
880
  2024-02-14 12:22:46,293 INFO Thread-12 :19614 [dir_watcher.py:_on_file_modified():288] file/dir modified: /content/wandb/run-20240214_115405-518dheci/files/wandb-summary.json
881
  2024-02-14 12:22:51,050 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
879
  2024-02-14 12:22:46,049 INFO SenderThread:19614 [sender.py:_save_file():1403] saving file wandb-summary.json with policy end
880
  2024-02-14 12:22:46,293 INFO Thread-12 :19614 [dir_watcher.py:_on_file_modified():288] file/dir modified: /content/wandb/run-20240214_115405-518dheci/files/wandb-summary.json
881
  2024-02-14 12:22:51,050 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
882
+ 2024-02-14 12:22:52,245 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: pause
883
+ 2024-02-14 12:22:52,245 INFO HandlerThread:19614 [handler.py:handle_request_pause():708] stopping system metrics thread
884
+ 2024-02-14 12:22:52,245 INFO HandlerThread:19614 [system_monitor.py:finish():203] Stopping system monitor
885
+ 2024-02-14 12:22:52,245 DEBUG SystemMonitor:19614 [system_monitor.py:_start():179] Finished system metrics aggregation loop
886
+ 2024-02-14 12:22:52,246 DEBUG SystemMonitor:19614 [system_monitor.py:_start():183] Publishing last batch of metrics
887
+ 2024-02-14 12:22:52,246 INFO HandlerThread:19614 [interfaces.py:finish():202] Joined cpu monitor
888
+ 2024-02-14 12:22:52,247 INFO HandlerThread:19614 [interfaces.py:finish():202] Joined disk monitor
889
+ 2024-02-14 12:22:52,254 INFO HandlerThread:19614 [interfaces.py:finish():202] Joined gpu monitor
890
+ 2024-02-14 12:22:52,254 INFO HandlerThread:19614 [interfaces.py:finish():202] Joined memory monitor
891
+ 2024-02-14 12:22:52,254 INFO HandlerThread:19614 [interfaces.py:finish():202] Joined network monitor
892
+ 2024-02-14 12:22:52,255 DEBUG SenderThread:19614 [sender.py:send():382] send: stats
893
+ 2024-02-14 12:22:54,094 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
894
+ 2024-02-14 12:22:54,094 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
895
+ 2024-02-14 12:22:54,094 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
896
+ 2024-02-14 12:22:56,215 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
897
+ 2024-02-14 12:23:01,216 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
898
+ 2024-02-14 12:23:06,221 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
899
+ 2024-02-14 12:23:07,323 INFO Thread-12 :19614 [dir_watcher.py:_on_file_modified():288] file/dir modified: /content/wandb/run-20240214_115405-518dheci/files/config.yaml
900
+ 2024-02-14 12:23:09,094 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
901
+ 2024-02-14 12:23:09,094 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
902
+ 2024-02-14 12:23:09,095 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
903
+ 2024-02-14 12:23:12,274 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
904
+ 2024-02-14 12:23:17,275 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
905
+ 2024-02-14 12:23:22,276 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
906
+ 2024-02-14 12:23:24,094 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
907
+ 2024-02-14 12:23:24,095 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
908
+ 2024-02-14 12:23:24,095 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
909
+ 2024-02-14 12:23:28,181 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
910
+ 2024-02-14 12:23:33,182 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
911
+ 2024-02-14 12:23:38,184 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
912
+ 2024-02-14 12:23:39,095 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
913
+ 2024-02-14 12:23:39,095 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
914
+ 2024-02-14 12:23:39,095 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
915
+ 2024-02-14 12:23:44,165 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
916
+ 2024-02-14 12:23:49,166 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
917
+ 2024-02-14 12:23:54,095 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
918
+ 2024-02-14 12:23:54,095 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
919
+ 2024-02-14 12:23:54,095 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
920
+ 2024-02-14 12:23:54,249 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
921
+ 2024-02-14 12:23:59,250 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
922
+ 2024-02-14 12:24:04,251 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
923
+ 2024-02-14 12:24:09,095 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
924
+ 2024-02-14 12:24:09,095 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
925
+ 2024-02-14 12:24:09,096 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
926
+ 2024-02-14 12:24:10,244 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
927
+ 2024-02-14 12:24:15,244 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
928
+ 2024-02-14 12:24:20,245 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
929
+ 2024-02-14 12:24:24,095 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
930
+ 2024-02-14 12:24:24,095 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
931
+ 2024-02-14 12:24:24,096 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
932
+ 2024-02-14 12:24:26,189 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
933
+ 2024-02-14 12:24:31,190 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
934
+ 2024-02-14 12:24:36,191 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
935
+ 2024-02-14 12:24:39,095 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
936
+ 2024-02-14 12:24:39,096 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
937
+ 2024-02-14 12:24:39,096 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
938
+ 2024-02-14 12:24:41,200 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
939
+ 2024-02-14 12:24:46,201 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
940
+ 2024-02-14 12:24:51,202 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
941
+ 2024-02-14 12:24:54,095 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
942
+ 2024-02-14 12:24:54,096 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
943
+ 2024-02-14 12:24:54,096 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
944
+ 2024-02-14 12:24:57,166 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
945
+ 2024-02-14 12:25:02,167 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
946
+ 2024-02-14 12:25:07,168 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
947
+ 2024-02-14 12:25:09,096 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
948
+ 2024-02-14 12:25:09,096 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
949
+ 2024-02-14 12:25:09,096 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
950
+ 2024-02-14 12:25:12,201 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
951
+ 2024-02-14 12:25:17,202 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
952
+ 2024-02-14 12:25:22,203 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
953
+ 2024-02-14 12:25:24,096 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
954
+ 2024-02-14 12:25:24,096 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
955
+ 2024-02-14 12:25:24,097 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
956
+ 2024-02-14 12:25:28,169 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
957
+ 2024-02-14 12:25:33,170 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
958
+ 2024-02-14 12:25:38,171 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
959
+ 2024-02-14 12:25:39,096 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
960
+ 2024-02-14 12:25:39,096 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
961
+ 2024-02-14 12:25:39,097 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
962
+ 2024-02-14 12:25:43,248 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
963
+ 2024-02-14 12:25:48,249 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
964
+ 2024-02-14 12:25:53,250 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
965
+ 2024-02-14 12:25:54,096 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
966
+ 2024-02-14 12:25:54,096 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
967
+ 2024-02-14 12:25:54,097 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
968
+ 2024-02-14 12:25:59,197 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
969
+ 2024-02-14 12:26:04,198 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
970
+ 2024-02-14 12:26:09,096 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
971
+ 2024-02-14 12:26:09,096 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
972
+ 2024-02-14 12:26:09,097 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
973
+ 2024-02-14 12:26:09,208 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
974
+ 2024-02-14 12:26:14,209 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
975
+ 2024-02-14 12:26:19,210 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
976
+ 2024-02-14 12:26:24,096 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
977
+ 2024-02-14 12:26:24,097 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
978
+ 2024-02-14 12:26:24,097 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
979
+ 2024-02-14 12:26:24,244 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
980
+ 2024-02-14 12:26:29,245 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
981
+ 2024-02-14 12:26:34,246 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
982
+ 2024-02-14 12:26:39,096 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
983
+ 2024-02-14 12:26:39,097 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
984
+ 2024-02-14 12:26:39,097 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
985
+ 2024-02-14 12:26:40,214 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
986
+ 2024-02-14 12:26:45,215 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
987
+ 2024-02-14 12:26:50,216 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
988
+ 2024-02-14 12:26:54,097 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
989
+ 2024-02-14 12:26:54,097 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
990
+ 2024-02-14 12:26:54,097 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
991
+ 2024-02-14 12:26:55,237 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
992
+ 2024-02-14 12:27:00,238 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
993
+ 2024-02-14 12:27:05,239 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
994
+ 2024-02-14 12:27:09,097 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
995
+ 2024-02-14 12:27:09,097 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
996
+ 2024-02-14 12:27:09,098 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
997
+ 2024-02-14 12:27:11,238 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
998
+ 2024-02-14 12:27:16,239 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
999
+ 2024-02-14 12:27:21,240 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1000
+ 2024-02-14 12:27:24,097 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1001
+ 2024-02-14 12:27:24,097 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1002
+ 2024-02-14 12:27:24,098 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1003
+ 2024-02-14 12:27:27,154 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1004
+ 2024-02-14 12:27:32,155 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1005
+ 2024-02-14 12:27:37,156 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1006
+ 2024-02-14 12:27:39,097 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1007
+ 2024-02-14 12:27:39,097 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1008
+ 2024-02-14 12:27:39,098 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1009
+ 2024-02-14 12:27:42,263 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1010
+ 2024-02-14 12:27:47,264 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1011
+ 2024-02-14 12:27:52,265 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1012
+ 2024-02-14 12:27:54,097 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1013
+ 2024-02-14 12:27:54,097 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1014
+ 2024-02-14 12:27:54,098 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1015
+ 2024-02-14 12:27:58,182 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1016
+ 2024-02-14 12:28:03,183 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1017
+ 2024-02-14 12:28:08,184 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1018
+ 2024-02-14 12:28:09,097 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1019
+ 2024-02-14 12:28:09,098 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1020
+ 2024-02-14 12:28:09,098 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1021
+ 2024-02-14 12:28:13,242 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1022
+ 2024-02-14 12:28:18,243 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1023
+ 2024-02-14 12:28:23,244 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1024
+ 2024-02-14 12:28:24,098 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1025
+ 2024-02-14 12:28:24,098 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1026
+ 2024-02-14 12:28:24,098 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1027
+ 2024-02-14 12:28:28,254 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1028
+ 2024-02-14 12:28:33,255 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1029
+ 2024-02-14 12:28:38,256 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1030
+ 2024-02-14 12:28:39,098 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1031
+ 2024-02-14 12:28:39,098 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1032
+ 2024-02-14 12:28:39,099 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1033
+ 2024-02-14 12:28:44,249 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1034
+ 2024-02-14 12:28:49,250 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1035
+ 2024-02-14 12:28:54,098 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1036
+ 2024-02-14 12:28:54,098 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1037
+ 2024-02-14 12:28:54,099 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1038
+ 2024-02-14 12:28:55,242 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1039
+ 2024-02-14 12:29:00,243 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1040
+ 2024-02-14 12:29:05,244 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1041
+ 2024-02-14 12:29:09,098 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1042
+ 2024-02-14 12:29:09,099 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1043
+ 2024-02-14 12:29:09,099 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1044
+ 2024-02-14 12:29:10,249 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1045
+ 2024-02-14 12:29:15,250 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1046
+ 2024-02-14 12:29:20,251 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1047
+ 2024-02-14 12:29:24,098 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1048
+ 2024-02-14 12:29:24,099 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1049
+ 2024-02-14 12:29:24,099 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1050
+ 2024-02-14 12:29:26,154 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1051
+ 2024-02-14 12:29:31,155 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1052
+ 2024-02-14 12:29:36,156 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1053
+ 2024-02-14 12:29:39,099 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1054
+ 2024-02-14 12:29:39,099 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1055
+ 2024-02-14 12:29:39,099 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1056
+ 2024-02-14 12:29:41,204 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1057
+ 2024-02-14 12:29:46,205 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1058
+ 2024-02-14 12:29:51,206 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1059
+ 2024-02-14 12:29:54,099 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1060
+ 2024-02-14 12:29:54,099 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1061
+ 2024-02-14 12:29:54,099 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1062
+ 2024-02-14 12:29:56,225 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1063
+ 2024-02-14 12:30:01,226 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1064
+ 2024-02-14 12:30:06,228 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1065
+ 2024-02-14 12:30:09,099 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1066
+ 2024-02-14 12:30:09,099 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1067
+ 2024-02-14 12:30:09,099 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1068
+ 2024-02-14 12:30:12,210 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1069
+ 2024-02-14 12:30:17,211 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1070
+ 2024-02-14 12:30:22,212 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1071
+ 2024-02-14 12:30:24,099 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1072
+ 2024-02-14 12:30:24,099 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1073
+ 2024-02-14 12:30:24,100 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1074
+ 2024-02-14 12:30:28,192 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1075
+ 2024-02-14 12:30:33,193 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1076
+ 2024-02-14 12:30:38,194 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1077
+ 2024-02-14 12:30:39,099 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1078
+ 2024-02-14 12:30:39,099 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1079
+ 2024-02-14 12:30:39,100 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1080
+ 2024-02-14 12:30:44,173 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1081
+ 2024-02-14 12:30:49,173 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1082
+ 2024-02-14 12:30:54,099 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1083
+ 2024-02-14 12:30:54,100 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1084
+ 2024-02-14 12:30:54,100 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1085
+ 2024-02-14 12:30:54,213 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1086
+ 2024-02-14 12:30:59,214 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1087
+ 2024-02-14 12:31:04,215 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1088
+ 2024-02-14 12:31:09,100 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1089
+ 2024-02-14 12:31:09,100 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1090
+ 2024-02-14 12:31:09,100 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1091
+ 2024-02-14 12:31:10,185 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1092
+ 2024-02-14 12:31:15,186 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1093
+ 2024-02-14 12:31:20,187 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1094
+ 2024-02-14 12:31:24,100 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1095
+ 2024-02-14 12:31:24,100 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1096
+ 2024-02-14 12:31:24,100 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1097
+ 2024-02-14 12:31:26,184 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1098
+ 2024-02-14 12:31:31,185 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1099
+ 2024-02-14 12:31:36,186 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1100
+ 2024-02-14 12:31:39,100 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1101
+ 2024-02-14 12:31:39,100 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1102
+ 2024-02-14 12:31:39,101 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1103
+ 2024-02-14 12:31:41,236 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1104
+ 2024-02-14 12:31:46,237 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1105
+ 2024-02-14 12:31:51,238 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1106
+ 2024-02-14 12:31:54,100 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1107
+ 2024-02-14 12:31:54,100 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1108
+ 2024-02-14 12:31:54,101 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1109
+ 2024-02-14 12:31:57,220 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1110
+ 2024-02-14 12:32:02,221 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1111
+ 2024-02-14 12:32:07,222 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1112
+ 2024-02-14 12:32:09,100 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1113
+ 2024-02-14 12:32:09,100 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1114
+ 2024-02-14 12:32:09,101 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1115
+ 2024-02-14 12:32:13,211 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1116
+ 2024-02-14 12:32:18,211 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1117
+ 2024-02-14 12:32:23,213 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1118
+ 2024-02-14 12:32:24,100 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1119
+ 2024-02-14 12:32:24,101 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1120
+ 2024-02-14 12:32:24,101 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1121
+ 2024-02-14 12:32:28,225 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1122
+ 2024-02-14 12:32:33,226 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1123
+ 2024-02-14 12:32:38,227 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1124
+ 2024-02-14 12:32:39,101 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1125
+ 2024-02-14 12:32:39,101 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1126
+ 2024-02-14 12:32:39,101 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1127
+ 2024-02-14 12:32:44,181 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1128
+ 2024-02-14 12:32:49,182 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1129
+ 2024-02-14 12:32:54,101 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1130
+ 2024-02-14 12:32:54,101 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1131
+ 2024-02-14 12:32:54,101 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1132
+ 2024-02-14 12:32:55,172 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1133
+ 2024-02-14 12:33:00,174 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1134
+ 2024-02-14 12:33:05,174 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1135
+ 2024-02-14 12:33:09,101 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1136
+ 2024-02-14 12:33:09,101 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1137
+ 2024-02-14 12:33:09,101 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1138
+ 2024-02-14 12:33:10,187 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1139
+ 2024-02-14 12:33:15,188 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1140
+ 2024-02-14 12:33:20,189 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1141
+ 2024-02-14 12:33:24,101 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1142
+ 2024-02-14 12:33:24,101 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1143
+ 2024-02-14 12:33:24,102 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1144
+ 2024-02-14 12:33:25,231 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1145
+ 2024-02-14 12:33:30,232 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1146
+ 2024-02-14 12:33:35,233 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1147
+ 2024-02-14 12:33:39,101 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1148
+ 2024-02-14 12:33:39,101 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1149
+ 2024-02-14 12:33:39,102 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1150
+ 2024-02-14 12:33:40,253 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1151
+ 2024-02-14 12:33:45,253 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1152
+ 2024-02-14 12:33:50,255 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1153
+ 2024-02-14 12:33:54,102 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1154
+ 2024-02-14 12:33:54,102 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1155
+ 2024-02-14 12:33:54,102 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1156
+ 2024-02-14 12:33:56,233 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1157
+ 2024-02-14 12:34:01,234 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1158
+ 2024-02-14 12:34:06,235 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1159
+ 2024-02-14 12:34:09,102 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1160
+ 2024-02-14 12:34:09,102 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1161
+ 2024-02-14 12:34:09,102 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1162
+ 2024-02-14 12:34:12,195 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1163
+ 2024-02-14 12:34:17,196 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1164
+ 2024-02-14 12:34:22,197 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1165
+ 2024-02-14 12:34:24,102 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1166
+ 2024-02-14 12:34:24,102 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1167
+ 2024-02-14 12:34:24,103 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1168
+ 2024-02-14 12:34:28,182 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1169
+ 2024-02-14 12:34:33,183 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1170
+ 2024-02-14 12:34:38,184 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1171
+ 2024-02-14 12:34:39,102 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1172
+ 2024-02-14 12:34:39,102 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1173
+ 2024-02-14 12:34:39,102 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1174
+ 2024-02-14 12:34:43,253 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1175
+ 2024-02-14 12:34:48,254 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1176
+ 2024-02-14 12:34:53,255 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1177
+ 2024-02-14 12:34:54,102 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1178
+ 2024-02-14 12:34:54,102 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1179
+ 2024-02-14 12:34:54,103 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1180
+ 2024-02-14 12:34:59,254 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1181
+ 2024-02-14 12:35:04,255 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1182
+ 2024-02-14 12:35:09,102 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1183
+ 2024-02-14 12:35:09,102 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1184
+ 2024-02-14 12:35:09,103 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1185
+ 2024-02-14 12:35:09,257 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1186
+ 2024-02-14 12:35:14,258 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1187
+ 2024-02-14 12:35:19,259 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1188
+ 2024-02-14 12:35:24,103 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1189
+ 2024-02-14 12:35:24,103 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1190
+ 2024-02-14 12:35:24,103 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1191
+ 2024-02-14 12:35:25,173 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1192
+ 2024-02-14 12:35:30,174 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1193
+ 2024-02-14 12:35:35,175 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1194
+ 2024-02-14 12:35:39,103 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1195
+ 2024-02-14 12:35:39,103 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1196
+ 2024-02-14 12:35:39,103 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1197
+ 2024-02-14 12:35:40,269 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1198
+ 2024-02-14 12:35:45,270 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1199
+ 2024-02-14 12:35:50,271 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1200
+ 2024-02-14 12:35:54,103 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1201
+ 2024-02-14 12:35:54,103 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1202
+ 2024-02-14 12:35:54,103 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1203
+ 2024-02-14 12:35:56,219 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1204
+ 2024-02-14 12:36:01,220 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1205
+ 2024-02-14 12:36:06,221 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1206
+ 2024-02-14 12:36:09,103 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1207
+ 2024-02-14 12:36:09,103 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1208
+ 2024-02-14 12:36:09,104 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1209
+ 2024-02-14 12:36:12,201 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1210
+ 2024-02-14 12:36:17,203 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1211
+ 2024-02-14 12:36:22,204 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1212
+ 2024-02-14 12:36:24,103 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1213
+ 2024-02-14 12:36:24,103 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1214
+ 2024-02-14 12:36:24,104 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1215
+ 2024-02-14 12:36:28,184 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1216
+ 2024-02-14 12:36:33,185 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1217
+ 2024-02-14 12:36:38,186 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1218
+ 2024-02-14 12:36:39,103 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1219
+ 2024-02-14 12:36:39,103 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1220
+ 2024-02-14 12:36:39,104 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1221
+ 2024-02-14 12:36:43,271 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1222
+ 2024-02-14 12:36:48,273 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1223
+ 2024-02-14 12:36:53,274 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1224
+ 2024-02-14 12:36:54,103 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1225
+ 2024-02-14 12:36:54,104 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1226
+ 2024-02-14 12:36:54,144 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1227
+ 2024-02-14 12:36:59,198 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1228
+ 2024-02-14 12:37:04,199 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1229
+ 2024-02-14 12:37:09,104 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1230
+ 2024-02-14 12:37:09,104 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1231
+ 2024-02-14 12:37:09,104 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1232
+ 2024-02-14 12:37:10,195 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1233
+ 2024-02-14 12:37:15,196 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1234
+ 2024-02-14 12:37:20,197 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1235
+ 2024-02-14 12:37:24,104 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1236
+ 2024-02-14 12:37:24,104 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1237
+ 2024-02-14 12:37:24,104 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1238
+ 2024-02-14 12:37:25,242 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1239
+ 2024-02-14 12:37:30,243 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1240
+ 2024-02-14 12:37:35,244 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1241
+ 2024-02-14 12:37:39,104 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1242
+ 2024-02-14 12:37:39,104 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1243
+ 2024-02-14 12:37:39,105 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1244
+ 2024-02-14 12:37:40,265 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1245
+ 2024-02-14 12:37:45,266 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1246
+ 2024-02-14 12:37:50,267 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1247
+ 2024-02-14 12:37:54,104 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1248
+ 2024-02-14 12:37:54,104 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1249
+ 2024-02-14 12:37:54,105 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1250
+ 2024-02-14 12:37:56,254 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1251
+ 2024-02-14 12:38:01,255 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1252
+ 2024-02-14 12:38:06,256 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1253
+ 2024-02-14 12:38:09,104 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1254
+ 2024-02-14 12:38:09,104 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1255
+ 2024-02-14 12:38:09,105 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1256
+ 2024-02-14 12:38:12,243 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1257
+ 2024-02-14 12:38:17,245 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1258
+ 2024-02-14 12:38:22,246 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1259
+ 2024-02-14 12:38:24,105 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1260
+ 2024-02-14 12:38:24,105 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1261
+ 2024-02-14 12:38:24,105 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1262
+ 2024-02-14 12:38:28,193 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1263
+ 2024-02-14 12:38:33,194 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1264
+ 2024-02-14 12:38:38,195 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1265
+ 2024-02-14 12:38:39,105 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1266
+ 2024-02-14 12:38:39,105 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1267
+ 2024-02-14 12:38:39,105 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1268
+ 2024-02-14 12:38:44,193 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1269
+ 2024-02-14 12:38:49,194 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1270
+ 2024-02-14 12:38:54,105 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1271
+ 2024-02-14 12:38:54,105 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1272
+ 2024-02-14 12:38:54,106 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1273
+ 2024-02-14 12:38:54,236 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1274
+ 2024-02-14 12:38:59,237 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1275
+ 2024-02-14 12:39:04,238 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1276
+ 2024-02-14 12:39:09,105 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1277
+ 2024-02-14 12:39:09,105 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1278
+ 2024-02-14 12:39:09,106 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1279
+ 2024-02-14 12:39:09,255 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1280
+ 2024-02-14 12:39:14,256 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1281
+ 2024-02-14 12:39:19,257 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1282
+ 2024-02-14 12:39:24,105 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1283
+ 2024-02-14 12:39:24,105 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1284
+ 2024-02-14 12:39:24,106 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1285
+ 2024-02-14 12:39:25,216 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1286
+ 2024-02-14 12:39:30,217 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1287
+ 2024-02-14 12:39:35,218 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1288
+ 2024-02-14 12:39:39,105 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1289
+ 2024-02-14 12:39:39,106 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1290
+ 2024-02-14 12:39:39,106 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1291
+ 2024-02-14 12:39:40,256 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1292
+ 2024-02-14 12:39:45,257 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1293
+ 2024-02-14 12:39:50,258 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1294
+ 2024-02-14 12:39:54,105 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1295
+ 2024-02-14 12:39:54,106 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1296
+ 2024-02-14 12:39:54,106 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1297
+ 2024-02-14 12:39:56,174 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1298
+ 2024-02-14 12:40:01,175 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1299
+ 2024-02-14 12:40:06,176 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1300
+ 2024-02-14 12:40:09,105 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1301
+ 2024-02-14 12:40:09,106 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1302
+ 2024-02-14 12:40:09,106 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1303
+ 2024-02-14 12:40:12,172 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1304
+ 2024-02-14 12:40:17,173 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1305
+ 2024-02-14 12:40:22,174 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1306
+ 2024-02-14 12:40:24,105 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1307
+ 2024-02-14 12:40:24,106 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1308
+ 2024-02-14 12:40:24,106 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1309
+ 2024-02-14 12:40:27,185 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1310
+ 2024-02-14 12:40:32,186 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1311
+ 2024-02-14 12:40:37,187 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1312
+ 2024-02-14 12:40:39,106 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1313
+ 2024-02-14 12:40:39,106 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1314
+ 2024-02-14 12:40:39,106 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1315
+ 2024-02-14 12:40:42,258 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1316
+ 2024-02-14 12:40:47,259 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1317
+ 2024-02-14 12:40:52,260 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1318
+ 2024-02-14 12:40:54,106 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1319
+ 2024-02-14 12:40:54,106 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1320
+ 2024-02-14 12:40:54,107 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1321
+ 2024-02-14 12:40:58,254 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1322
+ 2024-02-14 12:41:03,255 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1323
+ 2024-02-14 12:41:08,256 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1324
+ 2024-02-14 12:41:09,106 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1325
+ 2024-02-14 12:41:09,106 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1326
+ 2024-02-14 12:41:09,107 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1327
+ 2024-02-14 12:41:13,269 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1328
+ 2024-02-14 12:41:18,270 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1329
+ 2024-02-14 12:41:23,271 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1330
+ 2024-02-14 12:41:24,106 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1331
+ 2024-02-14 12:41:24,107 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1332
+ 2024-02-14 12:41:24,107 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1333
+ 2024-02-14 12:41:29,233 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1334
+ 2024-02-14 12:41:34,234 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1335
+ 2024-02-14 12:41:39,106 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1336
+ 2024-02-14 12:41:39,107 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1337
+ 2024-02-14 12:41:39,107 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1338
+ 2024-02-14 12:41:39,257 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1339
+ 2024-02-14 12:41:44,258 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1340
+ 2024-02-14 12:41:49,259 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1341
+ 2024-02-14 12:41:54,106 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1342
+ 2024-02-14 12:41:54,107 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1343
+ 2024-02-14 12:41:54,107 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1344
+ 2024-02-14 12:41:55,257 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1345
+ 2024-02-14 12:42:00,258 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1346
+ 2024-02-14 12:42:05,259 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1347
+ 2024-02-14 12:42:09,107 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1348
+ 2024-02-14 12:42:09,107 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1349
+ 2024-02-14 12:42:09,107 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1350
+ 2024-02-14 12:42:10,266 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1351
+ 2024-02-14 12:42:15,267 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1352
+ 2024-02-14 12:42:20,268 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1353
+ 2024-02-14 12:42:24,107 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1354
+ 2024-02-14 12:42:24,107 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1355
+ 2024-02-14 12:42:24,108 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1356
+ 2024-02-14 12:42:26,221 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1357
+ 2024-02-14 12:42:31,221 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1358
+ 2024-02-14 12:42:36,222 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1359
+ 2024-02-14 12:42:39,107 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1360
+ 2024-02-14 12:42:39,107 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1361
+ 2024-02-14 12:42:39,108 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1362
+ 2024-02-14 12:42:41,277 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1363
+ 2024-02-14 12:42:46,278 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1364
+ 2024-02-14 12:42:51,279 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1365
+ 2024-02-14 12:42:54,107 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1366
+ 2024-02-14 12:42:54,107 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1367
+ 2024-02-14 12:42:54,108 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1368
+ 2024-02-14 12:42:57,228 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1369
+ 2024-02-14 12:43:02,229 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1370
+ 2024-02-14 12:43:07,230 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1371
+ 2024-02-14 12:43:09,107 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1372
+ 2024-02-14 12:43:09,108 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1373
+ 2024-02-14 12:43:09,108 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1374
+ 2024-02-14 12:43:13,193 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1375
+ 2024-02-14 12:43:18,194 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1376
+ 2024-02-14 12:43:23,195 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1377
+ 2024-02-14 12:43:24,107 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1378
+ 2024-02-14 12:43:24,108 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1379
+ 2024-02-14 12:43:24,108 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1380
+ 2024-02-14 12:43:29,184 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1381
+ 2024-02-14 12:43:34,185 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1382
+ 2024-02-14 12:43:39,107 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1383
+ 2024-02-14 12:43:39,108 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1384
+ 2024-02-14 12:43:39,108 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1385
+ 2024-02-14 12:43:39,186 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1386
+ 2024-02-14 12:43:44,187 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1387
+ 2024-02-14 12:43:49,188 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1388
+ 2024-02-14 12:43:54,108 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1389
+ 2024-02-14 12:43:54,108 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1390
+ 2024-02-14 12:43:54,108 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1391
+ 2024-02-14 12:43:54,216 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1392
+ 2024-02-14 12:43:59,217 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1393
+ 2024-02-14 12:44:04,217 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1394
+ 2024-02-14 12:44:09,108 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1395
+ 2024-02-14 12:44:09,108 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1396
+ 2024-02-14 12:44:09,108 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1397
+ 2024-02-14 12:44:10,204 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1398
+ 2024-02-14 12:44:15,204 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1399
+ 2024-02-14 12:44:20,206 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1400
+ 2024-02-14 12:44:24,108 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1401
+ 2024-02-14 12:44:24,108 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1402
+ 2024-02-14 12:44:24,109 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1403
+ 2024-02-14 12:44:25,258 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1404
+ 2024-02-14 12:44:30,259 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1405
+ 2024-02-14 12:44:35,261 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1406
+ 2024-02-14 12:44:39,108 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1407
+ 2024-02-14 12:44:39,108 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1408
+ 2024-02-14 12:44:39,109 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1409
+ 2024-02-14 12:44:41,259 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1410
+ 2024-02-14 12:44:46,260 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1411
+ 2024-02-14 12:44:51,261 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1412
+ 2024-02-14 12:44:54,108 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1413
+ 2024-02-14 12:44:54,108 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1414
+ 2024-02-14 12:44:54,109 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1415
+ 2024-02-14 12:44:57,243 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1416
+ 2024-02-14 12:45:02,244 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1417
+ 2024-02-14 12:45:07,245 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1418
+ 2024-02-14 12:45:09,108 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1419
+ 2024-02-14 12:45:09,109 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1420
+ 2024-02-14 12:45:09,109 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1421
+ 2024-02-14 12:45:12,268 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1422
+ 2024-02-14 12:45:17,269 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1423
+ 2024-02-14 12:45:22,270 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1424
+ 2024-02-14 12:45:24,108 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1425
+ 2024-02-14 12:45:24,109 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1426
+ 2024-02-14 12:45:24,109 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1427
+ 2024-02-14 12:45:28,249 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1428
+ 2024-02-14 12:45:33,249 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1429
+ 2024-02-14 12:45:38,250 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1430
+ 2024-02-14 12:45:39,109 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1431
+ 2024-02-14 12:45:39,109 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1432
+ 2024-02-14 12:45:39,109 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1433
+ 2024-02-14 12:45:44,224 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1434
+ 2024-02-14 12:45:49,225 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1435
+ 2024-02-14 12:45:54,109 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1436
+ 2024-02-14 12:45:54,109 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1437
+ 2024-02-14 12:45:54,109 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1438
+ 2024-02-14 12:45:55,220 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1439
+ 2024-02-14 12:46:00,221 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1440
+ 2024-02-14 12:46:05,222 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1441
+ 2024-02-14 12:46:09,109 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1442
+ 2024-02-14 12:46:09,109 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1443
+ 2024-02-14 12:46:09,109 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1444
+ 2024-02-14 12:46:11,177 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1445
+ 2024-02-14 12:46:16,178 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1446
+ 2024-02-14 12:46:21,179 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1447
+ 2024-02-14 12:46:24,109 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1448
+ 2024-02-14 12:46:24,109 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1449
+ 2024-02-14 12:46:24,109 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1450
+ 2024-02-14 12:46:26,240 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1451
+ 2024-02-14 12:46:31,241 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1452
+ 2024-02-14 12:46:36,242 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1453
+ 2024-02-14 12:46:39,109 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1454
+ 2024-02-14 12:46:39,109 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1455
+ 2024-02-14 12:46:39,110 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1456
+ 2024-02-14 12:46:41,264 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1457
+ 2024-02-14 12:46:46,265 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1458
+ 2024-02-14 12:46:51,266 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1459
+ 2024-02-14 12:46:54,109 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1460
+ 2024-02-14 12:46:54,109 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1461
+ 2024-02-14 12:46:54,110 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1462
+ 2024-02-14 12:46:57,214 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1463
+ 2024-02-14 12:47:02,215 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1464
+ 2024-02-14 12:47:07,216 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1465
+ 2024-02-14 12:47:09,109 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1466
+ 2024-02-14 12:47:09,110 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1467
+ 2024-02-14 12:47:09,110 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1468
+ 2024-02-14 12:47:13,175 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1469
+ 2024-02-14 12:47:18,176 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1470
+ 2024-02-14 12:47:23,177 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1471
+ 2024-02-14 12:47:24,109 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1472
+ 2024-02-14 12:47:24,110 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1473
+ 2024-02-14 12:47:24,110 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1474
+ 2024-02-14 12:47:28,178 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1475
+ 2024-02-14 12:47:33,179 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1476
+ 2024-02-14 12:47:38,180 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1477
+ 2024-02-14 12:47:39,109 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1478
+ 2024-02-14 12:47:39,110 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1479
+ 2024-02-14 12:47:39,110 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1480
+ 2024-02-14 12:47:43,207 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1481
+ 2024-02-14 12:47:48,208 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1482
+ 2024-02-14 12:47:53,209 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1483
+ 2024-02-14 12:47:54,110 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1484
+ 2024-02-14 12:47:54,110 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1485
+ 2024-02-14 12:47:54,110 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1486
+ 2024-02-14 12:47:59,204 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1487
+ 2024-02-14 12:48:04,205 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1488
+ 2024-02-14 12:48:09,110 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1489
+ 2024-02-14 12:48:09,110 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1490
+ 2024-02-14 12:48:09,110 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1491
+ 2024-02-14 12:48:09,217 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1492
+ 2024-02-14 12:48:14,218 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1493
+ 2024-02-14 12:48:19,219 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1494
+ 2024-02-14 12:48:24,110 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1495
+ 2024-02-14 12:48:24,110 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1496
+ 2024-02-14 12:48:24,111 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1497
+ 2024-02-14 12:48:24,263 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1498
+ 2024-02-14 12:48:29,264 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1499
+ 2024-02-14 12:48:34,265 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1500
+ 2024-02-14 12:48:39,110 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1501
+ 2024-02-14 12:48:39,110 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1502
+ 2024-02-14 12:48:39,111 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1503
+ 2024-02-14 12:48:40,177 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1504
+ 2024-02-14 12:48:45,178 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1505
+ 2024-02-14 12:48:50,179 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1506
+ 2024-02-14 12:48:54,110 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1507
+ 2024-02-14 12:48:54,111 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1508
+ 2024-02-14 12:48:54,111 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1509
+ 2024-02-14 12:48:55,241 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1510
+ 2024-02-14 12:49:00,242 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1511
+ 2024-02-14 12:49:05,243 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1512
+ 2024-02-14 12:49:09,111 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1513
+ 2024-02-14 12:49:09,111 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1514
+ 2024-02-14 12:49:09,111 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1515
+ 2024-02-14 12:49:11,190 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1516
+ 2024-02-14 12:49:16,191 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1517
+ 2024-02-14 12:49:21,191 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1518
+ 2024-02-14 12:49:24,110 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1519
+ 2024-02-14 12:49:24,111 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1520
+ 2024-02-14 12:49:24,111 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1521
+ 2024-02-14 12:49:26,227 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1522
+ 2024-02-14 12:49:31,228 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1523
+ 2024-02-14 12:49:31,961 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: resume
1524
+ 2024-02-14 12:49:31,961 INFO HandlerThread:19614 [handler.py:handle_request_resume():699] starting system metrics thread
1525
+ 2024-02-14 12:49:31,961 INFO HandlerThread:19614 [system_monitor.py:start():194] Starting system monitor
1526
+ 2024-02-14 12:49:31,961 INFO SystemMonitor:19614 [system_monitor.py:_start():158] Starting system asset monitoring threads
1527
+ 2024-02-14 12:49:31,961 INFO SystemMonitor:19614 [interfaces.py:start():190] Started cpu monitoring
1528
+ 2024-02-14 12:49:31,962 INFO SystemMonitor:19614 [interfaces.py:start():190] Started disk monitoring
1529
+ 2024-02-14 12:49:31,963 INFO SystemMonitor:19614 [interfaces.py:start():190] Started gpu monitoring
1530
+ 2024-02-14 12:49:31,964 INFO SystemMonitor:19614 [interfaces.py:start():190] Started memory monitoring
1531
+ 2024-02-14 12:49:31,964 INFO SystemMonitor:19614 [interfaces.py:start():190] Started network monitoring
1532
+ 2024-02-14 12:49:32,001 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: pause
1533
+ 2024-02-14 12:49:32,001 INFO HandlerThread:19614 [handler.py:handle_request_pause():708] stopping system metrics thread
1534
+ 2024-02-14 12:49:32,001 INFO HandlerThread:19614 [system_monitor.py:finish():203] Stopping system monitor
1535
+ 2024-02-14 12:49:32,001 DEBUG SystemMonitor:19614 [system_monitor.py:_start():172] Starting system metrics aggregation loop
1536
+ 2024-02-14 12:49:32,001 DEBUG SystemMonitor:19614 [system_monitor.py:_start():179] Finished system metrics aggregation loop
1537
+ 2024-02-14 12:49:32,001 DEBUG SystemMonitor:19614 [system_monitor.py:_start():183] Publishing last batch of metrics
1538
+ 2024-02-14 12:49:32,002 INFO HandlerThread:19614 [interfaces.py:finish():202] Joined cpu monitor
1539
+ 2024-02-14 12:49:32,002 INFO HandlerThread:19614 [interfaces.py:finish():202] Joined disk monitor
1540
+ 2024-02-14 12:49:32,008 INFO HandlerThread:19614 [interfaces.py:finish():202] Joined gpu monitor
1541
+ 2024-02-14 12:49:32,008 INFO HandlerThread:19614 [interfaces.py:finish():202] Joined memory monitor
1542
+ 2024-02-14 12:49:32,008 INFO HandlerThread:19614 [interfaces.py:finish():202] Joined network monitor
1543
+ 2024-02-14 12:49:32,008 DEBUG SenderThread:19614 [sender.py:send():382] send: stats
1544
+ 2024-02-14 12:49:34,169 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: resume
1545
+ 2024-02-14 12:49:34,169 INFO HandlerThread:19614 [handler.py:handle_request_resume():699] starting system metrics thread
1546
+ 2024-02-14 12:49:34,169 INFO HandlerThread:19614 [system_monitor.py:start():194] Starting system monitor
1547
+ 2024-02-14 12:49:34,170 INFO SystemMonitor:19614 [system_monitor.py:_start():158] Starting system asset monitoring threads
1548
+ 2024-02-14 12:49:34,170 INFO SystemMonitor:19614 [interfaces.py:start():190] Started cpu monitoring
1549
+ 2024-02-14 12:49:34,171 INFO SystemMonitor:19614 [interfaces.py:start():190] Started disk monitoring
1550
+ 2024-02-14 12:49:34,171 INFO SystemMonitor:19614 [interfaces.py:start():190] Started gpu monitoring
1551
+ 2024-02-14 12:49:34,172 INFO SystemMonitor:19614 [interfaces.py:start():190] Started memory monitoring
1552
+ 2024-02-14 12:49:34,172 INFO SystemMonitor:19614 [interfaces.py:start():190] Started network monitoring
1553
+ 2024-02-14 12:49:34,433 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: pause
1554
+ 2024-02-14 12:49:34,433 INFO HandlerThread:19614 [handler.py:handle_request_pause():708] stopping system metrics thread
1555
+ 2024-02-14 12:49:34,433 INFO HandlerThread:19614 [system_monitor.py:finish():203] Stopping system monitor
1556
+ 2024-02-14 12:49:34,434 DEBUG SystemMonitor:19614 [system_monitor.py:_start():172] Starting system metrics aggregation loop
1557
+ 2024-02-14 12:49:34,434 DEBUG SystemMonitor:19614 [system_monitor.py:_start():179] Finished system metrics aggregation loop
1558
+ 2024-02-14 12:49:34,434 DEBUG SystemMonitor:19614 [system_monitor.py:_start():183] Publishing last batch of metrics
1559
+ 2024-02-14 12:49:34,435 INFO HandlerThread:19614 [interfaces.py:finish():202] Joined cpu monitor
1560
+ 2024-02-14 12:49:34,435 INFO HandlerThread:19614 [interfaces.py:finish():202] Joined disk monitor
1561
+ 2024-02-14 12:49:34,440 INFO HandlerThread:19614 [interfaces.py:finish():202] Joined gpu monitor
1562
+ 2024-02-14 12:49:34,441 INFO HandlerThread:19614 [interfaces.py:finish():202] Joined memory monitor
1563
+ 2024-02-14 12:49:34,441 INFO HandlerThread:19614 [interfaces.py:finish():202] Joined network monitor
1564
+ 2024-02-14 12:49:34,441 DEBUG SenderThread:19614 [sender.py:send():382] send: stats
1565
+ 2024-02-14 12:49:36,442 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1566
+ 2024-02-14 12:49:36,952 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: resume
1567
+ 2024-02-14 12:49:36,952 INFO HandlerThread:19614 [handler.py:handle_request_resume():699] starting system metrics thread
1568
+ 2024-02-14 12:49:36,952 INFO HandlerThread:19614 [system_monitor.py:start():194] Starting system monitor
1569
+ 2024-02-14 12:49:36,952 INFO SystemMonitor:19614 [system_monitor.py:_start():158] Starting system asset monitoring threads
1570
+ 2024-02-14 12:49:36,953 INFO SystemMonitor:19614 [interfaces.py:start():190] Started cpu monitoring
1571
+ 2024-02-14 12:49:36,953 INFO SystemMonitor:19614 [interfaces.py:start():190] Started disk monitoring
1572
+ 2024-02-14 12:49:36,954 INFO SystemMonitor:19614 [interfaces.py:start():190] Started gpu monitoring
1573
+ 2024-02-14 12:49:36,954 INFO SystemMonitor:19614 [interfaces.py:start():190] Started memory monitoring
1574
+ 2024-02-14 12:49:36,955 INFO SystemMonitor:19614 [interfaces.py:start():190] Started network monitoring
1575
+ 2024-02-14 12:49:36,958 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: pause
1576
+ 2024-02-14 12:49:36,958 INFO HandlerThread:19614 [handler.py:handle_request_pause():708] stopping system metrics thread
1577
+ 2024-02-14 12:49:36,958 INFO HandlerThread:19614 [system_monitor.py:finish():203] Stopping system monitor
1578
+ 2024-02-14 12:49:36,959 INFO HandlerThread:19614 [interfaces.py:finish():202] Joined cpu monitor
1579
+ 2024-02-14 12:49:36,959 INFO HandlerThread:19614 [interfaces.py:finish():202] Joined disk monitor
1580
+ 2024-02-14 12:49:36,959 DEBUG SystemMonitor:19614 [system_monitor.py:_start():172] Starting system metrics aggregation loop
1581
+ 2024-02-14 12:49:36,959 DEBUG SystemMonitor:19614 [system_monitor.py:_start():179] Finished system metrics aggregation loop
1582
+ 2024-02-14 12:49:36,959 DEBUG SystemMonitor:19614 [system_monitor.py:_start():183] Publishing last batch of metrics
1583
+ 2024-02-14 12:49:36,966 INFO HandlerThread:19614 [interfaces.py:finish():202] Joined gpu monitor
1584
+ 2024-02-14 12:49:36,966 INFO HandlerThread:19614 [interfaces.py:finish():202] Joined memory monitor
1585
+ 2024-02-14 12:49:36,966 INFO HandlerThread:19614 [interfaces.py:finish():202] Joined network monitor
1586
+ 2024-02-14 12:49:36,967 DEBUG SenderThread:19614 [sender.py:send():382] send: stats
1587
+ 2024-02-14 12:49:38,539 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: resume
1588
+ 2024-02-14 12:49:38,540 INFO HandlerThread:19614 [handler.py:handle_request_resume():699] starting system metrics thread
1589
+ 2024-02-14 12:49:38,540 INFO HandlerThread:19614 [system_monitor.py:start():194] Starting system monitor
1590
+ 2024-02-14 12:49:38,540 INFO SystemMonitor:19614 [system_monitor.py:_start():158] Starting system asset monitoring threads
1591
+ 2024-02-14 12:49:38,540 INFO SystemMonitor:19614 [interfaces.py:start():190] Started cpu monitoring
1592
+ 2024-02-14 12:49:38,541 INFO SystemMonitor:19614 [interfaces.py:start():190] Started disk monitoring
1593
+ 2024-02-14 12:49:38,542 INFO SystemMonitor:19614 [interfaces.py:start():190] Started gpu monitoring
1594
+ 2024-02-14 12:49:38,542 INFO SystemMonitor:19614 [interfaces.py:start():190] Started memory monitoring
1595
+ 2024-02-14 12:49:38,544 INFO SystemMonitor:19614 [interfaces.py:start():190] Started network monitoring
1596
+ 2024-02-14 12:49:39,006 DEBUG SenderThread:19614 [sender.py:send():382] send: config
1597
+ 2024-02-14 12:49:39,007 DEBUG SenderThread:19614 [sender.py:send():382] send: metric
1598
+ 2024-02-14 12:49:39,007 DEBUG SenderThread:19614 [sender.py:send():382] send: metric
1599
+ 2024-02-14 12:49:39,007 WARNING SenderThread:19614 [sender.py:send_metric():1354] Seen metric with glob (shouldn't happen)
1600
+ 2024-02-14 12:49:39,110 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1601
+ 2024-02-14 12:49:39,111 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1602
+ 2024-02-14 12:49:39,111 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1603
+ 2024-02-14 12:49:42,264 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1604
+ 2024-02-14 12:49:47,265 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1605
+ 2024-02-14 12:49:52,266 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1606
+ 2024-02-14 12:49:54,111 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1607
+ 2024-02-14 12:49:54,111 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1608
+ 2024-02-14 12:49:54,112 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1609
+ 2024-02-14 12:49:58,222 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1610
+ 2024-02-14 12:50:03,223 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1611
+ 2024-02-14 12:50:08,229 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1612
+ 2024-02-14 12:50:08,884 INFO Thread-12 :19614 [dir_watcher.py:_on_file_modified():288] file/dir modified: /content/wandb/run-20240214_115405-518dheci/files/config.yaml
1613
+ 2024-02-14 12:50:09,111 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1614
+ 2024-02-14 12:50:09,111 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1615
+ 2024-02-14 12:50:09,112 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1616
+ 2024-02-14 12:50:14,223 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1617
+ 2024-02-14 12:50:19,224 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1618
+ 2024-02-14 12:50:24,111 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1619
+ 2024-02-14 12:50:24,111 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1620
+ 2024-02-14 12:50:24,112 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1621
+ 2024-02-14 12:50:25,201 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1622
+ 2024-02-14 12:50:30,202 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1623
+ 2024-02-14 12:50:35,203 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1624
+ 2024-02-14 12:50:38,544 DEBUG SystemMonitor:19614 [system_monitor.py:_start():172] Starting system metrics aggregation loop
1625
+ 2024-02-14 12:50:38,545 DEBUG SenderThread:19614 [sender.py:send():382] send: stats
1626
+ 2024-02-14 12:50:39,111 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1627
+ 2024-02-14 12:50:39,112 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1628
+ 2024-02-14 12:50:39,112 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1629
+ 2024-02-14 12:50:41,197 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1630
+ 2024-02-14 12:50:46,198 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1631
+ 2024-02-14 12:50:51,199 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1632
+ 2024-02-14 12:50:54,111 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1633
+ 2024-02-14 12:50:54,112 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1634
+ 2024-02-14 12:50:54,112 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1635
+ 2024-02-14 12:50:57,198 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1636
+ 2024-02-14 12:50:57,234 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: partial_history
1637
+ 2024-02-14 12:50:57,235 DEBUG SenderThread:19614 [sender.py:send():382] send: history
1638
+ 2024-02-14 12:50:57,236 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: summary_record
1639
+ 2024-02-14 12:50:57,236 INFO SenderThread:19614 [sender.py:_save_file():1403] saving file wandb-summary.json with policy end
1640
+ 2024-02-14 12:50:57,900 INFO Thread-12 :19614 [dir_watcher.py:_on_file_modified():288] file/dir modified: /content/wandb/run-20240214_115405-518dheci/files/wandb-summary.json
1641
+ 2024-02-14 12:50:57,901 INFO Thread-12 :19614 [dir_watcher.py:_on_file_created():271] file/dir created: /content/wandb/run-20240214_115405-518dheci/files/output.log
1642
+ 2024-02-14 12:50:59,901 INFO Thread-12 :19614 [dir_watcher.py:_on_file_modified():288] file/dir modified: /content/wandb/run-20240214_115405-518dheci/files/output.log
1643
+ 2024-02-14 12:51:02,277 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1644
+ 2024-02-14 12:51:07,278 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1645
+ 2024-02-14 12:51:08,563 DEBUG SenderThread:19614 [sender.py:send():382] send: stats
1646
+ 2024-02-14 12:51:09,112 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1647
+ 2024-02-14 12:51:09,112 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1648
+ 2024-02-14 12:51:09,112 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
wandb/debug.log CHANGED
@@ -28,3 +28,16 @@ config: {}
28
  2024-02-14 11:54:08,936 INFO MainThread:4105 [wandb_run.py:_redirect():2186] Redirects installed.
29
  2024-02-14 11:54:08,937 INFO MainThread:4105 [wandb_init.py:init():847] run started, returning control to user process
30
  2024-02-14 11:54:08,940 INFO MainThread:4105 [wandb_run.py:_config_callback():1343] config_cb None None {'vocab_size': 65024, 'hidden_size': 4544, 'num_hidden_layers': 32, 'num_attention_heads': 71, 'layer_norm_epsilon': 1e-05, 'initializer_range': 0.02, 'use_cache': False, 'hidden_dropout': 0.0, 'attention_dropout': 0.0, 'bos_token_id': 11, 'eos_token_id': 11, 'num_kv_heads': 71, 'alibi': False, 'new_decoder_architecture': False, 'multi_query': True, 'parallel_attn': True, 'bias': False, 'return_dict': True, 'output_hidden_states': False, 'output_attentions': False, 'torchscript': False, 'torch_dtype': 'bfloat16', 'use_bfloat16': False, 'tf_legacy_loss': False, 'pruned_heads': {}, 'tie_word_embeddings': True, 'chunk_size_feed_forward': 0, 'is_encoder_decoder': False, 'is_decoder': False, 'cross_attention_hidden_size': None, 'add_cross_attention': False, 'tie_encoder_decoder': False, 'max_length': 20, 'min_length': 0, 'do_sample': False, 'early_stopping': False, 'num_beams': 1, 'num_beam_groups': 1, 'diversity_penalty': 0.0, 'temperature': 1.0, 'top_k': 50, 'top_p': 1.0, 'typical_p': 1.0, 'repetition_penalty': 1.0, 'length_penalty': 1.0, 'no_repeat_ngram_size': 0, 'encoder_no_repeat_ngram_size': 0, 'bad_words_ids': None, 'num_return_sequences': 1, 'output_scores': False, 'return_dict_in_generate': False, 'forced_bos_token_id': None, 'forced_eos_token_id': None, 'remove_invalid_values': False, 'exponential_decay_length_penalty': None, 'suppress_tokens': None, 'begin_suppress_tokens': None, 'architectures': ['FalconForCausalLM'], 'finetuning_task': None, 'id2label': {0: 'LABEL_0', 1: 'LABEL_1'}, 'label2id': {'LABEL_0': 0, 'LABEL_1': 1}, 'tokenizer_class': None, 'prefix': None, 'pad_token_id': None, 'sep_token_id': None, 'decoder_start_token_id': None, 'task_specific_params': None, 'problem_type': None, '_name_or_path': 'vilsonrodrigues/falcon-7b-instruct-sharded', 'transformers_version': '4.37.2', 'apply_residual_connection_post_layernorm': False, 'auto_map': {'AutoConfig': 'vilsonrodrigues/falcon-7b-instruct-sharded--configuration_falcon.FalconConfig', 'AutoModel': 'vilsonrodrigues/falcon-7b-instruct-sharded--modeling_falcon.FalconModel', 'AutoModelForSequenceClassification': 'vilsonrodrigues/falcon-7b-instruct-sharded--modeling_falcon.FalconForSequenceClassification', 'AutoModelForTokenClassification': 'vilsonrodrigues/falcon-7b-instruct-sharded--modeling_falcon.FalconForTokenClassification', 'AutoModelForQuestionAnswering': 'vilsonrodrigues/falcon-7b-instruct-sharded--modeling_falcon.FalconForQuestionAnswering', 'AutoModelForCausalLM': 'vilsonrodrigues/falcon-7b-instruct-sharded--modeling_falcon.FalconForCausalLM'}, 'model_type': 'falcon', 'quantization_config': {'quant_method': 'QuantizationMethod.BITS_AND_BYTES', 'load_in_8bit': False, 'load_in_4bit': True, 'llm_int8_threshold': 6.0, 'llm_int8_skip_modules': None, 'llm_int8_enable_fp32_cpu_offload': False, 'llm_int8_has_fp16_weight': False, 'bnb_4bit_quant_type': 'nf4', 'bnb_4bit_use_double_quant': True, 'bnb_4bit_compute_dtype': 'float16'}, 'output_dir': './', 'overwrite_output_dir': False, 'do_train': False, 'do_eval': False, 'do_predict': False, 'evaluation_strategy': 'no', 'prediction_loss_only': False, 'per_device_train_batch_size': 4, 'per_device_eval_batch_size': 8, 'per_gpu_train_batch_size': None, 'per_gpu_eval_batch_size': None, 'gradient_accumulation_steps': 4, 'eval_accumulation_steps': None, 'eval_delay': 0, 'learning_rate': 0.0002, 'weight_decay': 0.0, 'adam_beta1': 0.9, 'adam_beta2': 0.999, 'adam_epsilon': 1e-08, 'max_grad_norm': 0.3, 'num_train_epochs': 3.0, 'max_steps': 180, 'lr_scheduler_type': 'cosine', 'lr_scheduler_kwargs': {}, 'warmup_ratio': 0.03, 'warmup_steps': 0, 'log_level': 'passive', 'log_level_replica': 'warning', 'log_on_each_node': True, 'logging_dir': './runs/Feb14_11-53-30_f4b889916ade', 'logging_strategy': 'steps', 'logging_first_step': False, 'logging_steps': 10, 'logging_nan_inf_filter': True, 'save_strategy': 'steps', 'save_steps': 10, 'save_total_limit': None, 'save_safetensors': True, 'save_on_each_node': False, 'save_only_model': False, 'no_cuda': False, 'use_cpu': False, 'use_mps_device': False, 'seed': 42, 'data_seed': None, 'jit_mode_eval': False, 'use_ipex': False, 'bf16': False, 'fp16': True, 'fp16_opt_level': 'O1', 'half_precision_backend': 'auto', 'bf16_full_eval': False, 'fp16_full_eval': False, 'tf32': None, 'local_rank': 0, 'ddp_backend': None, 'tpu_num_cores': None, 'tpu_metrics_debug': False, 'debug': [], 'dataloader_drop_last': False, 'eval_steps': None, 'dataloader_num_workers': 0, 'past_index': -1, 'run_name': './', 'disable_tqdm': False, 'remove_unused_columns': True, 'label_names': None, 'load_best_model_at_end': False, 'metric_for_best_model': None, 'greater_is_better': None, 'ignore_data_skip': False, 'fsdp': [], 'fsdp_min_num_params': 0, 'fsdp_config': {'min_num_params': 0, 'xla': False, 'xla_fsdp_grad_ckpt': False}, 'fsdp_transformer_layer_cls_to_wrap': None, 'deepspeed': None, 'label_smoothing_factor': 0.0, 'optim': 'paged_adamw_32bit', 'optim_args': None, 'adafactor': False, 'group_by_length': True, 'length_column_name': 'length', 'report_to': ['tensorboard', 'wandb'], 'ddp_find_unused_parameters': None, 'ddp_bucket_cap_mb': None, 'ddp_broadcast_buffers': None, 'dataloader_pin_memory': True, 'dataloader_persistent_workers': False, 'skip_memory_metrics': True, 'use_legacy_prediction_loop': False, 'push_to_hub': True, 'resume_from_checkpoint': None, 'hub_model_id': None, 'hub_strategy': 'every_save', 'hub_token': '<HUB_TOKEN>', 'hub_private_repo': False, 'hub_always_push': False, 'gradient_checkpointing': False, 'gradient_checkpointing_kwargs': None, 'include_inputs_for_metrics': False, 'fp16_backend': 'auto', 'push_to_hub_model_id': None, 'push_to_hub_organization': None, 'push_to_hub_token': '<PUSH_TO_HUB_TOKEN>', 'mp_parameters': '', 'auto_find_batch_size': False, 'full_determinism': False, 'torchdynamo': None, 'ray_scope': 'last', 'ddp_timeout': 1800, 'torch_compile': False, 'torch_compile_backend': None, 'torch_compile_mode': None, 'dispatch_batches': None, 'split_batches': False, 'include_tokens_per_second': False, 'include_num_input_tokens_seen': False, 'neftune_noise_alpha': None}
 
 
 
 
 
 
 
 
 
 
 
 
 
 
28
  2024-02-14 11:54:08,936 INFO MainThread:4105 [wandb_run.py:_redirect():2186] Redirects installed.
29
  2024-02-14 11:54:08,937 INFO MainThread:4105 [wandb_init.py:init():847] run started, returning control to user process
30
  2024-02-14 11:54:08,940 INFO MainThread:4105 [wandb_run.py:_config_callback():1343] config_cb None None {'vocab_size': 65024, 'hidden_size': 4544, 'num_hidden_layers': 32, 'num_attention_heads': 71, 'layer_norm_epsilon': 1e-05, 'initializer_range': 0.02, 'use_cache': False, 'hidden_dropout': 0.0, 'attention_dropout': 0.0, 'bos_token_id': 11, 'eos_token_id': 11, 'num_kv_heads': 71, 'alibi': False, 'new_decoder_architecture': False, 'multi_query': True, 'parallel_attn': True, 'bias': False, 'return_dict': True, 'output_hidden_states': False, 'output_attentions': False, 'torchscript': False, 'torch_dtype': 'bfloat16', 'use_bfloat16': False, 'tf_legacy_loss': False, 'pruned_heads': {}, 'tie_word_embeddings': True, 'chunk_size_feed_forward': 0, 'is_encoder_decoder': False, 'is_decoder': False, 'cross_attention_hidden_size': None, 'add_cross_attention': False, 'tie_encoder_decoder': False, 'max_length': 20, 'min_length': 0, 'do_sample': False, 'early_stopping': False, 'num_beams': 1, 'num_beam_groups': 1, 'diversity_penalty': 0.0, 'temperature': 1.0, 'top_k': 50, 'top_p': 1.0, 'typical_p': 1.0, 'repetition_penalty': 1.0, 'length_penalty': 1.0, 'no_repeat_ngram_size': 0, 'encoder_no_repeat_ngram_size': 0, 'bad_words_ids': None, 'num_return_sequences': 1, 'output_scores': False, 'return_dict_in_generate': False, 'forced_bos_token_id': None, 'forced_eos_token_id': None, 'remove_invalid_values': False, 'exponential_decay_length_penalty': None, 'suppress_tokens': None, 'begin_suppress_tokens': None, 'architectures': ['FalconForCausalLM'], 'finetuning_task': None, 'id2label': {0: 'LABEL_0', 1: 'LABEL_1'}, 'label2id': {'LABEL_0': 0, 'LABEL_1': 1}, 'tokenizer_class': None, 'prefix': None, 'pad_token_id': None, 'sep_token_id': None, 'decoder_start_token_id': None, 'task_specific_params': None, 'problem_type': None, '_name_or_path': 'vilsonrodrigues/falcon-7b-instruct-sharded', 'transformers_version': '4.37.2', 'apply_residual_connection_post_layernorm': False, 'auto_map': {'AutoConfig': 'vilsonrodrigues/falcon-7b-instruct-sharded--configuration_falcon.FalconConfig', 'AutoModel': 'vilsonrodrigues/falcon-7b-instruct-sharded--modeling_falcon.FalconModel', 'AutoModelForSequenceClassification': 'vilsonrodrigues/falcon-7b-instruct-sharded--modeling_falcon.FalconForSequenceClassification', 'AutoModelForTokenClassification': 'vilsonrodrigues/falcon-7b-instruct-sharded--modeling_falcon.FalconForTokenClassification', 'AutoModelForQuestionAnswering': 'vilsonrodrigues/falcon-7b-instruct-sharded--modeling_falcon.FalconForQuestionAnswering', 'AutoModelForCausalLM': 'vilsonrodrigues/falcon-7b-instruct-sharded--modeling_falcon.FalconForCausalLM'}, 'model_type': 'falcon', 'quantization_config': {'quant_method': 'QuantizationMethod.BITS_AND_BYTES', 'load_in_8bit': False, 'load_in_4bit': True, 'llm_int8_threshold': 6.0, 'llm_int8_skip_modules': None, 'llm_int8_enable_fp32_cpu_offload': False, 'llm_int8_has_fp16_weight': False, 'bnb_4bit_quant_type': 'nf4', 'bnb_4bit_use_double_quant': True, 'bnb_4bit_compute_dtype': 'float16'}, 'output_dir': './', 'overwrite_output_dir': False, 'do_train': False, 'do_eval': False, 'do_predict': False, 'evaluation_strategy': 'no', 'prediction_loss_only': False, 'per_device_train_batch_size': 4, 'per_device_eval_batch_size': 8, 'per_gpu_train_batch_size': None, 'per_gpu_eval_batch_size': None, 'gradient_accumulation_steps': 4, 'eval_accumulation_steps': None, 'eval_delay': 0, 'learning_rate': 0.0002, 'weight_decay': 0.0, 'adam_beta1': 0.9, 'adam_beta2': 0.999, 'adam_epsilon': 1e-08, 'max_grad_norm': 0.3, 'num_train_epochs': 3.0, 'max_steps': 180, 'lr_scheduler_type': 'cosine', 'lr_scheduler_kwargs': {}, 'warmup_ratio': 0.03, 'warmup_steps': 0, 'log_level': 'passive', 'log_level_replica': 'warning', 'log_on_each_node': True, 'logging_dir': './runs/Feb14_11-53-30_f4b889916ade', 'logging_strategy': 'steps', 'logging_first_step': False, 'logging_steps': 10, 'logging_nan_inf_filter': True, 'save_strategy': 'steps', 'save_steps': 10, 'save_total_limit': None, 'save_safetensors': True, 'save_on_each_node': False, 'save_only_model': False, 'no_cuda': False, 'use_cpu': False, 'use_mps_device': False, 'seed': 42, 'data_seed': None, 'jit_mode_eval': False, 'use_ipex': False, 'bf16': False, 'fp16': True, 'fp16_opt_level': 'O1', 'half_precision_backend': 'auto', 'bf16_full_eval': False, 'fp16_full_eval': False, 'tf32': None, 'local_rank': 0, 'ddp_backend': None, 'tpu_num_cores': None, 'tpu_metrics_debug': False, 'debug': [], 'dataloader_drop_last': False, 'eval_steps': None, 'dataloader_num_workers': 0, 'past_index': -1, 'run_name': './', 'disable_tqdm': False, 'remove_unused_columns': True, 'label_names': None, 'load_best_model_at_end': False, 'metric_for_best_model': None, 'greater_is_better': None, 'ignore_data_skip': False, 'fsdp': [], 'fsdp_min_num_params': 0, 'fsdp_config': {'min_num_params': 0, 'xla': False, 'xla_fsdp_grad_ckpt': False}, 'fsdp_transformer_layer_cls_to_wrap': None, 'deepspeed': None, 'label_smoothing_factor': 0.0, 'optim': 'paged_adamw_32bit', 'optim_args': None, 'adafactor': False, 'group_by_length': True, 'length_column_name': 'length', 'report_to': ['tensorboard', 'wandb'], 'ddp_find_unused_parameters': None, 'ddp_bucket_cap_mb': None, 'ddp_broadcast_buffers': None, 'dataloader_pin_memory': True, 'dataloader_persistent_workers': False, 'skip_memory_metrics': True, 'use_legacy_prediction_loop': False, 'push_to_hub': True, 'resume_from_checkpoint': None, 'hub_model_id': None, 'hub_strategy': 'every_save', 'hub_token': '<HUB_TOKEN>', 'hub_private_repo': False, 'hub_always_push': False, 'gradient_checkpointing': False, 'gradient_checkpointing_kwargs': None, 'include_inputs_for_metrics': False, 'fp16_backend': 'auto', 'push_to_hub_model_id': None, 'push_to_hub_organization': None, 'push_to_hub_token': '<PUSH_TO_HUB_TOKEN>', 'mp_parameters': '', 'auto_find_batch_size': False, 'full_determinism': False, 'torchdynamo': None, 'ray_scope': 'last', 'ddp_timeout': 1800, 'torch_compile': False, 'torch_compile_backend': None, 'torch_compile_mode': None, 'dispatch_batches': None, 'split_batches': False, 'include_tokens_per_second': False, 'include_num_input_tokens_seen': False, 'neftune_noise_alpha': None}
31
+ 2024-02-14 12:22:52,244 INFO MainThread:4105 [jupyter.py:save_ipynb():373] not saving jupyter notebook
32
+ 2024-02-14 12:22:52,244 INFO MainThread:4105 [wandb_init.py:_pause_backend():437] pausing backend
33
+ 2024-02-14 12:49:31,960 INFO MainThread:4105 [wandb_init.py:_resume_backend():442] resuming backend
34
+ 2024-02-14 12:49:31,964 INFO MainThread:4105 [jupyter.py:save_ipynb():373] not saving jupyter notebook
35
+ 2024-02-14 12:49:31,964 INFO MainThread:4105 [wandb_init.py:_pause_backend():437] pausing backend
36
+ 2024-02-14 12:49:34,168 INFO MainThread:4105 [wandb_init.py:_resume_backend():442] resuming backend
37
+ 2024-02-14 12:49:34,432 INFO MainThread:4105 [jupyter.py:save_ipynb():373] not saving jupyter notebook
38
+ 2024-02-14 12:49:34,433 INFO MainThread:4105 [wandb_init.py:_pause_backend():437] pausing backend
39
+ 2024-02-14 12:49:36,951 INFO MainThread:4105 [wandb_init.py:_resume_backend():442] resuming backend
40
+ 2024-02-14 12:49:36,958 INFO MainThread:4105 [jupyter.py:save_ipynb():373] not saving jupyter notebook
41
+ 2024-02-14 12:49:36,958 INFO MainThread:4105 [wandb_init.py:_pause_backend():437] pausing backend
42
+ 2024-02-14 12:49:38,538 INFO MainThread:4105 [wandb_init.py:_resume_backend():442] resuming backend
43
+ 2024-02-14 12:49:39,001 INFO MainThread:4105 [wandb_run.py:_config_callback():1343] config_cb None None {'vocab_size': 65024, 'hidden_size': 4544, 'num_hidden_layers': 32, 'num_attention_heads': 71, 'layer_norm_epsilon': 1e-05, 'initializer_range': 0.02, 'use_cache': False, 'hidden_dropout': 0.0, 'attention_dropout': 0.0, 'bos_token_id': 11, 'eos_token_id': 11, 'num_kv_heads': 71, 'alibi': False, 'new_decoder_architecture': False, 'multi_query': True, 'parallel_attn': True, 'bias': False, 'return_dict': True, 'output_hidden_states': False, 'output_attentions': False, 'torchscript': False, 'torch_dtype': 'bfloat16', 'use_bfloat16': False, 'tf_legacy_loss': False, 'pruned_heads': {}, 'tie_word_embeddings': True, 'chunk_size_feed_forward': 0, 'is_encoder_decoder': False, 'is_decoder': False, 'cross_attention_hidden_size': None, 'add_cross_attention': False, 'tie_encoder_decoder': False, 'max_length': 20, 'min_length': 0, 'do_sample': False, 'early_stopping': False, 'num_beams': 1, 'num_beam_groups': 1, 'diversity_penalty': 0.0, 'temperature': 1.0, 'top_k': 50, 'top_p': 1.0, 'typical_p': 1.0, 'repetition_penalty': 1.0, 'length_penalty': 1.0, 'no_repeat_ngram_size': 0, 'encoder_no_repeat_ngram_size': 0, 'bad_words_ids': None, 'num_return_sequences': 1, 'output_scores': False, 'return_dict_in_generate': False, 'forced_bos_token_id': None, 'forced_eos_token_id': None, 'remove_invalid_values': False, 'exponential_decay_length_penalty': None, 'suppress_tokens': None, 'begin_suppress_tokens': None, 'architectures': ['FalconForCausalLM'], 'finetuning_task': None, 'id2label': {0: 'LABEL_0', 1: 'LABEL_1'}, 'label2id': {'LABEL_0': 0, 'LABEL_1': 1}, 'tokenizer_class': None, 'prefix': None, 'pad_token_id': None, 'sep_token_id': None, 'decoder_start_token_id': None, 'task_specific_params': None, 'problem_type': None, '_name_or_path': 'vilsonrodrigues/falcon-7b-instruct-sharded', 'transformers_version': '4.37.2', 'apply_residual_connection_post_layernorm': False, 'auto_map': {'AutoConfig': 'vilsonrodrigues/falcon-7b-instruct-sharded--configuration_falcon.FalconConfig', 'AutoModel': 'vilsonrodrigues/falcon-7b-instruct-sharded--modeling_falcon.FalconModel', 'AutoModelForSequenceClassification': 'vilsonrodrigues/falcon-7b-instruct-sharded--modeling_falcon.FalconForSequenceClassification', 'AutoModelForTokenClassification': 'vilsonrodrigues/falcon-7b-instruct-sharded--modeling_falcon.FalconForTokenClassification', 'AutoModelForQuestionAnswering': 'vilsonrodrigues/falcon-7b-instruct-sharded--modeling_falcon.FalconForQuestionAnswering', 'AutoModelForCausalLM': 'vilsonrodrigues/falcon-7b-instruct-sharded--modeling_falcon.FalconForCausalLM'}, 'model_type': 'falcon', 'quantization_config': {'quant_method': 'QuantizationMethod.BITS_AND_BYTES', 'load_in_8bit': False, 'load_in_4bit': True, 'llm_int8_threshold': 6.0, 'llm_int8_skip_modules': None, 'llm_int8_enable_fp32_cpu_offload': False, 'llm_int8_has_fp16_weight': False, 'bnb_4bit_quant_type': 'nf4', 'bnb_4bit_use_double_quant': True, 'bnb_4bit_compute_dtype': 'float16'}, 'output_dir': './', 'overwrite_output_dir': False, 'do_train': False, 'do_eval': False, 'do_predict': False, 'evaluation_strategy': 'no', 'prediction_loss_only': False, 'per_device_train_batch_size': 4, 'per_device_eval_batch_size': 8, 'per_gpu_train_batch_size': None, 'per_gpu_eval_batch_size': None, 'gradient_accumulation_steps': 4, 'eval_accumulation_steps': None, 'eval_delay': 0, 'learning_rate': 0.0002, 'weight_decay': 0.0, 'adam_beta1': 0.9, 'adam_beta2': 0.999, 'adam_epsilon': 1e-08, 'max_grad_norm': 0.3, 'num_train_epochs': 3.0, 'max_steps': 250, 'lr_scheduler_type': 'cosine', 'lr_scheduler_kwargs': {}, 'warmup_ratio': 0.03, 'warmup_steps': 0, 'log_level': 'passive', 'log_level_replica': 'warning', 'log_on_each_node': True, 'logging_dir': './runs/Feb14_12-49-31_f4b889916ade', 'logging_strategy': 'steps', 'logging_first_step': False, 'logging_steps': 10, 'logging_nan_inf_filter': True, 'save_strategy': 'steps', 'save_steps': 10, 'save_total_limit': None, 'save_safetensors': True, 'save_on_each_node': False, 'save_only_model': False, 'no_cuda': False, 'use_cpu': False, 'use_mps_device': False, 'seed': 42, 'data_seed': None, 'jit_mode_eval': False, 'use_ipex': False, 'bf16': False, 'fp16': True, 'fp16_opt_level': 'O1', 'half_precision_backend': 'auto', 'bf16_full_eval': False, 'fp16_full_eval': False, 'tf32': None, 'local_rank': 0, 'ddp_backend': None, 'tpu_num_cores': None, 'tpu_metrics_debug': False, 'debug': [], 'dataloader_drop_last': False, 'eval_steps': None, 'dataloader_num_workers': 0, 'past_index': -1, 'run_name': './', 'disable_tqdm': False, 'remove_unused_columns': True, 'label_names': None, 'load_best_model_at_end': False, 'metric_for_best_model': None, 'greater_is_better': None, 'ignore_data_skip': False, 'fsdp': [], 'fsdp_min_num_params': 0, 'fsdp_config': {'min_num_params': 0, 'xla': False, 'xla_fsdp_grad_ckpt': False}, 'fsdp_transformer_layer_cls_to_wrap': None, 'deepspeed': None, 'label_smoothing_factor': 0.0, 'optim': 'paged_adamw_32bit', 'optim_args': None, 'adafactor': False, 'group_by_length': True, 'length_column_name': 'length', 'report_to': ['tensorboard', 'wandb'], 'ddp_find_unused_parameters': None, 'ddp_bucket_cap_mb': None, 'ddp_broadcast_buffers': None, 'dataloader_pin_memory': True, 'dataloader_persistent_workers': False, 'skip_memory_metrics': True, 'use_legacy_prediction_loop': False, 'push_to_hub': True, 'resume_from_checkpoint': None, 'hub_model_id': None, 'hub_strategy': 'every_save', 'hub_token': '<HUB_TOKEN>', 'hub_private_repo': False, 'hub_always_push': False, 'gradient_checkpointing': False, 'gradient_checkpointing_kwargs': None, 'include_inputs_for_metrics': False, 'fp16_backend': 'auto', 'push_to_hub_model_id': None, 'push_to_hub_organization': None, 'push_to_hub_token': '<PUSH_TO_HUB_TOKEN>', 'mp_parameters': '', 'auto_find_batch_size': False, 'full_determinism': False, 'torchdynamo': None, 'ray_scope': 'last', 'ddp_timeout': 1800, 'torch_compile': False, 'torch_compile_backend': None, 'torch_compile_mode': None, 'dispatch_batches': None, 'split_batches': False, 'include_tokens_per_second': False, 'include_num_input_tokens_seen': False, 'neftune_noise_alpha': None}
wandb/run-20240214_115405-518dheci/files/config.yaml CHANGED
@@ -68,6 +68,26 @@ _wandb:
68
  5: 1
69
  6:
70
  - 1
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
71
  vocab_size:
72
  desc: null
73
  value: 65024
@@ -376,7 +396,7 @@ num_train_epochs:
376
  value: 3.0
377
  max_steps:
378
  desc: null
379
- value: 180
380
  lr_scheduler_type:
381
  desc: null
382
  value: cosine
@@ -400,7 +420,7 @@ log_on_each_node:
400
  value: true
401
  logging_dir:
402
  desc: null
403
- value: ./runs/Feb14_11-53-30_f4b889916ade
404
  logging_strategy:
405
  desc: null
406
  value: steps
 
68
  5: 1
69
  6:
70
  - 1
71
+ - 1: train/train_runtime
72
+ 5: 1
73
+ 6:
74
+ - 1
75
+ - 1: train/train_samples_per_second
76
+ 5: 1
77
+ 6:
78
+ - 1
79
+ - 1: train/train_steps_per_second
80
+ 5: 1
81
+ 6:
82
+ - 1
83
+ - 1: train/total_flos
84
+ 5: 1
85
+ 6:
86
+ - 1
87
+ - 1: train/train_loss
88
+ 5: 1
89
+ 6:
90
+ - 1
91
  vocab_size:
92
  desc: null
93
  value: 65024
 
396
  value: 3.0
397
  max_steps:
398
  desc: null
399
+ value: 250
400
  lr_scheduler_type:
401
  desc: null
402
  value: cosine
 
420
  value: true
421
  logging_dir:
422
  desc: null
423
+ value: ./runs/Feb14_12-49-31_f4b889916ade
424
  logging_strategy:
425
  desc: null
426
  value: steps
wandb/run-20240214_115405-518dheci/files/output.log ADDED
@@ -0,0 +1 @@
 
 
1
+
wandb/run-20240214_115405-518dheci/files/wandb-summary.json CHANGED
@@ -1 +1 @@
1
- {"train/loss": 0.1103, "train/learning_rate": 0.0, "train/epoch": 14.4, "train/global_step": 180, "_timestamp": 1707913366.0455978, "_runtime": 1720.5264468193054, "_step": 18, "train/train_runtime": 1744.0643, "train/train_samples_per_second": 1.651, "train/train_steps_per_second": 0.103, "train/total_flos": 1.1953245987458304e+16, "train/train_loss": 0.36850863695144653}
 
1
+ {"train/loss": 0.1494, "train/learning_rate": 0.00019996629653035126, "train/epoch": 0.8, "train/global_step": 10, "_timestamp": 1707915057.2341144, "_runtime": 3411.7149634361267, "_step": 19, "train/train_runtime": 1744.0643, "train/train_samples_per_second": 1.651, "train/train_steps_per_second": 0.103, "train/total_flos": 1.1953245987458304e+16, "train/train_loss": 0.36850863695144653}
wandb/run-20240214_115405-518dheci/logs/debug-internal.log CHANGED
@@ -879,3 +879,770 @@
879
  2024-02-14 12:22:46,049 INFO SenderThread:19614 [sender.py:_save_file():1403] saving file wandb-summary.json with policy end
880
  2024-02-14 12:22:46,293 INFO Thread-12 :19614 [dir_watcher.py:_on_file_modified():288] file/dir modified: /content/wandb/run-20240214_115405-518dheci/files/wandb-summary.json
881
  2024-02-14 12:22:51,050 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
879
  2024-02-14 12:22:46,049 INFO SenderThread:19614 [sender.py:_save_file():1403] saving file wandb-summary.json with policy end
880
  2024-02-14 12:22:46,293 INFO Thread-12 :19614 [dir_watcher.py:_on_file_modified():288] file/dir modified: /content/wandb/run-20240214_115405-518dheci/files/wandb-summary.json
881
  2024-02-14 12:22:51,050 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
882
+ 2024-02-14 12:22:52,245 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: pause
883
+ 2024-02-14 12:22:52,245 INFO HandlerThread:19614 [handler.py:handle_request_pause():708] stopping system metrics thread
884
+ 2024-02-14 12:22:52,245 INFO HandlerThread:19614 [system_monitor.py:finish():203] Stopping system monitor
885
+ 2024-02-14 12:22:52,245 DEBUG SystemMonitor:19614 [system_monitor.py:_start():179] Finished system metrics aggregation loop
886
+ 2024-02-14 12:22:52,246 DEBUG SystemMonitor:19614 [system_monitor.py:_start():183] Publishing last batch of metrics
887
+ 2024-02-14 12:22:52,246 INFO HandlerThread:19614 [interfaces.py:finish():202] Joined cpu monitor
888
+ 2024-02-14 12:22:52,247 INFO HandlerThread:19614 [interfaces.py:finish():202] Joined disk monitor
889
+ 2024-02-14 12:22:52,254 INFO HandlerThread:19614 [interfaces.py:finish():202] Joined gpu monitor
890
+ 2024-02-14 12:22:52,254 INFO HandlerThread:19614 [interfaces.py:finish():202] Joined memory monitor
891
+ 2024-02-14 12:22:52,254 INFO HandlerThread:19614 [interfaces.py:finish():202] Joined network monitor
892
+ 2024-02-14 12:22:52,255 DEBUG SenderThread:19614 [sender.py:send():382] send: stats
893
+ 2024-02-14 12:22:54,094 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
894
+ 2024-02-14 12:22:54,094 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
895
+ 2024-02-14 12:22:54,094 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
896
+ 2024-02-14 12:22:56,215 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
897
+ 2024-02-14 12:23:01,216 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
898
+ 2024-02-14 12:23:06,221 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
899
+ 2024-02-14 12:23:07,323 INFO Thread-12 :19614 [dir_watcher.py:_on_file_modified():288] file/dir modified: /content/wandb/run-20240214_115405-518dheci/files/config.yaml
900
+ 2024-02-14 12:23:09,094 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
901
+ 2024-02-14 12:23:09,094 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
902
+ 2024-02-14 12:23:09,095 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
903
+ 2024-02-14 12:23:12,274 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
904
+ 2024-02-14 12:23:17,275 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
905
+ 2024-02-14 12:23:22,276 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
906
+ 2024-02-14 12:23:24,094 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
907
+ 2024-02-14 12:23:24,095 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
908
+ 2024-02-14 12:23:24,095 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
909
+ 2024-02-14 12:23:28,181 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
910
+ 2024-02-14 12:23:33,182 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
911
+ 2024-02-14 12:23:38,184 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
912
+ 2024-02-14 12:23:39,095 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
913
+ 2024-02-14 12:23:39,095 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
914
+ 2024-02-14 12:23:39,095 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
915
+ 2024-02-14 12:23:44,165 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
916
+ 2024-02-14 12:23:49,166 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
917
+ 2024-02-14 12:23:54,095 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
918
+ 2024-02-14 12:23:54,095 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
919
+ 2024-02-14 12:23:54,095 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
920
+ 2024-02-14 12:23:54,249 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
921
+ 2024-02-14 12:23:59,250 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
922
+ 2024-02-14 12:24:04,251 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
923
+ 2024-02-14 12:24:09,095 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
924
+ 2024-02-14 12:24:09,095 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
925
+ 2024-02-14 12:24:09,096 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
926
+ 2024-02-14 12:24:10,244 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
927
+ 2024-02-14 12:24:15,244 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
928
+ 2024-02-14 12:24:20,245 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
929
+ 2024-02-14 12:24:24,095 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
930
+ 2024-02-14 12:24:24,095 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
931
+ 2024-02-14 12:24:24,096 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
932
+ 2024-02-14 12:24:26,189 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
933
+ 2024-02-14 12:24:31,190 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
934
+ 2024-02-14 12:24:36,191 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
935
+ 2024-02-14 12:24:39,095 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
936
+ 2024-02-14 12:24:39,096 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
937
+ 2024-02-14 12:24:39,096 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
938
+ 2024-02-14 12:24:41,200 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
939
+ 2024-02-14 12:24:46,201 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
940
+ 2024-02-14 12:24:51,202 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
941
+ 2024-02-14 12:24:54,095 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
942
+ 2024-02-14 12:24:54,096 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
943
+ 2024-02-14 12:24:54,096 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
944
+ 2024-02-14 12:24:57,166 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
945
+ 2024-02-14 12:25:02,167 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
946
+ 2024-02-14 12:25:07,168 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
947
+ 2024-02-14 12:25:09,096 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
948
+ 2024-02-14 12:25:09,096 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
949
+ 2024-02-14 12:25:09,096 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
950
+ 2024-02-14 12:25:12,201 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
951
+ 2024-02-14 12:25:17,202 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
952
+ 2024-02-14 12:25:22,203 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
953
+ 2024-02-14 12:25:24,096 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
954
+ 2024-02-14 12:25:24,096 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
955
+ 2024-02-14 12:25:24,097 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
956
+ 2024-02-14 12:25:28,169 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
957
+ 2024-02-14 12:25:33,170 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
958
+ 2024-02-14 12:25:38,171 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
959
+ 2024-02-14 12:25:39,096 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
960
+ 2024-02-14 12:25:39,096 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
961
+ 2024-02-14 12:25:39,097 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
962
+ 2024-02-14 12:25:43,248 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
963
+ 2024-02-14 12:25:48,249 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
964
+ 2024-02-14 12:25:53,250 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
965
+ 2024-02-14 12:25:54,096 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
966
+ 2024-02-14 12:25:54,096 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
967
+ 2024-02-14 12:25:54,097 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
968
+ 2024-02-14 12:25:59,197 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
969
+ 2024-02-14 12:26:04,198 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
970
+ 2024-02-14 12:26:09,096 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
971
+ 2024-02-14 12:26:09,096 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
972
+ 2024-02-14 12:26:09,097 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
973
+ 2024-02-14 12:26:09,208 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
974
+ 2024-02-14 12:26:14,209 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
975
+ 2024-02-14 12:26:19,210 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
976
+ 2024-02-14 12:26:24,096 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
977
+ 2024-02-14 12:26:24,097 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
978
+ 2024-02-14 12:26:24,097 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
979
+ 2024-02-14 12:26:24,244 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
980
+ 2024-02-14 12:26:29,245 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
981
+ 2024-02-14 12:26:34,246 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
982
+ 2024-02-14 12:26:39,096 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
983
+ 2024-02-14 12:26:39,097 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
984
+ 2024-02-14 12:26:39,097 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
985
+ 2024-02-14 12:26:40,214 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
986
+ 2024-02-14 12:26:45,215 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
987
+ 2024-02-14 12:26:50,216 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
988
+ 2024-02-14 12:26:54,097 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
989
+ 2024-02-14 12:26:54,097 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
990
+ 2024-02-14 12:26:54,097 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
991
+ 2024-02-14 12:26:55,237 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
992
+ 2024-02-14 12:27:00,238 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
993
+ 2024-02-14 12:27:05,239 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
994
+ 2024-02-14 12:27:09,097 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
995
+ 2024-02-14 12:27:09,097 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
996
+ 2024-02-14 12:27:09,098 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
997
+ 2024-02-14 12:27:11,238 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
998
+ 2024-02-14 12:27:16,239 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
999
+ 2024-02-14 12:27:21,240 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1000
+ 2024-02-14 12:27:24,097 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1001
+ 2024-02-14 12:27:24,097 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1002
+ 2024-02-14 12:27:24,098 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1003
+ 2024-02-14 12:27:27,154 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1004
+ 2024-02-14 12:27:32,155 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1005
+ 2024-02-14 12:27:37,156 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1006
+ 2024-02-14 12:27:39,097 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1007
+ 2024-02-14 12:27:39,097 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1008
+ 2024-02-14 12:27:39,098 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1009
+ 2024-02-14 12:27:42,263 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1010
+ 2024-02-14 12:27:47,264 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1011
+ 2024-02-14 12:27:52,265 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1012
+ 2024-02-14 12:27:54,097 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1013
+ 2024-02-14 12:27:54,097 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1014
+ 2024-02-14 12:27:54,098 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1015
+ 2024-02-14 12:27:58,182 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1016
+ 2024-02-14 12:28:03,183 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1017
+ 2024-02-14 12:28:08,184 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1018
+ 2024-02-14 12:28:09,097 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1019
+ 2024-02-14 12:28:09,098 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1020
+ 2024-02-14 12:28:09,098 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1021
+ 2024-02-14 12:28:13,242 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1022
+ 2024-02-14 12:28:18,243 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1023
+ 2024-02-14 12:28:23,244 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1024
+ 2024-02-14 12:28:24,098 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1025
+ 2024-02-14 12:28:24,098 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1026
+ 2024-02-14 12:28:24,098 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1027
+ 2024-02-14 12:28:28,254 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1028
+ 2024-02-14 12:28:33,255 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1029
+ 2024-02-14 12:28:38,256 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1030
+ 2024-02-14 12:28:39,098 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1031
+ 2024-02-14 12:28:39,098 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1032
+ 2024-02-14 12:28:39,099 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1033
+ 2024-02-14 12:28:44,249 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1034
+ 2024-02-14 12:28:49,250 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1035
+ 2024-02-14 12:28:54,098 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1036
+ 2024-02-14 12:28:54,098 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1037
+ 2024-02-14 12:28:54,099 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1038
+ 2024-02-14 12:28:55,242 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1039
+ 2024-02-14 12:29:00,243 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1040
+ 2024-02-14 12:29:05,244 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1041
+ 2024-02-14 12:29:09,098 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1042
+ 2024-02-14 12:29:09,099 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1043
+ 2024-02-14 12:29:09,099 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1044
+ 2024-02-14 12:29:10,249 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1045
+ 2024-02-14 12:29:15,250 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1046
+ 2024-02-14 12:29:20,251 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1047
+ 2024-02-14 12:29:24,098 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1048
+ 2024-02-14 12:29:24,099 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1049
+ 2024-02-14 12:29:24,099 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1050
+ 2024-02-14 12:29:26,154 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1051
+ 2024-02-14 12:29:31,155 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1052
+ 2024-02-14 12:29:36,156 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1053
+ 2024-02-14 12:29:39,099 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1054
+ 2024-02-14 12:29:39,099 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1055
+ 2024-02-14 12:29:39,099 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1056
+ 2024-02-14 12:29:41,204 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1057
+ 2024-02-14 12:29:46,205 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1058
+ 2024-02-14 12:29:51,206 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1059
+ 2024-02-14 12:29:54,099 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1060
+ 2024-02-14 12:29:54,099 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1061
+ 2024-02-14 12:29:54,099 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1062
+ 2024-02-14 12:29:56,225 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1063
+ 2024-02-14 12:30:01,226 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1064
+ 2024-02-14 12:30:06,228 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1065
+ 2024-02-14 12:30:09,099 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1066
+ 2024-02-14 12:30:09,099 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1067
+ 2024-02-14 12:30:09,099 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1068
+ 2024-02-14 12:30:12,210 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1069
+ 2024-02-14 12:30:17,211 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1070
+ 2024-02-14 12:30:22,212 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1071
+ 2024-02-14 12:30:24,099 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1072
+ 2024-02-14 12:30:24,099 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1073
+ 2024-02-14 12:30:24,100 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1074
+ 2024-02-14 12:30:28,192 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1075
+ 2024-02-14 12:30:33,193 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1076
+ 2024-02-14 12:30:38,194 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1077
+ 2024-02-14 12:30:39,099 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1078
+ 2024-02-14 12:30:39,099 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1079
+ 2024-02-14 12:30:39,100 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1080
+ 2024-02-14 12:30:44,173 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1081
+ 2024-02-14 12:30:49,173 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1082
+ 2024-02-14 12:30:54,099 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1083
+ 2024-02-14 12:30:54,100 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1084
+ 2024-02-14 12:30:54,100 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1085
+ 2024-02-14 12:30:54,213 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1086
+ 2024-02-14 12:30:59,214 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1087
+ 2024-02-14 12:31:04,215 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1088
+ 2024-02-14 12:31:09,100 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1089
+ 2024-02-14 12:31:09,100 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1090
+ 2024-02-14 12:31:09,100 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1091
+ 2024-02-14 12:31:10,185 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1092
+ 2024-02-14 12:31:15,186 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1093
+ 2024-02-14 12:31:20,187 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1094
+ 2024-02-14 12:31:24,100 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1095
+ 2024-02-14 12:31:24,100 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1096
+ 2024-02-14 12:31:24,100 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1097
+ 2024-02-14 12:31:26,184 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1098
+ 2024-02-14 12:31:31,185 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1099
+ 2024-02-14 12:31:36,186 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1100
+ 2024-02-14 12:31:39,100 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1101
+ 2024-02-14 12:31:39,100 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1102
+ 2024-02-14 12:31:39,101 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1103
+ 2024-02-14 12:31:41,236 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1104
+ 2024-02-14 12:31:46,237 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1105
+ 2024-02-14 12:31:51,238 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1106
+ 2024-02-14 12:31:54,100 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1107
+ 2024-02-14 12:31:54,100 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1108
+ 2024-02-14 12:31:54,101 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1109
+ 2024-02-14 12:31:57,220 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1110
+ 2024-02-14 12:32:02,221 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1111
+ 2024-02-14 12:32:07,222 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1112
+ 2024-02-14 12:32:09,100 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1113
+ 2024-02-14 12:32:09,100 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1114
+ 2024-02-14 12:32:09,101 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1115
+ 2024-02-14 12:32:13,211 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1116
+ 2024-02-14 12:32:18,211 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1117
+ 2024-02-14 12:32:23,213 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1118
+ 2024-02-14 12:32:24,100 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1119
+ 2024-02-14 12:32:24,101 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1120
+ 2024-02-14 12:32:24,101 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1121
+ 2024-02-14 12:32:28,225 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1122
+ 2024-02-14 12:32:33,226 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1123
+ 2024-02-14 12:32:38,227 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1124
+ 2024-02-14 12:32:39,101 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1125
+ 2024-02-14 12:32:39,101 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1126
+ 2024-02-14 12:32:39,101 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1127
+ 2024-02-14 12:32:44,181 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1128
+ 2024-02-14 12:32:49,182 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1129
+ 2024-02-14 12:32:54,101 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1130
+ 2024-02-14 12:32:54,101 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1131
+ 2024-02-14 12:32:54,101 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1132
+ 2024-02-14 12:32:55,172 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1133
+ 2024-02-14 12:33:00,174 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1134
+ 2024-02-14 12:33:05,174 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1135
+ 2024-02-14 12:33:09,101 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1136
+ 2024-02-14 12:33:09,101 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1137
+ 2024-02-14 12:33:09,101 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1138
+ 2024-02-14 12:33:10,187 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1139
+ 2024-02-14 12:33:15,188 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1140
+ 2024-02-14 12:33:20,189 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1141
+ 2024-02-14 12:33:24,101 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1142
+ 2024-02-14 12:33:24,101 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1143
+ 2024-02-14 12:33:24,102 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1144
+ 2024-02-14 12:33:25,231 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1145
+ 2024-02-14 12:33:30,232 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1146
+ 2024-02-14 12:33:35,233 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1147
+ 2024-02-14 12:33:39,101 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1148
+ 2024-02-14 12:33:39,101 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1149
+ 2024-02-14 12:33:39,102 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1150
+ 2024-02-14 12:33:40,253 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1151
+ 2024-02-14 12:33:45,253 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1152
+ 2024-02-14 12:33:50,255 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1153
+ 2024-02-14 12:33:54,102 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1154
+ 2024-02-14 12:33:54,102 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1155
+ 2024-02-14 12:33:54,102 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1156
+ 2024-02-14 12:33:56,233 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1157
+ 2024-02-14 12:34:01,234 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1158
+ 2024-02-14 12:34:06,235 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1159
+ 2024-02-14 12:34:09,102 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1160
+ 2024-02-14 12:34:09,102 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1161
+ 2024-02-14 12:34:09,102 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1162
+ 2024-02-14 12:34:12,195 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1163
+ 2024-02-14 12:34:17,196 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1164
+ 2024-02-14 12:34:22,197 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1165
+ 2024-02-14 12:34:24,102 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1166
+ 2024-02-14 12:34:24,102 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1167
+ 2024-02-14 12:34:24,103 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1168
+ 2024-02-14 12:34:28,182 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1169
+ 2024-02-14 12:34:33,183 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1170
+ 2024-02-14 12:34:38,184 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1171
+ 2024-02-14 12:34:39,102 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1172
+ 2024-02-14 12:34:39,102 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1173
+ 2024-02-14 12:34:39,102 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1174
+ 2024-02-14 12:34:43,253 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1175
+ 2024-02-14 12:34:48,254 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1176
+ 2024-02-14 12:34:53,255 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1177
+ 2024-02-14 12:34:54,102 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1178
+ 2024-02-14 12:34:54,102 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1179
+ 2024-02-14 12:34:54,103 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1180
+ 2024-02-14 12:34:59,254 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1181
+ 2024-02-14 12:35:04,255 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1182
+ 2024-02-14 12:35:09,102 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1183
+ 2024-02-14 12:35:09,102 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1184
+ 2024-02-14 12:35:09,103 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1185
+ 2024-02-14 12:35:09,257 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1186
+ 2024-02-14 12:35:14,258 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1187
+ 2024-02-14 12:35:19,259 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1188
+ 2024-02-14 12:35:24,103 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1189
+ 2024-02-14 12:35:24,103 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1190
+ 2024-02-14 12:35:24,103 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1191
+ 2024-02-14 12:35:25,173 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1192
+ 2024-02-14 12:35:30,174 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1193
+ 2024-02-14 12:35:35,175 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1194
+ 2024-02-14 12:35:39,103 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1195
+ 2024-02-14 12:35:39,103 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1196
+ 2024-02-14 12:35:39,103 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1197
+ 2024-02-14 12:35:40,269 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1198
+ 2024-02-14 12:35:45,270 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1199
+ 2024-02-14 12:35:50,271 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1200
+ 2024-02-14 12:35:54,103 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1201
+ 2024-02-14 12:35:54,103 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1202
+ 2024-02-14 12:35:54,103 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1203
+ 2024-02-14 12:35:56,219 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1204
+ 2024-02-14 12:36:01,220 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1205
+ 2024-02-14 12:36:06,221 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1206
+ 2024-02-14 12:36:09,103 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1207
+ 2024-02-14 12:36:09,103 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1208
+ 2024-02-14 12:36:09,104 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1209
+ 2024-02-14 12:36:12,201 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1210
+ 2024-02-14 12:36:17,203 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1211
+ 2024-02-14 12:36:22,204 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1212
+ 2024-02-14 12:36:24,103 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1213
+ 2024-02-14 12:36:24,103 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1214
+ 2024-02-14 12:36:24,104 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1215
+ 2024-02-14 12:36:28,184 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1216
+ 2024-02-14 12:36:33,185 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1217
+ 2024-02-14 12:36:38,186 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1218
+ 2024-02-14 12:36:39,103 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1219
+ 2024-02-14 12:36:39,103 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1220
+ 2024-02-14 12:36:39,104 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1221
+ 2024-02-14 12:36:43,271 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1222
+ 2024-02-14 12:36:48,273 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1223
+ 2024-02-14 12:36:53,274 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1224
+ 2024-02-14 12:36:54,103 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1225
+ 2024-02-14 12:36:54,104 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1226
+ 2024-02-14 12:36:54,144 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1227
+ 2024-02-14 12:36:59,198 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1228
+ 2024-02-14 12:37:04,199 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1229
+ 2024-02-14 12:37:09,104 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1230
+ 2024-02-14 12:37:09,104 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1231
+ 2024-02-14 12:37:09,104 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1232
+ 2024-02-14 12:37:10,195 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1233
+ 2024-02-14 12:37:15,196 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1234
+ 2024-02-14 12:37:20,197 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1235
+ 2024-02-14 12:37:24,104 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1236
+ 2024-02-14 12:37:24,104 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1237
+ 2024-02-14 12:37:24,104 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1238
+ 2024-02-14 12:37:25,242 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1239
+ 2024-02-14 12:37:30,243 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1240
+ 2024-02-14 12:37:35,244 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1241
+ 2024-02-14 12:37:39,104 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1242
+ 2024-02-14 12:37:39,104 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1243
+ 2024-02-14 12:37:39,105 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1244
+ 2024-02-14 12:37:40,265 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1245
+ 2024-02-14 12:37:45,266 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1246
+ 2024-02-14 12:37:50,267 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1247
+ 2024-02-14 12:37:54,104 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1248
+ 2024-02-14 12:37:54,104 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1249
+ 2024-02-14 12:37:54,105 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1250
+ 2024-02-14 12:37:56,254 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1251
+ 2024-02-14 12:38:01,255 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1252
+ 2024-02-14 12:38:06,256 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1253
+ 2024-02-14 12:38:09,104 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1254
+ 2024-02-14 12:38:09,104 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1255
+ 2024-02-14 12:38:09,105 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1256
+ 2024-02-14 12:38:12,243 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1257
+ 2024-02-14 12:38:17,245 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1258
+ 2024-02-14 12:38:22,246 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1259
+ 2024-02-14 12:38:24,105 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1260
+ 2024-02-14 12:38:24,105 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1261
+ 2024-02-14 12:38:24,105 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1262
+ 2024-02-14 12:38:28,193 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1263
+ 2024-02-14 12:38:33,194 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1264
+ 2024-02-14 12:38:38,195 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1265
+ 2024-02-14 12:38:39,105 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1266
+ 2024-02-14 12:38:39,105 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1267
+ 2024-02-14 12:38:39,105 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1268
+ 2024-02-14 12:38:44,193 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1269
+ 2024-02-14 12:38:49,194 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1270
+ 2024-02-14 12:38:54,105 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1271
+ 2024-02-14 12:38:54,105 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1272
+ 2024-02-14 12:38:54,106 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1273
+ 2024-02-14 12:38:54,236 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1274
+ 2024-02-14 12:38:59,237 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1275
+ 2024-02-14 12:39:04,238 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1276
+ 2024-02-14 12:39:09,105 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1277
+ 2024-02-14 12:39:09,105 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1278
+ 2024-02-14 12:39:09,106 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1279
+ 2024-02-14 12:39:09,255 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1280
+ 2024-02-14 12:39:14,256 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1281
+ 2024-02-14 12:39:19,257 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1282
+ 2024-02-14 12:39:24,105 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1283
+ 2024-02-14 12:39:24,105 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1284
+ 2024-02-14 12:39:24,106 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1285
+ 2024-02-14 12:39:25,216 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1286
+ 2024-02-14 12:39:30,217 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1287
+ 2024-02-14 12:39:35,218 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1288
+ 2024-02-14 12:39:39,105 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1289
+ 2024-02-14 12:39:39,106 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1290
+ 2024-02-14 12:39:39,106 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1291
+ 2024-02-14 12:39:40,256 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1292
+ 2024-02-14 12:39:45,257 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1293
+ 2024-02-14 12:39:50,258 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1294
+ 2024-02-14 12:39:54,105 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1295
+ 2024-02-14 12:39:54,106 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1296
+ 2024-02-14 12:39:54,106 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1297
+ 2024-02-14 12:39:56,174 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1298
+ 2024-02-14 12:40:01,175 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1299
+ 2024-02-14 12:40:06,176 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1300
+ 2024-02-14 12:40:09,105 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1301
+ 2024-02-14 12:40:09,106 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1302
+ 2024-02-14 12:40:09,106 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1303
+ 2024-02-14 12:40:12,172 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1304
+ 2024-02-14 12:40:17,173 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1305
+ 2024-02-14 12:40:22,174 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1306
+ 2024-02-14 12:40:24,105 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1307
+ 2024-02-14 12:40:24,106 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1308
+ 2024-02-14 12:40:24,106 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1309
+ 2024-02-14 12:40:27,185 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1310
+ 2024-02-14 12:40:32,186 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1311
+ 2024-02-14 12:40:37,187 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1312
+ 2024-02-14 12:40:39,106 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1313
+ 2024-02-14 12:40:39,106 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1314
+ 2024-02-14 12:40:39,106 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1315
+ 2024-02-14 12:40:42,258 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1316
+ 2024-02-14 12:40:47,259 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1317
+ 2024-02-14 12:40:52,260 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1318
+ 2024-02-14 12:40:54,106 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1319
+ 2024-02-14 12:40:54,106 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1320
+ 2024-02-14 12:40:54,107 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1321
+ 2024-02-14 12:40:58,254 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1322
+ 2024-02-14 12:41:03,255 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1323
+ 2024-02-14 12:41:08,256 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1324
+ 2024-02-14 12:41:09,106 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1325
+ 2024-02-14 12:41:09,106 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1326
+ 2024-02-14 12:41:09,107 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1327
+ 2024-02-14 12:41:13,269 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1328
+ 2024-02-14 12:41:18,270 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1329
+ 2024-02-14 12:41:23,271 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1330
+ 2024-02-14 12:41:24,106 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1331
+ 2024-02-14 12:41:24,107 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1332
+ 2024-02-14 12:41:24,107 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1333
+ 2024-02-14 12:41:29,233 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1334
+ 2024-02-14 12:41:34,234 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1335
+ 2024-02-14 12:41:39,106 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1336
+ 2024-02-14 12:41:39,107 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1337
+ 2024-02-14 12:41:39,107 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1338
+ 2024-02-14 12:41:39,257 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1339
+ 2024-02-14 12:41:44,258 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1340
+ 2024-02-14 12:41:49,259 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1341
+ 2024-02-14 12:41:54,106 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1342
+ 2024-02-14 12:41:54,107 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1343
+ 2024-02-14 12:41:54,107 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1344
+ 2024-02-14 12:41:55,257 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1345
+ 2024-02-14 12:42:00,258 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1346
+ 2024-02-14 12:42:05,259 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1347
+ 2024-02-14 12:42:09,107 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1348
+ 2024-02-14 12:42:09,107 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1349
+ 2024-02-14 12:42:09,107 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1350
+ 2024-02-14 12:42:10,266 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1351
+ 2024-02-14 12:42:15,267 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1352
+ 2024-02-14 12:42:20,268 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1353
+ 2024-02-14 12:42:24,107 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1354
+ 2024-02-14 12:42:24,107 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1355
+ 2024-02-14 12:42:24,108 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1356
+ 2024-02-14 12:42:26,221 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1357
+ 2024-02-14 12:42:31,221 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1358
+ 2024-02-14 12:42:36,222 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1359
+ 2024-02-14 12:42:39,107 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1360
+ 2024-02-14 12:42:39,107 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1361
+ 2024-02-14 12:42:39,108 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1362
+ 2024-02-14 12:42:41,277 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1363
+ 2024-02-14 12:42:46,278 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1364
+ 2024-02-14 12:42:51,279 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1365
+ 2024-02-14 12:42:54,107 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1366
+ 2024-02-14 12:42:54,107 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1367
+ 2024-02-14 12:42:54,108 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1368
+ 2024-02-14 12:42:57,228 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1369
+ 2024-02-14 12:43:02,229 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1370
+ 2024-02-14 12:43:07,230 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1371
+ 2024-02-14 12:43:09,107 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1372
+ 2024-02-14 12:43:09,108 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1373
+ 2024-02-14 12:43:09,108 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1374
+ 2024-02-14 12:43:13,193 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1375
+ 2024-02-14 12:43:18,194 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1376
+ 2024-02-14 12:43:23,195 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1377
+ 2024-02-14 12:43:24,107 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1378
+ 2024-02-14 12:43:24,108 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1379
+ 2024-02-14 12:43:24,108 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1380
+ 2024-02-14 12:43:29,184 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1381
+ 2024-02-14 12:43:34,185 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1382
+ 2024-02-14 12:43:39,107 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1383
+ 2024-02-14 12:43:39,108 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1384
+ 2024-02-14 12:43:39,108 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1385
+ 2024-02-14 12:43:39,186 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1386
+ 2024-02-14 12:43:44,187 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1387
+ 2024-02-14 12:43:49,188 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1388
+ 2024-02-14 12:43:54,108 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1389
+ 2024-02-14 12:43:54,108 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1390
+ 2024-02-14 12:43:54,108 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1391
+ 2024-02-14 12:43:54,216 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1392
+ 2024-02-14 12:43:59,217 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1393
+ 2024-02-14 12:44:04,217 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1394
+ 2024-02-14 12:44:09,108 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1395
+ 2024-02-14 12:44:09,108 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1396
+ 2024-02-14 12:44:09,108 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1397
+ 2024-02-14 12:44:10,204 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1398
+ 2024-02-14 12:44:15,204 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1399
+ 2024-02-14 12:44:20,206 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1400
+ 2024-02-14 12:44:24,108 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1401
+ 2024-02-14 12:44:24,108 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1402
+ 2024-02-14 12:44:24,109 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1403
+ 2024-02-14 12:44:25,258 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1404
+ 2024-02-14 12:44:30,259 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1405
+ 2024-02-14 12:44:35,261 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1406
+ 2024-02-14 12:44:39,108 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1407
+ 2024-02-14 12:44:39,108 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1408
+ 2024-02-14 12:44:39,109 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1409
+ 2024-02-14 12:44:41,259 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1410
+ 2024-02-14 12:44:46,260 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1411
+ 2024-02-14 12:44:51,261 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1412
+ 2024-02-14 12:44:54,108 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1413
+ 2024-02-14 12:44:54,108 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1414
+ 2024-02-14 12:44:54,109 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1415
+ 2024-02-14 12:44:57,243 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1416
+ 2024-02-14 12:45:02,244 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1417
+ 2024-02-14 12:45:07,245 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1418
+ 2024-02-14 12:45:09,108 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1419
+ 2024-02-14 12:45:09,109 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1420
+ 2024-02-14 12:45:09,109 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1421
+ 2024-02-14 12:45:12,268 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1422
+ 2024-02-14 12:45:17,269 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1423
+ 2024-02-14 12:45:22,270 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1424
+ 2024-02-14 12:45:24,108 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1425
+ 2024-02-14 12:45:24,109 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1426
+ 2024-02-14 12:45:24,109 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1427
+ 2024-02-14 12:45:28,249 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1428
+ 2024-02-14 12:45:33,249 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1429
+ 2024-02-14 12:45:38,250 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1430
+ 2024-02-14 12:45:39,109 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1431
+ 2024-02-14 12:45:39,109 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1432
+ 2024-02-14 12:45:39,109 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1433
+ 2024-02-14 12:45:44,224 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1434
+ 2024-02-14 12:45:49,225 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1435
+ 2024-02-14 12:45:54,109 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1436
+ 2024-02-14 12:45:54,109 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1437
+ 2024-02-14 12:45:54,109 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1438
+ 2024-02-14 12:45:55,220 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1439
+ 2024-02-14 12:46:00,221 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1440
+ 2024-02-14 12:46:05,222 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1441
+ 2024-02-14 12:46:09,109 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1442
+ 2024-02-14 12:46:09,109 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1443
+ 2024-02-14 12:46:09,109 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1444
+ 2024-02-14 12:46:11,177 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1445
+ 2024-02-14 12:46:16,178 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1446
+ 2024-02-14 12:46:21,179 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1447
+ 2024-02-14 12:46:24,109 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1448
+ 2024-02-14 12:46:24,109 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1449
+ 2024-02-14 12:46:24,109 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1450
+ 2024-02-14 12:46:26,240 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1451
+ 2024-02-14 12:46:31,241 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1452
+ 2024-02-14 12:46:36,242 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1453
+ 2024-02-14 12:46:39,109 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1454
+ 2024-02-14 12:46:39,109 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1455
+ 2024-02-14 12:46:39,110 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1456
+ 2024-02-14 12:46:41,264 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1457
+ 2024-02-14 12:46:46,265 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1458
+ 2024-02-14 12:46:51,266 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1459
+ 2024-02-14 12:46:54,109 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1460
+ 2024-02-14 12:46:54,109 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1461
+ 2024-02-14 12:46:54,110 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1462
+ 2024-02-14 12:46:57,214 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1463
+ 2024-02-14 12:47:02,215 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1464
+ 2024-02-14 12:47:07,216 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1465
+ 2024-02-14 12:47:09,109 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1466
+ 2024-02-14 12:47:09,110 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1467
+ 2024-02-14 12:47:09,110 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1468
+ 2024-02-14 12:47:13,175 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1469
+ 2024-02-14 12:47:18,176 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1470
+ 2024-02-14 12:47:23,177 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1471
+ 2024-02-14 12:47:24,109 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1472
+ 2024-02-14 12:47:24,110 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1473
+ 2024-02-14 12:47:24,110 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1474
+ 2024-02-14 12:47:28,178 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1475
+ 2024-02-14 12:47:33,179 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1476
+ 2024-02-14 12:47:38,180 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1477
+ 2024-02-14 12:47:39,109 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1478
+ 2024-02-14 12:47:39,110 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1479
+ 2024-02-14 12:47:39,110 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1480
+ 2024-02-14 12:47:43,207 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1481
+ 2024-02-14 12:47:48,208 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1482
+ 2024-02-14 12:47:53,209 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1483
+ 2024-02-14 12:47:54,110 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1484
+ 2024-02-14 12:47:54,110 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1485
+ 2024-02-14 12:47:54,110 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1486
+ 2024-02-14 12:47:59,204 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1487
+ 2024-02-14 12:48:04,205 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1488
+ 2024-02-14 12:48:09,110 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1489
+ 2024-02-14 12:48:09,110 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1490
+ 2024-02-14 12:48:09,110 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1491
+ 2024-02-14 12:48:09,217 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1492
+ 2024-02-14 12:48:14,218 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1493
+ 2024-02-14 12:48:19,219 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1494
+ 2024-02-14 12:48:24,110 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1495
+ 2024-02-14 12:48:24,110 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1496
+ 2024-02-14 12:48:24,111 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1497
+ 2024-02-14 12:48:24,263 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1498
+ 2024-02-14 12:48:29,264 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1499
+ 2024-02-14 12:48:34,265 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1500
+ 2024-02-14 12:48:39,110 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1501
+ 2024-02-14 12:48:39,110 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1502
+ 2024-02-14 12:48:39,111 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1503
+ 2024-02-14 12:48:40,177 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1504
+ 2024-02-14 12:48:45,178 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1505
+ 2024-02-14 12:48:50,179 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1506
+ 2024-02-14 12:48:54,110 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1507
+ 2024-02-14 12:48:54,111 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1508
+ 2024-02-14 12:48:54,111 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1509
+ 2024-02-14 12:48:55,241 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1510
+ 2024-02-14 12:49:00,242 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1511
+ 2024-02-14 12:49:05,243 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1512
+ 2024-02-14 12:49:09,111 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1513
+ 2024-02-14 12:49:09,111 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1514
+ 2024-02-14 12:49:09,111 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1515
+ 2024-02-14 12:49:11,190 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1516
+ 2024-02-14 12:49:16,191 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1517
+ 2024-02-14 12:49:21,191 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1518
+ 2024-02-14 12:49:24,110 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1519
+ 2024-02-14 12:49:24,111 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1520
+ 2024-02-14 12:49:24,111 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1521
+ 2024-02-14 12:49:26,227 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1522
+ 2024-02-14 12:49:31,228 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1523
+ 2024-02-14 12:49:31,961 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: resume
1524
+ 2024-02-14 12:49:31,961 INFO HandlerThread:19614 [handler.py:handle_request_resume():699] starting system metrics thread
1525
+ 2024-02-14 12:49:31,961 INFO HandlerThread:19614 [system_monitor.py:start():194] Starting system monitor
1526
+ 2024-02-14 12:49:31,961 INFO SystemMonitor:19614 [system_monitor.py:_start():158] Starting system asset monitoring threads
1527
+ 2024-02-14 12:49:31,961 INFO SystemMonitor:19614 [interfaces.py:start():190] Started cpu monitoring
1528
+ 2024-02-14 12:49:31,962 INFO SystemMonitor:19614 [interfaces.py:start():190] Started disk monitoring
1529
+ 2024-02-14 12:49:31,963 INFO SystemMonitor:19614 [interfaces.py:start():190] Started gpu monitoring
1530
+ 2024-02-14 12:49:31,964 INFO SystemMonitor:19614 [interfaces.py:start():190] Started memory monitoring
1531
+ 2024-02-14 12:49:31,964 INFO SystemMonitor:19614 [interfaces.py:start():190] Started network monitoring
1532
+ 2024-02-14 12:49:32,001 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: pause
1533
+ 2024-02-14 12:49:32,001 INFO HandlerThread:19614 [handler.py:handle_request_pause():708] stopping system metrics thread
1534
+ 2024-02-14 12:49:32,001 INFO HandlerThread:19614 [system_monitor.py:finish():203] Stopping system monitor
1535
+ 2024-02-14 12:49:32,001 DEBUG SystemMonitor:19614 [system_monitor.py:_start():172] Starting system metrics aggregation loop
1536
+ 2024-02-14 12:49:32,001 DEBUG SystemMonitor:19614 [system_monitor.py:_start():179] Finished system metrics aggregation loop
1537
+ 2024-02-14 12:49:32,001 DEBUG SystemMonitor:19614 [system_monitor.py:_start():183] Publishing last batch of metrics
1538
+ 2024-02-14 12:49:32,002 INFO HandlerThread:19614 [interfaces.py:finish():202] Joined cpu monitor
1539
+ 2024-02-14 12:49:32,002 INFO HandlerThread:19614 [interfaces.py:finish():202] Joined disk monitor
1540
+ 2024-02-14 12:49:32,008 INFO HandlerThread:19614 [interfaces.py:finish():202] Joined gpu monitor
1541
+ 2024-02-14 12:49:32,008 INFO HandlerThread:19614 [interfaces.py:finish():202] Joined memory monitor
1542
+ 2024-02-14 12:49:32,008 INFO HandlerThread:19614 [interfaces.py:finish():202] Joined network monitor
1543
+ 2024-02-14 12:49:32,008 DEBUG SenderThread:19614 [sender.py:send():382] send: stats
1544
+ 2024-02-14 12:49:34,169 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: resume
1545
+ 2024-02-14 12:49:34,169 INFO HandlerThread:19614 [handler.py:handle_request_resume():699] starting system metrics thread
1546
+ 2024-02-14 12:49:34,169 INFO HandlerThread:19614 [system_monitor.py:start():194] Starting system monitor
1547
+ 2024-02-14 12:49:34,170 INFO SystemMonitor:19614 [system_monitor.py:_start():158] Starting system asset monitoring threads
1548
+ 2024-02-14 12:49:34,170 INFO SystemMonitor:19614 [interfaces.py:start():190] Started cpu monitoring
1549
+ 2024-02-14 12:49:34,171 INFO SystemMonitor:19614 [interfaces.py:start():190] Started disk monitoring
1550
+ 2024-02-14 12:49:34,171 INFO SystemMonitor:19614 [interfaces.py:start():190] Started gpu monitoring
1551
+ 2024-02-14 12:49:34,172 INFO SystemMonitor:19614 [interfaces.py:start():190] Started memory monitoring
1552
+ 2024-02-14 12:49:34,172 INFO SystemMonitor:19614 [interfaces.py:start():190] Started network monitoring
1553
+ 2024-02-14 12:49:34,433 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: pause
1554
+ 2024-02-14 12:49:34,433 INFO HandlerThread:19614 [handler.py:handle_request_pause():708] stopping system metrics thread
1555
+ 2024-02-14 12:49:34,433 INFO HandlerThread:19614 [system_monitor.py:finish():203] Stopping system monitor
1556
+ 2024-02-14 12:49:34,434 DEBUG SystemMonitor:19614 [system_monitor.py:_start():172] Starting system metrics aggregation loop
1557
+ 2024-02-14 12:49:34,434 DEBUG SystemMonitor:19614 [system_monitor.py:_start():179] Finished system metrics aggregation loop
1558
+ 2024-02-14 12:49:34,434 DEBUG SystemMonitor:19614 [system_monitor.py:_start():183] Publishing last batch of metrics
1559
+ 2024-02-14 12:49:34,435 INFO HandlerThread:19614 [interfaces.py:finish():202] Joined cpu monitor
1560
+ 2024-02-14 12:49:34,435 INFO HandlerThread:19614 [interfaces.py:finish():202] Joined disk monitor
1561
+ 2024-02-14 12:49:34,440 INFO HandlerThread:19614 [interfaces.py:finish():202] Joined gpu monitor
1562
+ 2024-02-14 12:49:34,441 INFO HandlerThread:19614 [interfaces.py:finish():202] Joined memory monitor
1563
+ 2024-02-14 12:49:34,441 INFO HandlerThread:19614 [interfaces.py:finish():202] Joined network monitor
1564
+ 2024-02-14 12:49:34,441 DEBUG SenderThread:19614 [sender.py:send():382] send: stats
1565
+ 2024-02-14 12:49:36,442 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1566
+ 2024-02-14 12:49:36,952 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: resume
1567
+ 2024-02-14 12:49:36,952 INFO HandlerThread:19614 [handler.py:handle_request_resume():699] starting system metrics thread
1568
+ 2024-02-14 12:49:36,952 INFO HandlerThread:19614 [system_monitor.py:start():194] Starting system monitor
1569
+ 2024-02-14 12:49:36,952 INFO SystemMonitor:19614 [system_monitor.py:_start():158] Starting system asset monitoring threads
1570
+ 2024-02-14 12:49:36,953 INFO SystemMonitor:19614 [interfaces.py:start():190] Started cpu monitoring
1571
+ 2024-02-14 12:49:36,953 INFO SystemMonitor:19614 [interfaces.py:start():190] Started disk monitoring
1572
+ 2024-02-14 12:49:36,954 INFO SystemMonitor:19614 [interfaces.py:start():190] Started gpu monitoring
1573
+ 2024-02-14 12:49:36,954 INFO SystemMonitor:19614 [interfaces.py:start():190] Started memory monitoring
1574
+ 2024-02-14 12:49:36,955 INFO SystemMonitor:19614 [interfaces.py:start():190] Started network monitoring
1575
+ 2024-02-14 12:49:36,958 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: pause
1576
+ 2024-02-14 12:49:36,958 INFO HandlerThread:19614 [handler.py:handle_request_pause():708] stopping system metrics thread
1577
+ 2024-02-14 12:49:36,958 INFO HandlerThread:19614 [system_monitor.py:finish():203] Stopping system monitor
1578
+ 2024-02-14 12:49:36,959 INFO HandlerThread:19614 [interfaces.py:finish():202] Joined cpu monitor
1579
+ 2024-02-14 12:49:36,959 INFO HandlerThread:19614 [interfaces.py:finish():202] Joined disk monitor
1580
+ 2024-02-14 12:49:36,959 DEBUG SystemMonitor:19614 [system_monitor.py:_start():172] Starting system metrics aggregation loop
1581
+ 2024-02-14 12:49:36,959 DEBUG SystemMonitor:19614 [system_monitor.py:_start():179] Finished system metrics aggregation loop
1582
+ 2024-02-14 12:49:36,959 DEBUG SystemMonitor:19614 [system_monitor.py:_start():183] Publishing last batch of metrics
1583
+ 2024-02-14 12:49:36,966 INFO HandlerThread:19614 [interfaces.py:finish():202] Joined gpu monitor
1584
+ 2024-02-14 12:49:36,966 INFO HandlerThread:19614 [interfaces.py:finish():202] Joined memory monitor
1585
+ 2024-02-14 12:49:36,966 INFO HandlerThread:19614 [interfaces.py:finish():202] Joined network monitor
1586
+ 2024-02-14 12:49:36,967 DEBUG SenderThread:19614 [sender.py:send():382] send: stats
1587
+ 2024-02-14 12:49:38,539 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: resume
1588
+ 2024-02-14 12:49:38,540 INFO HandlerThread:19614 [handler.py:handle_request_resume():699] starting system metrics thread
1589
+ 2024-02-14 12:49:38,540 INFO HandlerThread:19614 [system_monitor.py:start():194] Starting system monitor
1590
+ 2024-02-14 12:49:38,540 INFO SystemMonitor:19614 [system_monitor.py:_start():158] Starting system asset monitoring threads
1591
+ 2024-02-14 12:49:38,540 INFO SystemMonitor:19614 [interfaces.py:start():190] Started cpu monitoring
1592
+ 2024-02-14 12:49:38,541 INFO SystemMonitor:19614 [interfaces.py:start():190] Started disk monitoring
1593
+ 2024-02-14 12:49:38,542 INFO SystemMonitor:19614 [interfaces.py:start():190] Started gpu monitoring
1594
+ 2024-02-14 12:49:38,542 INFO SystemMonitor:19614 [interfaces.py:start():190] Started memory monitoring
1595
+ 2024-02-14 12:49:38,544 INFO SystemMonitor:19614 [interfaces.py:start():190] Started network monitoring
1596
+ 2024-02-14 12:49:39,006 DEBUG SenderThread:19614 [sender.py:send():382] send: config
1597
+ 2024-02-14 12:49:39,007 DEBUG SenderThread:19614 [sender.py:send():382] send: metric
1598
+ 2024-02-14 12:49:39,007 DEBUG SenderThread:19614 [sender.py:send():382] send: metric
1599
+ 2024-02-14 12:49:39,007 WARNING SenderThread:19614 [sender.py:send_metric():1354] Seen metric with glob (shouldn't happen)
1600
+ 2024-02-14 12:49:39,110 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1601
+ 2024-02-14 12:49:39,111 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1602
+ 2024-02-14 12:49:39,111 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1603
+ 2024-02-14 12:49:42,264 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1604
+ 2024-02-14 12:49:47,265 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1605
+ 2024-02-14 12:49:52,266 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1606
+ 2024-02-14 12:49:54,111 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1607
+ 2024-02-14 12:49:54,111 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1608
+ 2024-02-14 12:49:54,112 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1609
+ 2024-02-14 12:49:58,222 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1610
+ 2024-02-14 12:50:03,223 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1611
+ 2024-02-14 12:50:08,229 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1612
+ 2024-02-14 12:50:08,884 INFO Thread-12 :19614 [dir_watcher.py:_on_file_modified():288] file/dir modified: /content/wandb/run-20240214_115405-518dheci/files/config.yaml
1613
+ 2024-02-14 12:50:09,111 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1614
+ 2024-02-14 12:50:09,111 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1615
+ 2024-02-14 12:50:09,112 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1616
+ 2024-02-14 12:50:14,223 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1617
+ 2024-02-14 12:50:19,224 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1618
+ 2024-02-14 12:50:24,111 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1619
+ 2024-02-14 12:50:24,111 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1620
+ 2024-02-14 12:50:24,112 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1621
+ 2024-02-14 12:50:25,201 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1622
+ 2024-02-14 12:50:30,202 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1623
+ 2024-02-14 12:50:35,203 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1624
+ 2024-02-14 12:50:38,544 DEBUG SystemMonitor:19614 [system_monitor.py:_start():172] Starting system metrics aggregation loop
1625
+ 2024-02-14 12:50:38,545 DEBUG SenderThread:19614 [sender.py:send():382] send: stats
1626
+ 2024-02-14 12:50:39,111 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1627
+ 2024-02-14 12:50:39,112 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1628
+ 2024-02-14 12:50:39,112 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1629
+ 2024-02-14 12:50:41,197 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1630
+ 2024-02-14 12:50:46,198 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1631
+ 2024-02-14 12:50:51,199 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1632
+ 2024-02-14 12:50:54,111 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1633
+ 2024-02-14 12:50:54,112 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1634
+ 2024-02-14 12:50:54,112 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
1635
+ 2024-02-14 12:50:57,198 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1636
+ 2024-02-14 12:50:57,234 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: partial_history
1637
+ 2024-02-14 12:50:57,235 DEBUG SenderThread:19614 [sender.py:send():382] send: history
1638
+ 2024-02-14 12:50:57,236 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: summary_record
1639
+ 2024-02-14 12:50:57,236 INFO SenderThread:19614 [sender.py:_save_file():1403] saving file wandb-summary.json with policy end
1640
+ 2024-02-14 12:50:57,900 INFO Thread-12 :19614 [dir_watcher.py:_on_file_modified():288] file/dir modified: /content/wandb/run-20240214_115405-518dheci/files/wandb-summary.json
1641
+ 2024-02-14 12:50:57,901 INFO Thread-12 :19614 [dir_watcher.py:_on_file_created():271] file/dir created: /content/wandb/run-20240214_115405-518dheci/files/output.log
1642
+ 2024-02-14 12:50:59,901 INFO Thread-12 :19614 [dir_watcher.py:_on_file_modified():288] file/dir modified: /content/wandb/run-20240214_115405-518dheci/files/output.log
1643
+ 2024-02-14 12:51:02,277 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1644
+ 2024-02-14 12:51:07,278 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: status_report
1645
+ 2024-02-14 12:51:08,563 DEBUG SenderThread:19614 [sender.py:send():382] send: stats
1646
+ 2024-02-14 12:51:09,112 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: internal_messages
1647
+ 2024-02-14 12:51:09,112 DEBUG HandlerThread:19614 [handler.py:handle_request():146] handle_request: stop_status
1648
+ 2024-02-14 12:51:09,112 DEBUG SenderThread:19614 [sender.py:send_request():409] send_request: stop_status
wandb/run-20240214_115405-518dheci/logs/debug.log CHANGED
@@ -28,3 +28,16 @@ config: {}
28
  2024-02-14 11:54:08,936 INFO MainThread:4105 [wandb_run.py:_redirect():2186] Redirects installed.
29
  2024-02-14 11:54:08,937 INFO MainThread:4105 [wandb_init.py:init():847] run started, returning control to user process
30
  2024-02-14 11:54:08,940 INFO MainThread:4105 [wandb_run.py:_config_callback():1343] config_cb None None {'vocab_size': 65024, 'hidden_size': 4544, 'num_hidden_layers': 32, 'num_attention_heads': 71, 'layer_norm_epsilon': 1e-05, 'initializer_range': 0.02, 'use_cache': False, 'hidden_dropout': 0.0, 'attention_dropout': 0.0, 'bos_token_id': 11, 'eos_token_id': 11, 'num_kv_heads': 71, 'alibi': False, 'new_decoder_architecture': False, 'multi_query': True, 'parallel_attn': True, 'bias': False, 'return_dict': True, 'output_hidden_states': False, 'output_attentions': False, 'torchscript': False, 'torch_dtype': 'bfloat16', 'use_bfloat16': False, 'tf_legacy_loss': False, 'pruned_heads': {}, 'tie_word_embeddings': True, 'chunk_size_feed_forward': 0, 'is_encoder_decoder': False, 'is_decoder': False, 'cross_attention_hidden_size': None, 'add_cross_attention': False, 'tie_encoder_decoder': False, 'max_length': 20, 'min_length': 0, 'do_sample': False, 'early_stopping': False, 'num_beams': 1, 'num_beam_groups': 1, 'diversity_penalty': 0.0, 'temperature': 1.0, 'top_k': 50, 'top_p': 1.0, 'typical_p': 1.0, 'repetition_penalty': 1.0, 'length_penalty': 1.0, 'no_repeat_ngram_size': 0, 'encoder_no_repeat_ngram_size': 0, 'bad_words_ids': None, 'num_return_sequences': 1, 'output_scores': False, 'return_dict_in_generate': False, 'forced_bos_token_id': None, 'forced_eos_token_id': None, 'remove_invalid_values': False, 'exponential_decay_length_penalty': None, 'suppress_tokens': None, 'begin_suppress_tokens': None, 'architectures': ['FalconForCausalLM'], 'finetuning_task': None, 'id2label': {0: 'LABEL_0', 1: 'LABEL_1'}, 'label2id': {'LABEL_0': 0, 'LABEL_1': 1}, 'tokenizer_class': None, 'prefix': None, 'pad_token_id': None, 'sep_token_id': None, 'decoder_start_token_id': None, 'task_specific_params': None, 'problem_type': None, '_name_or_path': 'vilsonrodrigues/falcon-7b-instruct-sharded', 'transformers_version': '4.37.2', 'apply_residual_connection_post_layernorm': False, 'auto_map': {'AutoConfig': 'vilsonrodrigues/falcon-7b-instruct-sharded--configuration_falcon.FalconConfig', 'AutoModel': 'vilsonrodrigues/falcon-7b-instruct-sharded--modeling_falcon.FalconModel', 'AutoModelForSequenceClassification': 'vilsonrodrigues/falcon-7b-instruct-sharded--modeling_falcon.FalconForSequenceClassification', 'AutoModelForTokenClassification': 'vilsonrodrigues/falcon-7b-instruct-sharded--modeling_falcon.FalconForTokenClassification', 'AutoModelForQuestionAnswering': 'vilsonrodrigues/falcon-7b-instruct-sharded--modeling_falcon.FalconForQuestionAnswering', 'AutoModelForCausalLM': 'vilsonrodrigues/falcon-7b-instruct-sharded--modeling_falcon.FalconForCausalLM'}, 'model_type': 'falcon', 'quantization_config': {'quant_method': 'QuantizationMethod.BITS_AND_BYTES', 'load_in_8bit': False, 'load_in_4bit': True, 'llm_int8_threshold': 6.0, 'llm_int8_skip_modules': None, 'llm_int8_enable_fp32_cpu_offload': False, 'llm_int8_has_fp16_weight': False, 'bnb_4bit_quant_type': 'nf4', 'bnb_4bit_use_double_quant': True, 'bnb_4bit_compute_dtype': 'float16'}, 'output_dir': './', 'overwrite_output_dir': False, 'do_train': False, 'do_eval': False, 'do_predict': False, 'evaluation_strategy': 'no', 'prediction_loss_only': False, 'per_device_train_batch_size': 4, 'per_device_eval_batch_size': 8, 'per_gpu_train_batch_size': None, 'per_gpu_eval_batch_size': None, 'gradient_accumulation_steps': 4, 'eval_accumulation_steps': None, 'eval_delay': 0, 'learning_rate': 0.0002, 'weight_decay': 0.0, 'adam_beta1': 0.9, 'adam_beta2': 0.999, 'adam_epsilon': 1e-08, 'max_grad_norm': 0.3, 'num_train_epochs': 3.0, 'max_steps': 180, 'lr_scheduler_type': 'cosine', 'lr_scheduler_kwargs': {}, 'warmup_ratio': 0.03, 'warmup_steps': 0, 'log_level': 'passive', 'log_level_replica': 'warning', 'log_on_each_node': True, 'logging_dir': './runs/Feb14_11-53-30_f4b889916ade', 'logging_strategy': 'steps', 'logging_first_step': False, 'logging_steps': 10, 'logging_nan_inf_filter': True, 'save_strategy': 'steps', 'save_steps': 10, 'save_total_limit': None, 'save_safetensors': True, 'save_on_each_node': False, 'save_only_model': False, 'no_cuda': False, 'use_cpu': False, 'use_mps_device': False, 'seed': 42, 'data_seed': None, 'jit_mode_eval': False, 'use_ipex': False, 'bf16': False, 'fp16': True, 'fp16_opt_level': 'O1', 'half_precision_backend': 'auto', 'bf16_full_eval': False, 'fp16_full_eval': False, 'tf32': None, 'local_rank': 0, 'ddp_backend': None, 'tpu_num_cores': None, 'tpu_metrics_debug': False, 'debug': [], 'dataloader_drop_last': False, 'eval_steps': None, 'dataloader_num_workers': 0, 'past_index': -1, 'run_name': './', 'disable_tqdm': False, 'remove_unused_columns': True, 'label_names': None, 'load_best_model_at_end': False, 'metric_for_best_model': None, 'greater_is_better': None, 'ignore_data_skip': False, 'fsdp': [], 'fsdp_min_num_params': 0, 'fsdp_config': {'min_num_params': 0, 'xla': False, 'xla_fsdp_grad_ckpt': False}, 'fsdp_transformer_layer_cls_to_wrap': None, 'deepspeed': None, 'label_smoothing_factor': 0.0, 'optim': 'paged_adamw_32bit', 'optim_args': None, 'adafactor': False, 'group_by_length': True, 'length_column_name': 'length', 'report_to': ['tensorboard', 'wandb'], 'ddp_find_unused_parameters': None, 'ddp_bucket_cap_mb': None, 'ddp_broadcast_buffers': None, 'dataloader_pin_memory': True, 'dataloader_persistent_workers': False, 'skip_memory_metrics': True, 'use_legacy_prediction_loop': False, 'push_to_hub': True, 'resume_from_checkpoint': None, 'hub_model_id': None, 'hub_strategy': 'every_save', 'hub_token': '<HUB_TOKEN>', 'hub_private_repo': False, 'hub_always_push': False, 'gradient_checkpointing': False, 'gradient_checkpointing_kwargs': None, 'include_inputs_for_metrics': False, 'fp16_backend': 'auto', 'push_to_hub_model_id': None, 'push_to_hub_organization': None, 'push_to_hub_token': '<PUSH_TO_HUB_TOKEN>', 'mp_parameters': '', 'auto_find_batch_size': False, 'full_determinism': False, 'torchdynamo': None, 'ray_scope': 'last', 'ddp_timeout': 1800, 'torch_compile': False, 'torch_compile_backend': None, 'torch_compile_mode': None, 'dispatch_batches': None, 'split_batches': False, 'include_tokens_per_second': False, 'include_num_input_tokens_seen': False, 'neftune_noise_alpha': None}
 
 
 
 
 
 
 
 
 
 
 
 
 
 
28
  2024-02-14 11:54:08,936 INFO MainThread:4105 [wandb_run.py:_redirect():2186] Redirects installed.
29
  2024-02-14 11:54:08,937 INFO MainThread:4105 [wandb_init.py:init():847] run started, returning control to user process
30
  2024-02-14 11:54:08,940 INFO MainThread:4105 [wandb_run.py:_config_callback():1343] config_cb None None {'vocab_size': 65024, 'hidden_size': 4544, 'num_hidden_layers': 32, 'num_attention_heads': 71, 'layer_norm_epsilon': 1e-05, 'initializer_range': 0.02, 'use_cache': False, 'hidden_dropout': 0.0, 'attention_dropout': 0.0, 'bos_token_id': 11, 'eos_token_id': 11, 'num_kv_heads': 71, 'alibi': False, 'new_decoder_architecture': False, 'multi_query': True, 'parallel_attn': True, 'bias': False, 'return_dict': True, 'output_hidden_states': False, 'output_attentions': False, 'torchscript': False, 'torch_dtype': 'bfloat16', 'use_bfloat16': False, 'tf_legacy_loss': False, 'pruned_heads': {}, 'tie_word_embeddings': True, 'chunk_size_feed_forward': 0, 'is_encoder_decoder': False, 'is_decoder': False, 'cross_attention_hidden_size': None, 'add_cross_attention': False, 'tie_encoder_decoder': False, 'max_length': 20, 'min_length': 0, 'do_sample': False, 'early_stopping': False, 'num_beams': 1, 'num_beam_groups': 1, 'diversity_penalty': 0.0, 'temperature': 1.0, 'top_k': 50, 'top_p': 1.0, 'typical_p': 1.0, 'repetition_penalty': 1.0, 'length_penalty': 1.0, 'no_repeat_ngram_size': 0, 'encoder_no_repeat_ngram_size': 0, 'bad_words_ids': None, 'num_return_sequences': 1, 'output_scores': False, 'return_dict_in_generate': False, 'forced_bos_token_id': None, 'forced_eos_token_id': None, 'remove_invalid_values': False, 'exponential_decay_length_penalty': None, 'suppress_tokens': None, 'begin_suppress_tokens': None, 'architectures': ['FalconForCausalLM'], 'finetuning_task': None, 'id2label': {0: 'LABEL_0', 1: 'LABEL_1'}, 'label2id': {'LABEL_0': 0, 'LABEL_1': 1}, 'tokenizer_class': None, 'prefix': None, 'pad_token_id': None, 'sep_token_id': None, 'decoder_start_token_id': None, 'task_specific_params': None, 'problem_type': None, '_name_or_path': 'vilsonrodrigues/falcon-7b-instruct-sharded', 'transformers_version': '4.37.2', 'apply_residual_connection_post_layernorm': False, 'auto_map': {'AutoConfig': 'vilsonrodrigues/falcon-7b-instruct-sharded--configuration_falcon.FalconConfig', 'AutoModel': 'vilsonrodrigues/falcon-7b-instruct-sharded--modeling_falcon.FalconModel', 'AutoModelForSequenceClassification': 'vilsonrodrigues/falcon-7b-instruct-sharded--modeling_falcon.FalconForSequenceClassification', 'AutoModelForTokenClassification': 'vilsonrodrigues/falcon-7b-instruct-sharded--modeling_falcon.FalconForTokenClassification', 'AutoModelForQuestionAnswering': 'vilsonrodrigues/falcon-7b-instruct-sharded--modeling_falcon.FalconForQuestionAnswering', 'AutoModelForCausalLM': 'vilsonrodrigues/falcon-7b-instruct-sharded--modeling_falcon.FalconForCausalLM'}, 'model_type': 'falcon', 'quantization_config': {'quant_method': 'QuantizationMethod.BITS_AND_BYTES', 'load_in_8bit': False, 'load_in_4bit': True, 'llm_int8_threshold': 6.0, 'llm_int8_skip_modules': None, 'llm_int8_enable_fp32_cpu_offload': False, 'llm_int8_has_fp16_weight': False, 'bnb_4bit_quant_type': 'nf4', 'bnb_4bit_use_double_quant': True, 'bnb_4bit_compute_dtype': 'float16'}, 'output_dir': './', 'overwrite_output_dir': False, 'do_train': False, 'do_eval': False, 'do_predict': False, 'evaluation_strategy': 'no', 'prediction_loss_only': False, 'per_device_train_batch_size': 4, 'per_device_eval_batch_size': 8, 'per_gpu_train_batch_size': None, 'per_gpu_eval_batch_size': None, 'gradient_accumulation_steps': 4, 'eval_accumulation_steps': None, 'eval_delay': 0, 'learning_rate': 0.0002, 'weight_decay': 0.0, 'adam_beta1': 0.9, 'adam_beta2': 0.999, 'adam_epsilon': 1e-08, 'max_grad_norm': 0.3, 'num_train_epochs': 3.0, 'max_steps': 180, 'lr_scheduler_type': 'cosine', 'lr_scheduler_kwargs': {}, 'warmup_ratio': 0.03, 'warmup_steps': 0, 'log_level': 'passive', 'log_level_replica': 'warning', 'log_on_each_node': True, 'logging_dir': './runs/Feb14_11-53-30_f4b889916ade', 'logging_strategy': 'steps', 'logging_first_step': False, 'logging_steps': 10, 'logging_nan_inf_filter': True, 'save_strategy': 'steps', 'save_steps': 10, 'save_total_limit': None, 'save_safetensors': True, 'save_on_each_node': False, 'save_only_model': False, 'no_cuda': False, 'use_cpu': False, 'use_mps_device': False, 'seed': 42, 'data_seed': None, 'jit_mode_eval': False, 'use_ipex': False, 'bf16': False, 'fp16': True, 'fp16_opt_level': 'O1', 'half_precision_backend': 'auto', 'bf16_full_eval': False, 'fp16_full_eval': False, 'tf32': None, 'local_rank': 0, 'ddp_backend': None, 'tpu_num_cores': None, 'tpu_metrics_debug': False, 'debug': [], 'dataloader_drop_last': False, 'eval_steps': None, 'dataloader_num_workers': 0, 'past_index': -1, 'run_name': './', 'disable_tqdm': False, 'remove_unused_columns': True, 'label_names': None, 'load_best_model_at_end': False, 'metric_for_best_model': None, 'greater_is_better': None, 'ignore_data_skip': False, 'fsdp': [], 'fsdp_min_num_params': 0, 'fsdp_config': {'min_num_params': 0, 'xla': False, 'xla_fsdp_grad_ckpt': False}, 'fsdp_transformer_layer_cls_to_wrap': None, 'deepspeed': None, 'label_smoothing_factor': 0.0, 'optim': 'paged_adamw_32bit', 'optim_args': None, 'adafactor': False, 'group_by_length': True, 'length_column_name': 'length', 'report_to': ['tensorboard', 'wandb'], 'ddp_find_unused_parameters': None, 'ddp_bucket_cap_mb': None, 'ddp_broadcast_buffers': None, 'dataloader_pin_memory': True, 'dataloader_persistent_workers': False, 'skip_memory_metrics': True, 'use_legacy_prediction_loop': False, 'push_to_hub': True, 'resume_from_checkpoint': None, 'hub_model_id': None, 'hub_strategy': 'every_save', 'hub_token': '<HUB_TOKEN>', 'hub_private_repo': False, 'hub_always_push': False, 'gradient_checkpointing': False, 'gradient_checkpointing_kwargs': None, 'include_inputs_for_metrics': False, 'fp16_backend': 'auto', 'push_to_hub_model_id': None, 'push_to_hub_organization': None, 'push_to_hub_token': '<PUSH_TO_HUB_TOKEN>', 'mp_parameters': '', 'auto_find_batch_size': False, 'full_determinism': False, 'torchdynamo': None, 'ray_scope': 'last', 'ddp_timeout': 1800, 'torch_compile': False, 'torch_compile_backend': None, 'torch_compile_mode': None, 'dispatch_batches': None, 'split_batches': False, 'include_tokens_per_second': False, 'include_num_input_tokens_seen': False, 'neftune_noise_alpha': None}
31
+ 2024-02-14 12:22:52,244 INFO MainThread:4105 [jupyter.py:save_ipynb():373] not saving jupyter notebook
32
+ 2024-02-14 12:22:52,244 INFO MainThread:4105 [wandb_init.py:_pause_backend():437] pausing backend
33
+ 2024-02-14 12:49:31,960 INFO MainThread:4105 [wandb_init.py:_resume_backend():442] resuming backend
34
+ 2024-02-14 12:49:31,964 INFO MainThread:4105 [jupyter.py:save_ipynb():373] not saving jupyter notebook
35
+ 2024-02-14 12:49:31,964 INFO MainThread:4105 [wandb_init.py:_pause_backend():437] pausing backend
36
+ 2024-02-14 12:49:34,168 INFO MainThread:4105 [wandb_init.py:_resume_backend():442] resuming backend
37
+ 2024-02-14 12:49:34,432 INFO MainThread:4105 [jupyter.py:save_ipynb():373] not saving jupyter notebook
38
+ 2024-02-14 12:49:34,433 INFO MainThread:4105 [wandb_init.py:_pause_backend():437] pausing backend
39
+ 2024-02-14 12:49:36,951 INFO MainThread:4105 [wandb_init.py:_resume_backend():442] resuming backend
40
+ 2024-02-14 12:49:36,958 INFO MainThread:4105 [jupyter.py:save_ipynb():373] not saving jupyter notebook
41
+ 2024-02-14 12:49:36,958 INFO MainThread:4105 [wandb_init.py:_pause_backend():437] pausing backend
42
+ 2024-02-14 12:49:38,538 INFO MainThread:4105 [wandb_init.py:_resume_backend():442] resuming backend
43
+ 2024-02-14 12:49:39,001 INFO MainThread:4105 [wandb_run.py:_config_callback():1343] config_cb None None {'vocab_size': 65024, 'hidden_size': 4544, 'num_hidden_layers': 32, 'num_attention_heads': 71, 'layer_norm_epsilon': 1e-05, 'initializer_range': 0.02, 'use_cache': False, 'hidden_dropout': 0.0, 'attention_dropout': 0.0, 'bos_token_id': 11, 'eos_token_id': 11, 'num_kv_heads': 71, 'alibi': False, 'new_decoder_architecture': False, 'multi_query': True, 'parallel_attn': True, 'bias': False, 'return_dict': True, 'output_hidden_states': False, 'output_attentions': False, 'torchscript': False, 'torch_dtype': 'bfloat16', 'use_bfloat16': False, 'tf_legacy_loss': False, 'pruned_heads': {}, 'tie_word_embeddings': True, 'chunk_size_feed_forward': 0, 'is_encoder_decoder': False, 'is_decoder': False, 'cross_attention_hidden_size': None, 'add_cross_attention': False, 'tie_encoder_decoder': False, 'max_length': 20, 'min_length': 0, 'do_sample': False, 'early_stopping': False, 'num_beams': 1, 'num_beam_groups': 1, 'diversity_penalty': 0.0, 'temperature': 1.0, 'top_k': 50, 'top_p': 1.0, 'typical_p': 1.0, 'repetition_penalty': 1.0, 'length_penalty': 1.0, 'no_repeat_ngram_size': 0, 'encoder_no_repeat_ngram_size': 0, 'bad_words_ids': None, 'num_return_sequences': 1, 'output_scores': False, 'return_dict_in_generate': False, 'forced_bos_token_id': None, 'forced_eos_token_id': None, 'remove_invalid_values': False, 'exponential_decay_length_penalty': None, 'suppress_tokens': None, 'begin_suppress_tokens': None, 'architectures': ['FalconForCausalLM'], 'finetuning_task': None, 'id2label': {0: 'LABEL_0', 1: 'LABEL_1'}, 'label2id': {'LABEL_0': 0, 'LABEL_1': 1}, 'tokenizer_class': None, 'prefix': None, 'pad_token_id': None, 'sep_token_id': None, 'decoder_start_token_id': None, 'task_specific_params': None, 'problem_type': None, '_name_or_path': 'vilsonrodrigues/falcon-7b-instruct-sharded', 'transformers_version': '4.37.2', 'apply_residual_connection_post_layernorm': False, 'auto_map': {'AutoConfig': 'vilsonrodrigues/falcon-7b-instruct-sharded--configuration_falcon.FalconConfig', 'AutoModel': 'vilsonrodrigues/falcon-7b-instruct-sharded--modeling_falcon.FalconModel', 'AutoModelForSequenceClassification': 'vilsonrodrigues/falcon-7b-instruct-sharded--modeling_falcon.FalconForSequenceClassification', 'AutoModelForTokenClassification': 'vilsonrodrigues/falcon-7b-instruct-sharded--modeling_falcon.FalconForTokenClassification', 'AutoModelForQuestionAnswering': 'vilsonrodrigues/falcon-7b-instruct-sharded--modeling_falcon.FalconForQuestionAnswering', 'AutoModelForCausalLM': 'vilsonrodrigues/falcon-7b-instruct-sharded--modeling_falcon.FalconForCausalLM'}, 'model_type': 'falcon', 'quantization_config': {'quant_method': 'QuantizationMethod.BITS_AND_BYTES', 'load_in_8bit': False, 'load_in_4bit': True, 'llm_int8_threshold': 6.0, 'llm_int8_skip_modules': None, 'llm_int8_enable_fp32_cpu_offload': False, 'llm_int8_has_fp16_weight': False, 'bnb_4bit_quant_type': 'nf4', 'bnb_4bit_use_double_quant': True, 'bnb_4bit_compute_dtype': 'float16'}, 'output_dir': './', 'overwrite_output_dir': False, 'do_train': False, 'do_eval': False, 'do_predict': False, 'evaluation_strategy': 'no', 'prediction_loss_only': False, 'per_device_train_batch_size': 4, 'per_device_eval_batch_size': 8, 'per_gpu_train_batch_size': None, 'per_gpu_eval_batch_size': None, 'gradient_accumulation_steps': 4, 'eval_accumulation_steps': None, 'eval_delay': 0, 'learning_rate': 0.0002, 'weight_decay': 0.0, 'adam_beta1': 0.9, 'adam_beta2': 0.999, 'adam_epsilon': 1e-08, 'max_grad_norm': 0.3, 'num_train_epochs': 3.0, 'max_steps': 250, 'lr_scheduler_type': 'cosine', 'lr_scheduler_kwargs': {}, 'warmup_ratio': 0.03, 'warmup_steps': 0, 'log_level': 'passive', 'log_level_replica': 'warning', 'log_on_each_node': True, 'logging_dir': './runs/Feb14_12-49-31_f4b889916ade', 'logging_strategy': 'steps', 'logging_first_step': False, 'logging_steps': 10, 'logging_nan_inf_filter': True, 'save_strategy': 'steps', 'save_steps': 10, 'save_total_limit': None, 'save_safetensors': True, 'save_on_each_node': False, 'save_only_model': False, 'no_cuda': False, 'use_cpu': False, 'use_mps_device': False, 'seed': 42, 'data_seed': None, 'jit_mode_eval': False, 'use_ipex': False, 'bf16': False, 'fp16': True, 'fp16_opt_level': 'O1', 'half_precision_backend': 'auto', 'bf16_full_eval': False, 'fp16_full_eval': False, 'tf32': None, 'local_rank': 0, 'ddp_backend': None, 'tpu_num_cores': None, 'tpu_metrics_debug': False, 'debug': [], 'dataloader_drop_last': False, 'eval_steps': None, 'dataloader_num_workers': 0, 'past_index': -1, 'run_name': './', 'disable_tqdm': False, 'remove_unused_columns': True, 'label_names': None, 'load_best_model_at_end': False, 'metric_for_best_model': None, 'greater_is_better': None, 'ignore_data_skip': False, 'fsdp': [], 'fsdp_min_num_params': 0, 'fsdp_config': {'min_num_params': 0, 'xla': False, 'xla_fsdp_grad_ckpt': False}, 'fsdp_transformer_layer_cls_to_wrap': None, 'deepspeed': None, 'label_smoothing_factor': 0.0, 'optim': 'paged_adamw_32bit', 'optim_args': None, 'adafactor': False, 'group_by_length': True, 'length_column_name': 'length', 'report_to': ['tensorboard', 'wandb'], 'ddp_find_unused_parameters': None, 'ddp_bucket_cap_mb': None, 'ddp_broadcast_buffers': None, 'dataloader_pin_memory': True, 'dataloader_persistent_workers': False, 'skip_memory_metrics': True, 'use_legacy_prediction_loop': False, 'push_to_hub': True, 'resume_from_checkpoint': None, 'hub_model_id': None, 'hub_strategy': 'every_save', 'hub_token': '<HUB_TOKEN>', 'hub_private_repo': False, 'hub_always_push': False, 'gradient_checkpointing': False, 'gradient_checkpointing_kwargs': None, 'include_inputs_for_metrics': False, 'fp16_backend': 'auto', 'push_to_hub_model_id': None, 'push_to_hub_organization': None, 'push_to_hub_token': '<PUSH_TO_HUB_TOKEN>', 'mp_parameters': '', 'auto_find_batch_size': False, 'full_determinism': False, 'torchdynamo': None, 'ray_scope': 'last', 'ddp_timeout': 1800, 'torch_compile': False, 'torch_compile_backend': None, 'torch_compile_mode': None, 'dispatch_batches': None, 'split_batches': False, 'include_tokens_per_second': False, 'include_num_input_tokens_seen': False, 'neftune_noise_alpha': None}
wandb/run-20240214_115405-518dheci/run-518dheci.wandb CHANGED
Binary files a/wandb/run-20240214_115405-518dheci/run-518dheci.wandb and b/wandb/run-20240214_115405-518dheci/run-518dheci.wandb differ