2023-04-06 05:16:44,916 ---------------------------------------------------------------------------------------------------- 2023-04-06 05:16:44,917 Model: "SequenceTagger( (embeddings): StackedEmbeddings( (list_embedding_0): FlairEmbeddings( (lm): LanguageModel( (drop): Dropout(p=0.05, inplace=False) (encoder): Embedding(300, 100) (rnn): LSTM(100, 2048) ) ) (list_embedding_1): FlairEmbeddings( (lm): LanguageModel( (drop): Dropout(p=0.05, inplace=False) (encoder): Embedding(300, 100) (rnn): LSTM(100, 2048) ) ) ) (word_dropout): WordDropout(p=0.05) (locked_dropout): LockedDropout(p=0.5) (embedding2nn): Linear(in_features=4096, out_features=4096, bias=True) (rnn): LSTM(4096, 256, batch_first=True, bidirectional=True) (linear): Linear(in_features=512, out_features=19, bias=True) (loss_function): ViterbiLoss() (crf): CRF() )" 2023-04-06 05:16:44,917 ---------------------------------------------------------------------------------------------------- 2023-04-06 05:16:44,917 Corpus: "Corpus: 75187 train + 9603 dev + 9479 test sentences" 2023-04-06 05:16:44,917 ---------------------------------------------------------------------------------------------------- 2023-04-06 05:16:44,917 Parameters: 2023-04-06 05:16:44,917 - learning_rate: "0.100000" 2023-04-06 05:16:44,917 - mini_batch_size: "32" 2023-04-06 05:16:44,917 - patience: "3" 2023-04-06 05:16:44,917 - anneal_factor: "0.5" 2023-04-06 05:16:44,917 - max_epochs: "150" 2023-04-06 05:16:44,917 - shuffle: "True" 2023-04-06 05:16:44,917 - train_with_dev: "True" 2023-04-06 05:16:44,917 - batch_growth_annealing: "False" 2023-04-06 05:16:44,917 ---------------------------------------------------------------------------------------------------- 2023-04-06 05:16:44,917 Model training base path: "resources/taggers/release-upos-0" 2023-04-06 05:16:44,917 ---------------------------------------------------------------------------------------------------- 2023-04-06 05:16:44,917 Device: cuda:3 2023-04-06 05:16:44,917 ---------------------------------------------------------------------------------------------------- 2023-04-06 05:16:44,917 Embeddings storage mode: cpu 2023-04-06 05:16:44,917 ---------------------------------------------------------------------------------------------------- 2023-04-06 05:17:03,561 epoch 1 - iter 265/2650 - loss 0.50170632 - time (sec): 18.64 - samples/sec: 3251.97 - lr: 0.100000 2023-04-06 05:17:38,709 epoch 1 - iter 530/2650 - loss 0.29323606 - time (sec): 53.79 - samples/sec: 4032.43 - lr: 0.100000 2023-04-06 05:18:22,261 epoch 1 - iter 795/2650 - loss 0.24123785 - time (sec): 97.34 - samples/sec: 4036.97 - lr: 0.100000 2023-04-06 05:18:59,300 epoch 1 - iter 1060/2650 - loss 0.20810311 - time (sec): 134.38 - samples/sec: 4096.78 - lr: 0.100000 2023-04-06 05:19:21,568 epoch 1 - iter 1325/2650 - loss 0.18622596 - time (sec): 156.65 - samples/sec: 4279.87 - lr: 0.100000 2023-04-06 05:19:50,614 epoch 1 - iter 1590/2650 - loss 0.17027729 - time (sec): 185.70 - samples/sec: 4364.30 - lr: 0.100000 2023-04-06 05:20:35,819 epoch 1 - iter 1855/2650 - loss 0.16111996 - time (sec): 230.90 - samples/sec: 4440.08 - lr: 0.100000 2023-04-06 05:21:11,756 epoch 1 - iter 2120/2650 - loss 0.15631584 - time (sec): 266.84 - samples/sec: 4485.77 - lr: 0.100000 2023-04-06 05:21:42,635 epoch 1 - iter 2385/2650 - loss 0.15367147 - time (sec): 297.72 - samples/sec: 4388.92 - lr: 0.100000 2023-04-06 05:22:19,083 epoch 1 - iter 2650/2650 - loss 0.14973361 - time (sec): 334.17 - samples/sec: 4376.17 - lr: 0.100000 2023-04-06 05:22:19,083 ---------------------------------------------------------------------------------------------------- 2023-04-06 05:22:19,083 EPOCH 1 done: loss 0.1497 - lr 0.100000 2023-04-06 05:22:19,083 BAD EPOCHS (no improvement): 0 2023-04-06 05:22:19,119 ---------------------------------------------------------------------------------------------------- 2023-04-06 05:22:36,362 epoch 2 - iter 265/2650 - loss 0.10882262 - time (sec): 17.24 - samples/sec: 8464.84 - lr: 0.100000 2023-04-06 05:22:54,533 epoch 2 - iter 530/2650 - loss 0.10784413 - time (sec): 35.41 - samples/sec: 8310.34 - lr: 0.100000 2023-04-06 05:23:12,321 epoch 2 - iter 795/2650 - loss 0.10608341 - time (sec): 53.20 - samples/sec: 8276.54 - lr: 0.100000 2023-04-06 05:23:29,990 epoch 2 - iter 1060/2650 - loss 0.10484208 - time (sec): 70.87 - samples/sec: 8270.33 - lr: 0.100000 2023-04-06 05:23:47,820 epoch 2 - iter 1325/2650 - loss 0.10356510 - time (sec): 88.70 - samples/sec: 8271.50 - lr: 0.100000 2023-04-06 05:24:05,227 epoch 2 - iter 1590/2650 - loss 0.10270563 - time (sec): 106.11 - samples/sec: 8282.10 - lr: 0.100000 2023-04-06 05:24:23,510 epoch 2 - iter 1855/2650 - loss 0.10225705 - time (sec): 124.39 - samples/sec: 8262.63 - lr: 0.100000 2023-04-06 05:24:41,170 epoch 2 - iter 2120/2650 - loss 0.10148676 - time (sec): 142.05 - samples/sec: 8248.24 - lr: 0.100000 2023-04-06 05:24:58,701 epoch 2 - iter 2385/2650 - loss 0.10074839 - time (sec): 159.58 - samples/sec: 8253.14 - lr: 0.100000 2023-04-06 05:25:16,470 epoch 2 - iter 2650/2650 - loss 0.10013797 - time (sec): 177.35 - samples/sec: 8245.61 - lr: 0.100000 2023-04-06 05:25:16,470 ---------------------------------------------------------------------------------------------------- 2023-04-06 05:25:16,470 EPOCH 2 done: loss 0.1001 - lr 0.100000 2023-04-06 05:25:16,470 BAD EPOCHS (no improvement): 0 2023-04-06 05:25:16,473 ---------------------------------------------------------------------------------------------------- 2023-04-06 05:25:34,483 epoch 3 - iter 265/2650 - loss 0.09016036 - time (sec): 18.01 - samples/sec: 8148.15 - lr: 0.100000 2023-04-06 05:25:51,887 epoch 3 - iter 530/2650 - loss 0.08931367 - time (sec): 35.41 - samples/sec: 8220.78 - lr: 0.100000 2023-04-06 05:26:09,968 epoch 3 - iter 795/2650 - loss 0.08839482 - time (sec): 53.50 - samples/sec: 8181.64 - lr: 0.100000 2023-04-06 05:26:28,005 epoch 3 - iter 1060/2650 - loss 0.08817324 - time (sec): 71.53 - samples/sec: 8160.75 - lr: 0.100000 2023-04-06 05:26:45,992 epoch 3 - iter 1325/2650 - loss 0.08768491 - time (sec): 89.52 - samples/sec: 8172.07 - lr: 0.100000 2023-04-06 05:27:03,756 epoch 3 - iter 1590/2650 - loss 0.08765484 - time (sec): 107.28 - samples/sec: 8166.62 - lr: 0.100000 2023-04-06 05:27:21,483 epoch 3 - iter 1855/2650 - loss 0.08731157 - time (sec): 125.01 - samples/sec: 8194.42 - lr: 0.100000 2023-04-06 05:27:39,122 epoch 3 - iter 2120/2650 - loss 0.08708944 - time (sec): 142.65 - samples/sec: 8205.36 - lr: 0.100000 2023-04-06 05:27:56,887 epoch 3 - iter 2385/2650 - loss 0.08697050 - time (sec): 160.41 - samples/sec: 8215.13 - lr: 0.100000 2023-04-06 05:28:14,143 epoch 3 - iter 2650/2650 - loss 0.08678738 - time (sec): 177.67 - samples/sec: 8230.75 - lr: 0.100000 2023-04-06 05:28:14,144 ---------------------------------------------------------------------------------------------------- 2023-04-06 05:28:14,144 EPOCH 3 done: loss 0.0868 - lr 0.100000 2023-04-06 05:28:14,144 BAD EPOCHS (no improvement): 0 2023-04-06 05:28:14,147 ---------------------------------------------------------------------------------------------------- 2023-04-06 05:28:31,664 epoch 4 - iter 265/2650 - loss 0.08023069 - time (sec): 17.52 - samples/sec: 8314.31 - lr: 0.100000 2023-04-06 05:28:49,602 epoch 4 - iter 530/2650 - loss 0.08018118 - time (sec): 35.45 - samples/sec: 8256.19 - lr: 0.100000 2023-04-06 05:29:07,735 epoch 4 - iter 795/2650 - loss 0.08007137 - time (sec): 53.59 - samples/sec: 8185.95 - lr: 0.100000 2023-04-06 05:29:26,272 epoch 4 - iter 1060/2650 - loss 0.07999830 - time (sec): 72.12 - samples/sec: 8112.67 - lr: 0.100000 2023-04-06 05:29:44,909 epoch 4 - iter 1325/2650 - loss 0.07971874 - time (sec): 90.76 - samples/sec: 8061.70 - lr: 0.100000 2023-04-06 05:30:02,961 epoch 4 - iter 1590/2650 - loss 0.08004345 - time (sec): 108.81 - samples/sec: 8076.13 - lr: 0.100000 2023-04-06 05:30:20,501 epoch 4 - iter 1855/2650 - loss 0.07986212 - time (sec): 126.35 - samples/sec: 8098.54 - lr: 0.100000 2023-04-06 05:30:38,178 epoch 4 - iter 2120/2650 - loss 0.07983681 - time (sec): 144.03 - samples/sec: 8119.87 - lr: 0.100000 2023-04-06 05:30:56,122 epoch 4 - iter 2385/2650 - loss 0.07966297 - time (sec): 161.97 - samples/sec: 8125.08 - lr: 0.100000 2023-04-06 05:31:14,218 epoch 4 - iter 2650/2650 - loss 0.07961284 - time (sec): 180.07 - samples/sec: 8121.04 - lr: 0.100000 2023-04-06 05:31:14,218 ---------------------------------------------------------------------------------------------------- 2023-04-06 05:31:14,218 EPOCH 4 done: loss 0.0796 - lr 0.100000 2023-04-06 05:31:14,218 BAD EPOCHS (no improvement): 0 2023-04-06 05:31:14,222 ---------------------------------------------------------------------------------------------------- 2023-04-06 05:31:32,022 epoch 5 - iter 265/2650 - loss 0.07320589 - time (sec): 17.80 - samples/sec: 8189.93 - lr: 0.100000 2023-04-06 05:31:49,791 epoch 5 - iter 530/2650 - loss 0.07313238 - time (sec): 35.57 - samples/sec: 8169.53 - lr: 0.100000 2023-04-06 05:32:07,680 epoch 5 - iter 795/2650 - loss 0.07416056 - time (sec): 53.46 - samples/sec: 8187.44 - lr: 0.100000 2023-04-06 05:32:25,911 epoch 5 - iter 1060/2650 - loss 0.07474003 - time (sec): 71.69 - samples/sec: 8170.41 - lr: 0.100000 2023-04-06 05:32:44,176 epoch 5 - iter 1325/2650 - loss 0.07431123 - time (sec): 89.95 - samples/sec: 8129.09 - lr: 0.100000 2023-04-06 05:33:02,842 epoch 5 - iter 1590/2650 - loss 0.07488219 - time (sec): 108.62 - samples/sec: 8088.57 - lr: 0.100000 2023-04-06 05:33:20,820 epoch 5 - iter 1855/2650 - loss 0.07484782 - time (sec): 126.60 - samples/sec: 8098.93 - lr: 0.100000 2023-04-06 05:33:38,585 epoch 5 - iter 2120/2650 - loss 0.07470293 - time (sec): 144.36 - samples/sec: 8110.86 - lr: 0.100000 2023-04-06 05:33:56,556 epoch 5 - iter 2385/2650 - loss 0.07459230 - time (sec): 162.33 - samples/sec: 8117.33 - lr: 0.100000 2023-04-06 05:34:14,335 epoch 5 - iter 2650/2650 - loss 0.07486368 - time (sec): 180.11 - samples/sec: 8119.12 - lr: 0.100000 2023-04-06 05:34:14,335 ---------------------------------------------------------------------------------------------------- 2023-04-06 05:34:14,335 EPOCH 5 done: loss 0.0749 - lr 0.100000 2023-04-06 05:34:14,336 BAD EPOCHS (no improvement): 0 2023-04-06 05:34:14,339 ---------------------------------------------------------------------------------------------------- 2023-04-06 05:34:32,296 epoch 6 - iter 265/2650 - loss 0.07120396 - time (sec): 17.96 - samples/sec: 8067.83 - lr: 0.100000 2023-04-06 05:34:50,342 epoch 6 - iter 530/2650 - loss 0.07103244 - time (sec): 36.00 - samples/sec: 8061.98 - lr: 0.100000 2023-04-06 05:35:08,134 epoch 6 - iter 795/2650 - loss 0.07078885 - time (sec): 53.80 - samples/sec: 8148.79 - lr: 0.100000 2023-04-06 05:35:25,878 epoch 6 - iter 1060/2650 - loss 0.07095384 - time (sec): 71.54 - samples/sec: 8144.93 - lr: 0.100000 2023-04-06 05:35:44,025 epoch 6 - iter 1325/2650 - loss 0.07111825 - time (sec): 89.69 - samples/sec: 8133.18 - lr: 0.100000 2023-04-06 05:36:02,214 epoch 6 - iter 1590/2650 - loss 0.07119643 - time (sec): 107.87 - samples/sec: 8133.01 - lr: 0.100000 2023-04-06 05:36:19,979 epoch 6 - iter 1855/2650 - loss 0.07142705 - time (sec): 125.64 - samples/sec: 8143.81 - lr: 0.100000 2023-04-06 05:36:38,049 epoch 6 - iter 2120/2650 - loss 0.07141453 - time (sec): 143.71 - samples/sec: 8144.30 - lr: 0.100000 2023-04-06 05:36:55,630 epoch 6 - iter 2385/2650 - loss 0.07132772 - time (sec): 161.29 - samples/sec: 8150.10 - lr: 0.100000 2023-04-06 05:37:13,763 epoch 6 - iter 2650/2650 - loss 0.07135483 - time (sec): 179.42 - samples/sec: 8150.30 - lr: 0.100000 2023-04-06 05:37:13,764 ---------------------------------------------------------------------------------------------------- 2023-04-06 05:37:13,764 EPOCH 6 done: loss 0.0714 - lr 0.100000 2023-04-06 05:37:13,764 BAD EPOCHS (no improvement): 0 2023-04-06 05:37:13,767 ---------------------------------------------------------------------------------------------------- 2023-04-06 05:37:31,673 epoch 7 - iter 265/2650 - loss 0.06680863 - time (sec): 17.91 - samples/sec: 8143.50 - lr: 0.100000 2023-04-06 05:37:49,253 epoch 7 - iter 530/2650 - loss 0.06767624 - time (sec): 35.49 - samples/sec: 8216.94 - lr: 0.100000 2023-04-06 05:38:07,611 epoch 7 - iter 795/2650 - loss 0.06822992 - time (sec): 53.84 - samples/sec: 8139.74 - lr: 0.100000 2023-04-06 05:38:25,502 epoch 7 - iter 1060/2650 - loss 0.06790439 - time (sec): 71.73 - samples/sec: 8141.83 - lr: 0.100000 2023-04-06 05:38:43,523 epoch 7 - iter 1325/2650 - loss 0.06755988 - time (sec): 89.76 - samples/sec: 8145.88 - lr: 0.100000 2023-04-06 05:39:01,710 epoch 7 - iter 1590/2650 - loss 0.06752439 - time (sec): 107.94 - samples/sec: 8133.30 - lr: 0.100000 2023-04-06 05:39:19,649 epoch 7 - iter 1855/2650 - loss 0.06817412 - time (sec): 125.88 - samples/sec: 8128.05 - lr: 0.100000 2023-04-06 05:39:37,654 epoch 7 - iter 2120/2650 - loss 0.06811943 - time (sec): 143.89 - samples/sec: 8133.76 - lr: 0.100000 2023-04-06 05:39:55,900 epoch 7 - iter 2385/2650 - loss 0.06824145 - time (sec): 162.13 - samples/sec: 8129.32 - lr: 0.100000 2023-04-06 05:40:13,576 epoch 7 - iter 2650/2650 - loss 0.06858704 - time (sec): 179.81 - samples/sec: 8132.88 - lr: 0.100000 2023-04-06 05:40:13,576 ---------------------------------------------------------------------------------------------------- 2023-04-06 05:40:13,576 EPOCH 7 done: loss 0.0686 - lr 0.100000 2023-04-06 05:40:13,576 BAD EPOCHS (no improvement): 0 2023-04-06 05:40:13,580 ---------------------------------------------------------------------------------------------------- 2023-04-06 05:40:30,944 epoch 8 - iter 265/2650 - loss 0.06502864 - time (sec): 17.36 - samples/sec: 8308.66 - lr: 0.100000 2023-04-06 05:40:49,210 epoch 8 - iter 530/2650 - loss 0.06551519 - time (sec): 35.63 - samples/sec: 8182.74 - lr: 0.100000 2023-04-06 05:41:07,497 epoch 8 - iter 795/2650 - loss 0.06596298 - time (sec): 53.92 - samples/sec: 8121.30 - lr: 0.100000 2023-04-06 05:41:25,456 epoch 8 - iter 1060/2650 - loss 0.06578493 - time (sec): 71.88 - samples/sec: 8129.30 - lr: 0.100000 2023-04-06 05:41:43,993 epoch 8 - iter 1325/2650 - loss 0.06583074 - time (sec): 90.41 - samples/sec: 8082.74 - lr: 0.100000 2023-04-06 05:42:02,848 epoch 8 - iter 1590/2650 - loss 0.06592974 - time (sec): 109.27 - samples/sec: 8032.68 - lr: 0.100000 2023-04-06 05:42:21,102 epoch 8 - iter 1855/2650 - loss 0.06587642 - time (sec): 127.52 - samples/sec: 8030.82 - lr: 0.100000 2023-04-06 05:42:38,863 epoch 8 - iter 2120/2650 - loss 0.06591901 - time (sec): 145.28 - samples/sec: 8069.74 - lr: 0.100000 2023-04-06 05:42:56,357 epoch 8 - iter 2385/2650 - loss 0.06583688 - time (sec): 162.78 - samples/sec: 8088.64 - lr: 0.100000 2023-04-06 05:43:13,691 epoch 8 - iter 2650/2650 - loss 0.06588370 - time (sec): 180.11 - samples/sec: 8119.20 - lr: 0.100000 2023-04-06 05:43:13,692 ---------------------------------------------------------------------------------------------------- 2023-04-06 05:43:13,692 EPOCH 8 done: loss 0.0659 - lr 0.100000 2023-04-06 05:43:13,692 BAD EPOCHS (no improvement): 0 2023-04-06 05:43:13,695 ---------------------------------------------------------------------------------------------------- 2023-04-06 05:43:31,219 epoch 9 - iter 265/2650 - loss 0.06301666 - time (sec): 17.52 - samples/sec: 8250.97 - lr: 0.100000 2023-04-06 05:43:49,035 epoch 9 - iter 530/2650 - loss 0.06188372 - time (sec): 35.34 - samples/sec: 8213.05 - lr: 0.100000 2023-04-06 05:44:06,351 epoch 9 - iter 795/2650 - loss 0.06326915 - time (sec): 52.66 - samples/sec: 8247.34 - lr: 0.100000 2023-04-06 05:44:24,287 epoch 9 - iter 1060/2650 - loss 0.06316765 - time (sec): 70.59 - samples/sec: 8240.01 - lr: 0.100000 2023-04-06 05:44:41,878 epoch 9 - iter 1325/2650 - loss 0.06312843 - time (sec): 88.18 - samples/sec: 8256.35 - lr: 0.100000 2023-04-06 05:44:59,781 epoch 9 - iter 1590/2650 - loss 0.06365554 - time (sec): 106.09 - samples/sec: 8245.92 - lr: 0.100000 2023-04-06 05:45:17,557 epoch 9 - iter 1855/2650 - loss 0.06377379 - time (sec): 123.86 - samples/sec: 8240.51 - lr: 0.100000 2023-04-06 05:45:35,371 epoch 9 - iter 2120/2650 - loss 0.06373152 - time (sec): 141.68 - samples/sec: 8247.99 - lr: 0.100000 2023-04-06 05:45:53,413 epoch 9 - iter 2385/2650 - loss 0.06409843 - time (sec): 159.72 - samples/sec: 8230.11 - lr: 0.100000 2023-04-06 05:46:11,171 epoch 9 - iter 2650/2650 - loss 0.06430242 - time (sec): 177.48 - samples/sec: 8239.82 - lr: 0.100000 2023-04-06 05:46:11,171 ---------------------------------------------------------------------------------------------------- 2023-04-06 05:46:11,171 EPOCH 9 done: loss 0.0643 - lr 0.100000 2023-04-06 05:46:11,171 BAD EPOCHS (no improvement): 0 2023-04-06 05:46:11,177 ---------------------------------------------------------------------------------------------------- 2023-04-06 05:46:29,989 epoch 10 - iter 265/2650 - loss 0.06211111 - time (sec): 18.81 - samples/sec: 7715.01 - lr: 0.100000 2023-04-06 05:46:48,309 epoch 10 - iter 530/2650 - loss 0.06170219 - time (sec): 37.13 - samples/sec: 7862.78 - lr: 0.100000 2023-04-06 05:47:06,052 epoch 10 - iter 795/2650 - loss 0.06141386 - time (sec): 54.87 - samples/sec: 7978.77 - lr: 0.100000 2023-04-06 05:47:23,951 epoch 10 - iter 1060/2650 - loss 0.06151682 - time (sec): 72.77 - samples/sec: 8026.24 - lr: 0.100000 2023-04-06 05:47:41,756 epoch 10 - iter 1325/2650 - loss 0.06141383 - time (sec): 90.58 - samples/sec: 8069.27 - lr: 0.100000 2023-04-06 05:47:59,621 epoch 10 - iter 1590/2650 - loss 0.06176690 - time (sec): 108.44 - samples/sec: 8091.58 - lr: 0.100000 2023-04-06 05:48:17,213 epoch 10 - iter 1855/2650 - loss 0.06204404 - time (sec): 126.04 - samples/sec: 8115.20 - lr: 0.100000 2023-04-06 05:48:35,135 epoch 10 - iter 2120/2650 - loss 0.06209118 - time (sec): 143.96 - samples/sec: 8131.66 - lr: 0.100000 2023-04-06 05:48:52,842 epoch 10 - iter 2385/2650 - loss 0.06227366 - time (sec): 161.66 - samples/sec: 8155.82 - lr: 0.100000 2023-04-06 05:49:10,082 epoch 10 - iter 2650/2650 - loss 0.06236078 - time (sec): 178.90 - samples/sec: 8173.99 - lr: 0.100000 2023-04-06 05:49:10,082 ---------------------------------------------------------------------------------------------------- 2023-04-06 05:49:10,082 EPOCH 10 done: loss 0.0624 - lr 0.100000 2023-04-06 05:49:10,082 BAD EPOCHS (no improvement): 0 2023-04-06 05:49:10,086 ---------------------------------------------------------------------------------------------------- 2023-04-06 05:49:27,752 epoch 11 - iter 265/2650 - loss 0.06031236 - time (sec): 17.67 - samples/sec: 8244.40 - lr: 0.100000 2023-04-06 05:49:45,384 epoch 11 - iter 530/2650 - loss 0.06078448 - time (sec): 35.30 - samples/sec: 8260.48 - lr: 0.100000 2023-04-06 05:50:03,191 epoch 11 - iter 795/2650 - loss 0.06100689 - time (sec): 53.11 - samples/sec: 8208.91 - lr: 0.100000 2023-04-06 05:50:20,575 epoch 11 - iter 1060/2650 - loss 0.06144313 - time (sec): 70.49 - samples/sec: 8258.15 - lr: 0.100000 2023-04-06 05:50:38,803 epoch 11 - iter 1325/2650 - loss 0.06073678 - time (sec): 88.72 - samples/sec: 8222.84 - lr: 0.100000 2023-04-06 05:50:56,561 epoch 11 - iter 1590/2650 - loss 0.06096846 - time (sec): 106.47 - samples/sec: 8232.41 - lr: 0.100000 2023-04-06 05:51:14,121 epoch 11 - iter 1855/2650 - loss 0.06101897 - time (sec): 124.03 - samples/sec: 8252.53 - lr: 0.100000 2023-04-06 05:51:31,705 epoch 11 - iter 2120/2650 - loss 0.06107109 - time (sec): 141.62 - samples/sec: 8255.46 - lr: 0.100000 2023-04-06 05:51:49,899 epoch 11 - iter 2385/2650 - loss 0.06116282 - time (sec): 159.81 - samples/sec: 8230.00 - lr: 0.100000 2023-04-06 05:52:08,210 epoch 11 - iter 2650/2650 - loss 0.06128175 - time (sec): 178.12 - samples/sec: 8209.81 - lr: 0.100000 2023-04-06 05:52:08,210 ---------------------------------------------------------------------------------------------------- 2023-04-06 05:52:08,210 EPOCH 11 done: loss 0.0613 - lr 0.100000 2023-04-06 05:52:08,210 BAD EPOCHS (no improvement): 0 2023-04-06 05:52:08,214 ---------------------------------------------------------------------------------------------------- 2023-04-06 05:52:26,751 epoch 12 - iter 265/2650 - loss 0.05782786 - time (sec): 18.54 - samples/sec: 7917.54 - lr: 0.100000 2023-04-06 05:52:44,852 epoch 12 - iter 530/2650 - loss 0.05816596 - time (sec): 36.64 - samples/sec: 7990.81 - lr: 0.100000 2023-04-06 05:53:03,226 epoch 12 - iter 795/2650 - loss 0.05862835 - time (sec): 55.01 - samples/sec: 7987.87 - lr: 0.100000 2023-04-06 05:53:55,657 epoch 12 - iter 1060/2650 - loss 0.05857434 - time (sec): 107.44 - samples/sec: 5452.52 - lr: 0.100000 2023-04-06 05:54:14,487 epoch 12 - iter 1325/2650 - loss 0.05899821 - time (sec): 126.27 - samples/sec: 5798.18 - lr: 0.100000 2023-04-06 05:54:33,241 epoch 12 - iter 1590/2650 - loss 0.05927006 - time (sec): 145.03 - samples/sec: 6061.24 - lr: 0.100000 2023-04-06 05:54:52,131 epoch 12 - iter 1855/2650 - loss 0.05943934 - time (sec): 163.92 - samples/sec: 6257.91 - lr: 0.100000 2023-04-06 05:55:11,042 epoch 12 - iter 2120/2650 - loss 0.05959254 - time (sec): 182.83 - samples/sec: 6406.17 - lr: 0.100000 2023-04-06 05:55:29,548 epoch 12 - iter 2385/2650 - loss 0.05958965 - time (sec): 201.33 - samples/sec: 6540.82 - lr: 0.100000 2023-04-06 05:55:48,222 epoch 12 - iter 2650/2650 - loss 0.05957266 - time (sec): 220.01 - samples/sec: 6646.85 - lr: 0.100000 2023-04-06 05:55:48,222 ---------------------------------------------------------------------------------------------------- 2023-04-06 05:55:48,222 EPOCH 12 done: loss 0.0596 - lr 0.100000 2023-04-06 05:55:48,222 BAD EPOCHS (no improvement): 0 2023-04-06 05:55:48,226 ---------------------------------------------------------------------------------------------------- 2023-04-06 05:56:06,922 epoch 13 - iter 265/2650 - loss 0.05755710 - time (sec): 18.70 - samples/sec: 7870.17 - lr: 0.100000 2023-04-06 05:56:25,532 epoch 13 - iter 530/2650 - loss 0.05804948 - time (sec): 37.31 - samples/sec: 7893.29 - lr: 0.100000 2023-04-06 05:56:43,974 epoch 13 - iter 795/2650 - loss 0.05836537 - time (sec): 55.75 - samples/sec: 7907.79 - lr: 0.100000 2023-04-06 05:57:02,035 epoch 13 - iter 1060/2650 - loss 0.05783025 - time (sec): 73.81 - samples/sec: 7939.32 - lr: 0.100000 2023-04-06 05:57:20,423 epoch 13 - iter 1325/2650 - loss 0.05751979 - time (sec): 92.20 - samples/sec: 7950.33 - lr: 0.100000 2023-04-06 05:57:38,623 epoch 13 - iter 1590/2650 - loss 0.05781586 - time (sec): 110.40 - samples/sec: 7961.23 - lr: 0.100000 2023-04-06 05:57:57,359 epoch 13 - iter 1855/2650 - loss 0.05776421 - time (sec): 129.13 - samples/sec: 7940.68 - lr: 0.100000 2023-04-06 05:58:16,175 epoch 13 - iter 2120/2650 - loss 0.05796821 - time (sec): 147.95 - samples/sec: 7923.73 - lr: 0.100000 2023-04-06 05:58:34,229 epoch 13 - iter 2385/2650 - loss 0.05826556 - time (sec): 166.00 - samples/sec: 7932.36 - lr: 0.100000 2023-04-06 05:58:52,744 epoch 13 - iter 2650/2650 - loss 0.05839999 - time (sec): 184.52 - samples/sec: 7925.32 - lr: 0.100000 2023-04-06 05:58:52,744 ---------------------------------------------------------------------------------------------------- 2023-04-06 05:58:52,744 EPOCH 13 done: loss 0.0584 - lr 0.100000 2023-04-06 05:58:52,744 BAD EPOCHS (no improvement): 0 2023-04-06 05:58:52,786 ---------------------------------------------------------------------------------------------------- 2023-04-06 05:59:11,011 epoch 14 - iter 265/2650 - loss 0.05737967 - time (sec): 18.22 - samples/sec: 7936.14 - lr: 0.100000 2023-04-06 05:59:29,728 epoch 14 - iter 530/2650 - loss 0.05616992 - time (sec): 36.94 - samples/sec: 7879.45 - lr: 0.100000 2023-04-06 05:59:48,311 epoch 14 - iter 795/2650 - loss 0.05636806 - time (sec): 55.52 - samples/sec: 7895.21 - lr: 0.100000 2023-04-06 06:00:06,550 epoch 14 - iter 1060/2650 - loss 0.05682585 - time (sec): 73.76 - samples/sec: 7922.60 - lr: 0.100000 2023-04-06 06:00:24,917 epoch 14 - iter 1325/2650 - loss 0.05718452 - time (sec): 92.13 - samples/sec: 7914.97 - lr: 0.100000 2023-04-06 06:00:43,512 epoch 14 - iter 1590/2650 - loss 0.05715114 - time (sec): 110.73 - samples/sec: 7901.49 - lr: 0.100000 2023-04-06 06:01:02,315 epoch 14 - iter 1855/2650 - loss 0.05743515 - time (sec): 129.53 - samples/sec: 7893.42 - lr: 0.100000 2023-04-06 06:01:20,589 epoch 14 - iter 2120/2650 - loss 0.05759865 - time (sec): 147.80 - samples/sec: 7905.43 - lr: 0.100000 2023-04-06 06:01:38,886 epoch 14 - iter 2385/2650 - loss 0.05765435 - time (sec): 166.10 - samples/sec: 7919.07 - lr: 0.100000 2023-04-06 06:01:57,690 epoch 14 - iter 2650/2650 - loss 0.05758086 - time (sec): 184.90 - samples/sec: 7908.78 - lr: 0.100000 2023-04-06 06:01:57,690 ---------------------------------------------------------------------------------------------------- 2023-04-06 06:01:57,690 EPOCH 14 done: loss 0.0576 - lr 0.100000 2023-04-06 06:01:57,690 BAD EPOCHS (no improvement): 0 2023-04-06 06:01:57,695 ---------------------------------------------------------------------------------------------------- 2023-04-06 06:02:16,010 epoch 15 - iter 265/2650 - loss 0.05520686 - time (sec): 18.31 - samples/sec: 7876.42 - lr: 0.100000 2023-04-06 06:02:34,179 epoch 15 - iter 530/2650 - loss 0.05583117 - time (sec): 36.48 - samples/sec: 7945.05 - lr: 0.100000 2023-04-06 06:02:52,625 epoch 15 - iter 795/2650 - loss 0.05589579 - time (sec): 54.93 - samples/sec: 7963.38 - lr: 0.100000 2023-04-06 06:03:10,651 epoch 15 - iter 1060/2650 - loss 0.05624005 - time (sec): 72.96 - samples/sec: 7998.64 - lr: 0.100000 2023-04-06 06:03:29,250 epoch 15 - iter 1325/2650 - loss 0.05635941 - time (sec): 91.55 - samples/sec: 7984.03 - lr: 0.100000 2023-04-06 06:03:47,623 epoch 15 - iter 1590/2650 - loss 0.05655763 - time (sec): 109.93 - samples/sec: 7976.07 - lr: 0.100000 2023-04-06 06:04:05,857 epoch 15 - iter 1855/2650 - loss 0.05675457 - time (sec): 128.16 - samples/sec: 7984.08 - lr: 0.100000 2023-04-06 06:04:24,089 epoch 15 - iter 2120/2650 - loss 0.05676953 - time (sec): 146.39 - samples/sec: 7977.43 - lr: 0.100000 2023-04-06 06:04:42,699 epoch 15 - iter 2385/2650 - loss 0.05688931 - time (sec): 165.00 - samples/sec: 7975.34 - lr: 0.100000 2023-04-06 06:05:01,270 epoch 15 - iter 2650/2650 - loss 0.05704586 - time (sec): 183.58 - samples/sec: 7966.02 - lr: 0.100000 2023-04-06 06:05:01,270 ---------------------------------------------------------------------------------------------------- 2023-04-06 06:05:01,270 EPOCH 15 done: loss 0.0570 - lr 0.100000 2023-04-06 06:05:01,270 BAD EPOCHS (no improvement): 0 2023-04-06 06:05:01,274 ---------------------------------------------------------------------------------------------------- 2023-04-06 06:05:19,876 epoch 16 - iter 265/2650 - loss 0.05386714 - time (sec): 18.60 - samples/sec: 7900.38 - lr: 0.100000 2023-04-06 06:05:38,423 epoch 16 - iter 530/2650 - loss 0.05475834 - time (sec): 37.15 - samples/sec: 7900.88 - lr: 0.100000 2023-04-06 06:05:56,677 epoch 16 - iter 795/2650 - loss 0.05540631 - time (sec): 55.40 - samples/sec: 7942.23 - lr: 0.100000 2023-04-06 06:06:14,987 epoch 16 - iter 1060/2650 - loss 0.05545007 - time (sec): 73.71 - samples/sec: 7953.81 - lr: 0.100000 2023-04-06 06:06:33,730 epoch 16 - iter 1325/2650 - loss 0.05540327 - time (sec): 92.46 - samples/sec: 7923.13 - lr: 0.100000 2023-04-06 06:06:52,678 epoch 16 - iter 1590/2650 - loss 0.05579498 - time (sec): 111.40 - samples/sec: 7874.99 - lr: 0.100000 2023-04-06 06:07:11,331 epoch 16 - iter 1855/2650 - loss 0.05575775 - time (sec): 130.06 - samples/sec: 7876.01 - lr: 0.100000 2023-04-06 06:07:29,822 epoch 16 - iter 2120/2650 - loss 0.05551833 - time (sec): 148.55 - samples/sec: 7881.05 - lr: 0.100000 2023-04-06 06:07:48,624 epoch 16 - iter 2385/2650 - loss 0.05544533 - time (sec): 167.35 - samples/sec: 7863.32 - lr: 0.100000 2023-04-06 06:08:07,348 epoch 16 - iter 2650/2650 - loss 0.05576982 - time (sec): 186.07 - samples/sec: 7859.04 - lr: 0.100000 2023-04-06 06:08:07,348 ---------------------------------------------------------------------------------------------------- 2023-04-06 06:08:07,348 EPOCH 16 done: loss 0.0558 - lr 0.100000 2023-04-06 06:08:07,348 BAD EPOCHS (no improvement): 0 2023-04-06 06:08:07,351 ---------------------------------------------------------------------------------------------------- 2023-04-06 06:08:25,938 epoch 17 - iter 265/2650 - loss 0.05474174 - time (sec): 18.59 - samples/sec: 7839.34 - lr: 0.100000 2023-04-06 06:08:44,449 epoch 17 - iter 530/2650 - loss 0.05398932 - time (sec): 37.10 - samples/sec: 7882.88 - lr: 0.100000 2023-04-06 06:09:02,879 epoch 17 - iter 795/2650 - loss 0.05451603 - time (sec): 55.53 - samples/sec: 7890.56 - lr: 0.100000 2023-04-06 06:09:21,777 epoch 17 - iter 1060/2650 - loss 0.05494228 - time (sec): 74.43 - samples/sec: 7842.19 - lr: 0.100000 2023-04-06 06:09:40,102 epoch 17 - iter 1325/2650 - loss 0.05501016 - time (sec): 92.75 - samples/sec: 7858.55 - lr: 0.100000 2023-04-06 06:09:58,805 epoch 17 - iter 1590/2650 - loss 0.05497206 - time (sec): 111.45 - samples/sec: 7855.59 - lr: 0.100000 2023-04-06 06:10:18,009 epoch 17 - iter 1855/2650 - loss 0.05522482 - time (sec): 130.66 - samples/sec: 7824.43 - lr: 0.100000 2023-04-06 06:10:36,852 epoch 17 - iter 2120/2650 - loss 0.05533854 - time (sec): 149.50 - samples/sec: 7823.84 - lr: 0.100000 2023-04-06 06:10:55,300 epoch 17 - iter 2385/2650 - loss 0.05533662 - time (sec): 167.95 - samples/sec: 7835.63 - lr: 0.100000 2023-04-06 06:11:13,717 epoch 17 - iter 2650/2650 - loss 0.05542738 - time (sec): 186.37 - samples/sec: 7846.73 - lr: 0.100000 2023-04-06 06:11:13,717 ---------------------------------------------------------------------------------------------------- 2023-04-06 06:11:13,717 EPOCH 17 done: loss 0.0554 - lr 0.100000 2023-04-06 06:11:13,717 BAD EPOCHS (no improvement): 0 2023-04-06 06:11:13,720 ---------------------------------------------------------------------------------------------------- 2023-04-06 06:11:32,420 epoch 18 - iter 265/2650 - loss 0.05231873 - time (sec): 18.70 - samples/sec: 7802.39 - lr: 0.100000 2023-04-06 06:11:51,275 epoch 18 - iter 530/2650 - loss 0.05297770 - time (sec): 37.55 - samples/sec: 7785.37 - lr: 0.100000 2023-04-06 06:12:09,717 epoch 18 - iter 795/2650 - loss 0.05391244 - time (sec): 56.00 - samples/sec: 7838.15 - lr: 0.100000 2023-04-06 06:12:28,036 epoch 18 - iter 1060/2650 - loss 0.05374249 - time (sec): 74.32 - samples/sec: 7871.76 - lr: 0.100000 2023-04-06 06:12:46,545 epoch 18 - iter 1325/2650 - loss 0.05376008 - time (sec): 92.82 - samples/sec: 7881.88 - lr: 0.100000 2023-04-06 06:13:04,663 epoch 18 - iter 1590/2650 - loss 0.05334191 - time (sec): 110.94 - samples/sec: 7911.87 - lr: 0.100000 2023-04-06 06:13:23,259 epoch 18 - iter 1855/2650 - loss 0.05352741 - time (sec): 129.54 - samples/sec: 7903.42 - lr: 0.100000 2023-04-06 06:13:41,790 epoch 18 - iter 2120/2650 - loss 0.05369582 - time (sec): 148.07 - samples/sec: 7894.93 - lr: 0.100000 2023-04-06 06:14:00,140 epoch 18 - iter 2385/2650 - loss 0.05394263 - time (sec): 166.42 - samples/sec: 7913.75 - lr: 0.100000 2023-04-06 06:14:18,140 epoch 18 - iter 2650/2650 - loss 0.05417085 - time (sec): 184.42 - samples/sec: 7929.53 - lr: 0.100000 2023-04-06 06:14:18,141 ---------------------------------------------------------------------------------------------------- 2023-04-06 06:14:18,141 EPOCH 18 done: loss 0.0542 - lr 0.100000 2023-04-06 06:14:18,141 BAD EPOCHS (no improvement): 0 2023-04-06 06:14:18,144 ---------------------------------------------------------------------------------------------------- 2023-04-06 06:14:36,794 epoch 19 - iter 265/2650 - loss 0.05012973 - time (sec): 18.65 - samples/sec: 7934.34 - lr: 0.100000 2023-04-06 06:14:55,147 epoch 19 - iter 530/2650 - loss 0.05111135 - time (sec): 37.00 - samples/sec: 7961.42 - lr: 0.100000 2023-04-06 06:15:13,466 epoch 19 - iter 795/2650 - loss 0.05232237 - time (sec): 55.32 - samples/sec: 7992.58 - lr: 0.100000 2023-04-06 06:15:31,828 epoch 19 - iter 1060/2650 - loss 0.05253988 - time (sec): 73.68 - samples/sec: 7994.40 - lr: 0.100000 2023-04-06 06:15:49,786 epoch 19 - iter 1325/2650 - loss 0.05245113 - time (sec): 91.64 - samples/sec: 8013.56 - lr: 0.100000 2023-04-06 06:16:08,057 epoch 19 - iter 1590/2650 - loss 0.05271299 - time (sec): 109.91 - samples/sec: 7996.51 - lr: 0.100000 2023-04-06 06:16:26,795 epoch 19 - iter 1855/2650 - loss 0.05290452 - time (sec): 128.65 - samples/sec: 7975.42 - lr: 0.100000 2023-04-06 06:16:45,347 epoch 19 - iter 2120/2650 - loss 0.05298950 - time (sec): 147.20 - samples/sec: 7957.47 - lr: 0.100000 2023-04-06 06:17:03,341 epoch 19 - iter 2385/2650 - loss 0.05337646 - time (sec): 165.20 - samples/sec: 7968.79 - lr: 0.100000 2023-04-06 06:17:22,221 epoch 19 - iter 2650/2650 - loss 0.05349366 - time (sec): 184.08 - samples/sec: 7944.27 - lr: 0.100000 2023-04-06 06:17:22,222 ---------------------------------------------------------------------------------------------------- 2023-04-06 06:17:22,222 EPOCH 19 done: loss 0.0535 - lr 0.100000 2023-04-06 06:17:22,222 BAD EPOCHS (no improvement): 0 2023-04-06 06:17:22,224 ---------------------------------------------------------------------------------------------------- 2023-04-06 06:17:41,109 epoch 20 - iter 265/2650 - loss 0.05082597 - time (sec): 18.88 - samples/sec: 7779.17 - lr: 0.100000 2023-04-06 06:17:59,376 epoch 20 - iter 530/2650 - loss 0.05164230 - time (sec): 37.15 - samples/sec: 7841.72 - lr: 0.100000 2023-04-06 06:18:18,471 epoch 20 - iter 795/2650 - loss 0.05180971 - time (sec): 56.25 - samples/sec: 7802.03 - lr: 0.100000 2023-04-06 06:18:36,697 epoch 20 - iter 1060/2650 - loss 0.05166418 - time (sec): 74.47 - samples/sec: 7859.81 - lr: 0.100000 2023-04-06 06:18:55,237 epoch 20 - iter 1325/2650 - loss 0.05143689 - time (sec): 93.01 - samples/sec: 7855.90 - lr: 0.100000 2023-04-06 06:19:13,989 epoch 20 - iter 1590/2650 - loss 0.05216698 - time (sec): 111.76 - samples/sec: 7847.59 - lr: 0.100000 2023-04-06 06:19:32,931 epoch 20 - iter 1855/2650 - loss 0.05225846 - time (sec): 130.71 - samples/sec: 7832.69 - lr: 0.100000 2023-04-06 06:19:51,437 epoch 20 - iter 2120/2650 - loss 0.05254569 - time (sec): 149.21 - samples/sec: 7841.53 - lr: 0.100000 2023-04-06 06:20:10,464 epoch 20 - iter 2385/2650 - loss 0.05265991 - time (sec): 168.24 - samples/sec: 7829.00 - lr: 0.100000 2023-04-06 06:20:29,080 epoch 20 - iter 2650/2650 - loss 0.05278548 - time (sec): 186.86 - samples/sec: 7826.17 - lr: 0.100000 2023-04-06 06:20:29,080 ---------------------------------------------------------------------------------------------------- 2023-04-06 06:20:29,080 EPOCH 20 done: loss 0.0528 - lr 0.100000 2023-04-06 06:20:29,080 BAD EPOCHS (no improvement): 0 2023-04-06 06:20:29,083 ---------------------------------------------------------------------------------------------------- 2023-04-06 06:20:48,050 epoch 21 - iter 265/2650 - loss 0.05090119 - time (sec): 18.97 - samples/sec: 7707.79 - lr: 0.100000 2023-04-06 06:21:06,132 epoch 21 - iter 530/2650 - loss 0.05139792 - time (sec): 37.05 - samples/sec: 7846.63 - lr: 0.100000 2023-04-06 06:21:24,423 epoch 21 - iter 795/2650 - loss 0.05144333 - time (sec): 55.34 - samples/sec: 7903.15 - lr: 0.100000 2023-04-06 06:21:43,007 epoch 21 - iter 1060/2650 - loss 0.05160875 - time (sec): 73.92 - samples/sec: 7915.17 - lr: 0.100000 2023-04-06 06:22:01,586 epoch 21 - iter 1325/2650 - loss 0.05167945 - time (sec): 92.50 - samples/sec: 7905.37 - lr: 0.100000 2023-04-06 06:22:19,665 epoch 21 - iter 1590/2650 - loss 0.05160901 - time (sec): 110.58 - samples/sec: 7911.31 - lr: 0.100000 2023-04-06 06:22:38,722 epoch 21 - iter 1855/2650 - loss 0.05155517 - time (sec): 129.64 - samples/sec: 7874.78 - lr: 0.100000 2023-04-06 06:22:57,317 epoch 21 - iter 2120/2650 - loss 0.05180732 - time (sec): 148.23 - samples/sec: 7875.47 - lr: 0.100000 2023-04-06 06:23:16,940 epoch 21 - iter 2385/2650 - loss 0.05198217 - time (sec): 167.86 - samples/sec: 7839.65 - lr: 0.100000 2023-04-06 06:23:35,585 epoch 21 - iter 2650/2650 - loss 0.05228397 - time (sec): 186.50 - samples/sec: 7841.00 - lr: 0.100000 2023-04-06 06:23:35,585 ---------------------------------------------------------------------------------------------------- 2023-04-06 06:23:35,585 EPOCH 21 done: loss 0.0523 - lr 0.100000 2023-04-06 06:23:35,585 BAD EPOCHS (no improvement): 0 2023-04-06 06:23:35,588 ---------------------------------------------------------------------------------------------------- 2023-04-06 06:23:55,017 epoch 22 - iter 265/2650 - loss 0.04889619 - time (sec): 19.43 - samples/sec: 7628.42 - lr: 0.100000 2023-04-06 06:24:14,006 epoch 22 - iter 530/2650 - loss 0.05010925 - time (sec): 38.42 - samples/sec: 7682.44 - lr: 0.100000 2023-04-06 06:24:31,951 epoch 22 - iter 795/2650 - loss 0.05059762 - time (sec): 56.36 - samples/sec: 7813.03 - lr: 0.100000 2023-04-06 06:24:50,639 epoch 22 - iter 1060/2650 - loss 0.05103498 - time (sec): 75.05 - samples/sec: 7829.26 - lr: 0.100000 2023-04-06 06:25:08,490 epoch 22 - iter 1325/2650 - loss 0.05126495 - time (sec): 92.90 - samples/sec: 7868.68 - lr: 0.100000 2023-04-06 06:25:26,651 epoch 22 - iter 1590/2650 - loss 0.05171217 - time (sec): 111.06 - samples/sec: 7879.64 - lr: 0.100000 2023-04-06 06:25:45,342 epoch 22 - iter 1855/2650 - loss 0.05143837 - time (sec): 129.75 - samples/sec: 7879.19 - lr: 0.100000 2023-04-06 06:26:04,080 epoch 22 - iter 2120/2650 - loss 0.05145072 - time (sec): 148.49 - samples/sec: 7873.81 - lr: 0.100000 2023-04-06 06:26:22,614 epoch 22 - iter 2385/2650 - loss 0.05173143 - time (sec): 167.03 - samples/sec: 7882.00 - lr: 0.100000 2023-04-06 06:26:40,987 epoch 22 - iter 2650/2650 - loss 0.05188801 - time (sec): 185.40 - samples/sec: 7887.65 - lr: 0.100000 2023-04-06 06:26:40,988 ---------------------------------------------------------------------------------------------------- 2023-04-06 06:26:40,988 EPOCH 22 done: loss 0.0519 - lr 0.100000 2023-04-06 06:26:40,988 BAD EPOCHS (no improvement): 0 2023-04-06 06:26:40,991 ---------------------------------------------------------------------------------------------------- 2023-04-06 06:26:59,408 epoch 23 - iter 265/2650 - loss 0.04932590 - time (sec): 18.42 - samples/sec: 7980.65 - lr: 0.100000 2023-04-06 06:27:17,691 epoch 23 - iter 530/2650 - loss 0.05038041 - time (sec): 36.70 - samples/sec: 7996.66 - lr: 0.100000 2023-04-06 06:27:36,051 epoch 23 - iter 795/2650 - loss 0.05060234 - time (sec): 55.06 - samples/sec: 7974.50 - lr: 0.100000 2023-04-06 06:27:53,930 epoch 23 - iter 1060/2650 - loss 0.05041945 - time (sec): 72.94 - samples/sec: 8012.28 - lr: 0.100000 2023-04-06 06:28:12,409 epoch 23 - iter 1325/2650 - loss 0.05063988 - time (sec): 91.42 - samples/sec: 7995.50 - lr: 0.100000 2023-04-06 06:28:31,663 epoch 23 - iter 1590/2650 - loss 0.05081699 - time (sec): 110.67 - samples/sec: 7933.13 - lr: 0.100000 2023-04-06 06:28:50,148 epoch 23 - iter 1855/2650 - loss 0.05116787 - time (sec): 129.16 - samples/sec: 7927.07 - lr: 0.100000 2023-04-06 06:29:09,300 epoch 23 - iter 2120/2650 - loss 0.05149131 - time (sec): 148.31 - samples/sec: 7892.14 - lr: 0.100000 2023-04-06 06:29:27,621 epoch 23 - iter 2385/2650 - loss 0.05155796 - time (sec): 166.63 - samples/sec: 7894.40 - lr: 0.100000 2023-04-06 06:29:45,999 epoch 23 - iter 2650/2650 - loss 0.05172852 - time (sec): 185.01 - samples/sec: 7904.32 - lr: 0.100000 2023-04-06 06:29:45,999 ---------------------------------------------------------------------------------------------------- 2023-04-06 06:29:46,000 EPOCH 23 done: loss 0.0517 - lr 0.100000 2023-04-06 06:29:46,000 BAD EPOCHS (no improvement): 0 2023-04-06 06:29:46,003 ---------------------------------------------------------------------------------------------------- 2023-04-06 06:30:04,315 epoch 24 - iter 265/2650 - loss 0.05005291 - time (sec): 18.31 - samples/sec: 7935.98 - lr: 0.100000 2023-04-06 06:30:22,977 epoch 24 - iter 530/2650 - loss 0.04978158 - time (sec): 36.97 - samples/sec: 7939.78 - lr: 0.100000 2023-04-06 06:30:41,338 epoch 24 - iter 795/2650 - loss 0.04999644 - time (sec): 55.33 - samples/sec: 7963.88 - lr: 0.100000 2023-04-06 06:30:59,766 epoch 24 - iter 1060/2650 - loss 0.05001453 - time (sec): 73.76 - samples/sec: 7948.62 - lr: 0.100000 2023-04-06 06:31:17,684 epoch 24 - iter 1325/2650 - loss 0.05010837 - time (sec): 91.68 - samples/sec: 7977.32 - lr: 0.100000 2023-04-06 06:31:35,685 epoch 24 - iter 1590/2650 - loss 0.05046948 - time (sec): 109.68 - samples/sec: 7995.11 - lr: 0.100000 2023-04-06 06:31:53,885 epoch 24 - iter 1855/2650 - loss 0.05070964 - time (sec): 127.88 - samples/sec: 7997.66 - lr: 0.100000 2023-04-06 06:32:12,839 epoch 24 - iter 2120/2650 - loss 0.05064159 - time (sec): 146.84 - samples/sec: 7959.68 - lr: 0.100000 2023-04-06 06:32:30,904 epoch 24 - iter 2385/2650 - loss 0.05084405 - time (sec): 164.90 - samples/sec: 7971.43 - lr: 0.100000 2023-04-06 06:32:49,036 epoch 24 - iter 2650/2650 - loss 0.05090510 - time (sec): 183.03 - samples/sec: 7989.63 - lr: 0.100000 2023-04-06 06:32:49,036 ---------------------------------------------------------------------------------------------------- 2023-04-06 06:32:49,036 EPOCH 24 done: loss 0.0509 - lr 0.100000 2023-04-06 06:32:49,036 BAD EPOCHS (no improvement): 0 2023-04-06 06:32:49,040 ---------------------------------------------------------------------------------------------------- 2023-04-06 06:33:07,422 epoch 25 - iter 265/2650 - loss 0.04830702 - time (sec): 18.38 - samples/sec: 7951.83 - lr: 0.100000 2023-04-06 06:33:25,871 epoch 25 - iter 530/2650 - loss 0.04833002 - time (sec): 36.83 - samples/sec: 7945.80 - lr: 0.100000 2023-04-06 06:33:44,404 epoch 25 - iter 795/2650 - loss 0.04897133 - time (sec): 55.36 - samples/sec: 7914.60 - lr: 0.100000 2023-04-06 06:34:03,026 epoch 25 - iter 1060/2650 - loss 0.04941054 - time (sec): 73.99 - samples/sec: 7875.05 - lr: 0.100000 2023-04-06 06:34:21,167 epoch 25 - iter 1325/2650 - loss 0.04955150 - time (sec): 92.13 - samples/sec: 7912.37 - lr: 0.100000 2023-04-06 06:34:39,264 epoch 25 - iter 1590/2650 - loss 0.04963391 - time (sec): 110.22 - samples/sec: 7945.64 - lr: 0.100000 2023-04-06 06:34:57,643 epoch 25 - iter 1855/2650 - loss 0.04971431 - time (sec): 128.60 - samples/sec: 7949.81 - lr: 0.100000 2023-04-06 06:35:16,187 epoch 25 - iter 2120/2650 - loss 0.04993489 - time (sec): 147.15 - samples/sec: 7950.62 - lr: 0.100000 2023-04-06 06:35:34,387 epoch 25 - iter 2385/2650 - loss 0.05010125 - time (sec): 165.35 - samples/sec: 7956.14 - lr: 0.100000 2023-04-06 06:35:52,670 epoch 25 - iter 2650/2650 - loss 0.05031966 - time (sec): 183.63 - samples/sec: 7963.63 - lr: 0.100000 2023-04-06 06:35:52,671 ---------------------------------------------------------------------------------------------------- 2023-04-06 06:35:52,671 EPOCH 25 done: loss 0.0503 - lr 0.100000 2023-04-06 06:35:52,671 BAD EPOCHS (no improvement): 0 2023-04-06 06:35:52,674 ---------------------------------------------------------------------------------------------------- 2023-04-06 06:36:11,384 epoch 26 - iter 265/2650 - loss 0.04847671 - time (sec): 18.71 - samples/sec: 7904.07 - lr: 0.100000 2023-04-06 06:36:29,856 epoch 26 - iter 530/2650 - loss 0.04851492 - time (sec): 37.18 - samples/sec: 7919.71 - lr: 0.100000 2023-04-06 06:36:48,115 epoch 26 - iter 795/2650 - loss 0.04921834 - time (sec): 55.44 - samples/sec: 7931.68 - lr: 0.100000 2023-04-06 06:37:06,461 epoch 26 - iter 1060/2650 - loss 0.04931838 - time (sec): 73.79 - samples/sec: 7947.89 - lr: 0.100000 2023-04-06 06:37:24,762 epoch 26 - iter 1325/2650 - loss 0.04912329 - time (sec): 92.09 - samples/sec: 7954.44 - lr: 0.100000 2023-04-06 06:37:42,841 epoch 26 - iter 1590/2650 - loss 0.04951932 - time (sec): 110.17 - samples/sec: 7970.74 - lr: 0.100000 2023-04-06 06:38:01,158 epoch 26 - iter 1855/2650 - loss 0.04960409 - time (sec): 128.48 - samples/sec: 7973.72 - lr: 0.100000 2023-04-06 06:38:19,936 epoch 26 - iter 2120/2650 - loss 0.04984436 - time (sec): 147.26 - samples/sec: 7960.08 - lr: 0.100000 2023-04-06 06:38:37,675 epoch 26 - iter 2385/2650 - loss 0.04997306 - time (sec): 165.00 - samples/sec: 7981.39 - lr: 0.100000 2023-04-06 06:38:56,034 epoch 26 - iter 2650/2650 - loss 0.04997727 - time (sec): 183.36 - samples/sec: 7975.36 - lr: 0.100000 2023-04-06 06:38:56,035 ---------------------------------------------------------------------------------------------------- 2023-04-06 06:38:56,035 EPOCH 26 done: loss 0.0500 - lr 0.100000 2023-04-06 06:38:56,035 BAD EPOCHS (no improvement): 0 2023-04-06 06:38:56,041 ---------------------------------------------------------------------------------------------------- 2023-04-06 06:39:14,729 epoch 27 - iter 265/2650 - loss 0.04948407 - time (sec): 18.69 - samples/sec: 7915.52 - lr: 0.100000 2023-04-06 06:39:33,012 epoch 27 - iter 530/2650 - loss 0.04935260 - time (sec): 36.97 - samples/sec: 7956.42 - lr: 0.100000 2023-04-06 06:39:51,731 epoch 27 - iter 795/2650 - loss 0.04940735 - time (sec): 55.69 - samples/sec: 7951.72 - lr: 0.100000 2023-04-06 06:40:09,950 epoch 27 - iter 1060/2650 - loss 0.04956074 - time (sec): 73.91 - samples/sec: 7946.96 - lr: 0.100000 2023-04-06 06:40:28,387 epoch 27 - iter 1325/2650 - loss 0.04925890 - time (sec): 92.35 - samples/sec: 7958.53 - lr: 0.100000 2023-04-06 06:40:46,440 epoch 27 - iter 1590/2650 - loss 0.04942414 - time (sec): 110.40 - samples/sec: 7982.85 - lr: 0.100000 2023-04-06 06:41:04,567 epoch 27 - iter 1855/2650 - loss 0.04968041 - time (sec): 128.53 - samples/sec: 7992.04 - lr: 0.100000 2023-04-06 06:41:22,581 epoch 27 - iter 2120/2650 - loss 0.04999668 - time (sec): 146.54 - samples/sec: 7993.94 - lr: 0.100000 2023-04-06 06:41:41,039 epoch 27 - iter 2385/2650 - loss 0.05005147 - time (sec): 165.00 - samples/sec: 7986.90 - lr: 0.100000 2023-04-06 06:41:58,940 epoch 27 - iter 2650/2650 - loss 0.05021904 - time (sec): 182.90 - samples/sec: 7995.46 - lr: 0.100000 2023-04-06 06:41:58,941 ---------------------------------------------------------------------------------------------------- 2023-04-06 06:41:58,941 EPOCH 27 done: loss 0.0502 - lr 0.100000 2023-04-06 06:41:58,941 BAD EPOCHS (no improvement): 1 2023-04-06 06:41:58,944 ---------------------------------------------------------------------------------------------------- 2023-04-06 06:42:17,357 epoch 28 - iter 265/2650 - loss 0.04738829 - time (sec): 18.41 - samples/sec: 7937.17 - lr: 0.100000 2023-04-06 06:42:35,962 epoch 28 - iter 530/2650 - loss 0.04880337 - time (sec): 37.02 - samples/sec: 7948.15 - lr: 0.100000 2023-04-06 06:42:54,480 epoch 28 - iter 795/2650 - loss 0.04850656 - time (sec): 55.54 - samples/sec: 7927.90 - lr: 0.100000 2023-04-06 06:43:12,686 epoch 28 - iter 1060/2650 - loss 0.04835668 - time (sec): 73.74 - samples/sec: 7950.02 - lr: 0.100000 2023-04-06 06:43:31,072 epoch 28 - iter 1325/2650 - loss 0.04855453 - time (sec): 92.13 - samples/sec: 7954.75 - lr: 0.100000 2023-04-06 06:43:49,721 epoch 28 - iter 1590/2650 - loss 0.04858814 - time (sec): 110.78 - samples/sec: 7940.63 - lr: 0.100000 2023-04-06 06:44:08,322 epoch 28 - iter 1855/2650 - loss 0.04865366 - time (sec): 129.38 - samples/sec: 7923.50 - lr: 0.100000 2023-04-06 06:44:26,385 epoch 28 - iter 2120/2650 - loss 0.04876921 - time (sec): 147.44 - samples/sec: 7941.80 - lr: 0.100000 2023-04-06 06:44:44,453 epoch 28 - iter 2385/2650 - loss 0.04884676 - time (sec): 165.51 - samples/sec: 7953.88 - lr: 0.100000 2023-04-06 06:45:02,496 epoch 28 - iter 2650/2650 - loss 0.04916127 - time (sec): 183.55 - samples/sec: 7967.01 - lr: 0.100000 2023-04-06 06:45:02,496 ---------------------------------------------------------------------------------------------------- 2023-04-06 06:45:02,497 EPOCH 28 done: loss 0.0492 - lr 0.100000 2023-04-06 06:45:02,497 BAD EPOCHS (no improvement): 0 2023-04-06 06:45:02,500 ---------------------------------------------------------------------------------------------------- 2023-04-06 06:45:20,868 epoch 29 - iter 265/2650 - loss 0.04651407 - time (sec): 18.37 - samples/sec: 8017.71 - lr: 0.100000 2023-04-06 06:45:39,216 epoch 29 - iter 530/2650 - loss 0.04741991 - time (sec): 36.72 - samples/sec: 7968.17 - lr: 0.100000 2023-04-06 06:45:57,897 epoch 29 - iter 795/2650 - loss 0.04821370 - time (sec): 55.40 - samples/sec: 7934.21 - lr: 0.100000 2023-04-06 06:46:16,671 epoch 29 - iter 1060/2650 - loss 0.04858684 - time (sec): 74.17 - samples/sec: 7937.25 - lr: 0.100000 2023-04-06 06:46:35,146 epoch 29 - iter 1325/2650 - loss 0.04883254 - time (sec): 92.65 - samples/sec: 7940.40 - lr: 0.100000 2023-04-06 06:46:53,156 epoch 29 - iter 1590/2650 - loss 0.04876643 - time (sec): 110.66 - samples/sec: 7958.64 - lr: 0.100000 2023-04-06 06:47:11,467 epoch 29 - iter 1855/2650 - loss 0.04879164 - time (sec): 128.97 - samples/sec: 7944.71 - lr: 0.100000 2023-04-06 06:47:30,250 epoch 29 - iter 2120/2650 - loss 0.04867356 - time (sec): 147.75 - samples/sec: 7926.06 - lr: 0.100000 2023-04-06 06:47:48,205 epoch 29 - iter 2385/2650 - loss 0.04885466 - time (sec): 165.71 - samples/sec: 7943.29 - lr: 0.100000 2023-04-06 06:48:06,428 epoch 29 - iter 2650/2650 - loss 0.04907559 - time (sec): 183.93 - samples/sec: 7950.71 - lr: 0.100000 2023-04-06 06:48:06,429 ---------------------------------------------------------------------------------------------------- 2023-04-06 06:48:06,429 EPOCH 29 done: loss 0.0491 - lr 0.100000 2023-04-06 06:48:06,429 BAD EPOCHS (no improvement): 0 2023-04-06 06:48:06,432 ---------------------------------------------------------------------------------------------------- 2023-04-06 06:48:25,304 epoch 30 - iter 265/2650 - loss 0.04845482 - time (sec): 18.87 - samples/sec: 7827.30 - lr: 0.100000 2023-04-06 06:48:43,867 epoch 30 - iter 530/2650 - loss 0.04826927 - time (sec): 37.43 - samples/sec: 7886.62 - lr: 0.100000 2023-04-06 06:49:02,523 epoch 30 - iter 795/2650 - loss 0.04817849 - time (sec): 56.09 - samples/sec: 7896.26 - lr: 0.100000 2023-04-06 06:49:20,759 epoch 30 - iter 1060/2650 - loss 0.04843988 - time (sec): 74.33 - samples/sec: 7907.66 - lr: 0.100000 2023-04-06 06:49:38,691 epoch 30 - iter 1325/2650 - loss 0.04851299 - time (sec): 92.26 - samples/sec: 7946.36 - lr: 0.100000 2023-04-06 06:49:57,196 epoch 30 - iter 1590/2650 - loss 0.04839994 - time (sec): 110.76 - samples/sec: 7940.34 - lr: 0.100000 2023-04-06 06:50:15,551 epoch 30 - iter 1855/2650 - loss 0.04863167 - time (sec): 129.12 - samples/sec: 7935.49 - lr: 0.100000 2023-04-06 06:50:33,388 epoch 30 - iter 2120/2650 - loss 0.04904280 - time (sec): 146.96 - samples/sec: 7949.18 - lr: 0.100000 2023-04-06 06:50:52,020 epoch 30 - iter 2385/2650 - loss 0.04917319 - time (sec): 165.59 - samples/sec: 7950.68 - lr: 0.100000 2023-04-06 06:51:10,222 epoch 30 - iter 2650/2650 - loss 0.04919011 - time (sec): 183.79 - samples/sec: 7956.72 - lr: 0.100000 2023-04-06 06:51:10,222 ---------------------------------------------------------------------------------------------------- 2023-04-06 06:51:10,222 EPOCH 30 done: loss 0.0492 - lr 0.100000 2023-04-06 06:51:10,222 BAD EPOCHS (no improvement): 1 2023-04-06 06:51:10,226 ---------------------------------------------------------------------------------------------------- 2023-04-06 06:51:28,784 epoch 31 - iter 265/2650 - loss 0.04568604 - time (sec): 18.56 - samples/sec: 7924.29 - lr: 0.100000 2023-04-06 06:51:47,642 epoch 31 - iter 530/2650 - loss 0.04572573 - time (sec): 37.42 - samples/sec: 7914.85 - lr: 0.100000 2023-04-06 06:52:05,896 epoch 31 - iter 795/2650 - loss 0.04603279 - time (sec): 55.67 - samples/sec: 7927.74 - lr: 0.100000 2023-04-06 06:52:23,736 epoch 31 - iter 1060/2650 - loss 0.04640762 - time (sec): 73.51 - samples/sec: 7967.14 - lr: 0.100000 2023-04-06 06:52:41,870 epoch 31 - iter 1325/2650 - loss 0.04681475 - time (sec): 91.64 - samples/sec: 7968.88 - lr: 0.100000 2023-04-06 06:53:00,524 epoch 31 - iter 1590/2650 - loss 0.04737380 - time (sec): 110.30 - samples/sec: 7956.67 - lr: 0.100000 2023-04-06 06:53:19,211 epoch 31 - iter 1855/2650 - loss 0.04759767 - time (sec): 128.99 - samples/sec: 7943.94 - lr: 0.100000 2023-04-06 06:53:37,234 epoch 31 - iter 2120/2650 - loss 0.04766522 - time (sec): 147.01 - samples/sec: 7965.35 - lr: 0.100000 2023-04-06 06:53:55,346 epoch 31 - iter 2385/2650 - loss 0.04773263 - time (sec): 165.12 - samples/sec: 7963.57 - lr: 0.100000 2023-04-06 06:54:13,921 epoch 31 - iter 2650/2650 - loss 0.04807310 - time (sec): 183.70 - samples/sec: 7960.79 - lr: 0.100000 2023-04-06 06:54:13,922 ---------------------------------------------------------------------------------------------------- 2023-04-06 06:54:13,922 EPOCH 31 done: loss 0.0481 - lr 0.100000 2023-04-06 06:54:13,922 BAD EPOCHS (no improvement): 0 2023-04-06 06:54:13,925 ---------------------------------------------------------------------------------------------------- 2023-04-06 06:54:32,330 epoch 32 - iter 265/2650 - loss 0.04579398 - time (sec): 18.40 - samples/sec: 8049.28 - lr: 0.100000 2023-04-06 06:54:51,156 epoch 32 - iter 530/2650 - loss 0.04706835 - time (sec): 37.23 - samples/sec: 7936.36 - lr: 0.100000 2023-04-06 06:55:09,477 epoch 32 - iter 795/2650 - loss 0.04686093 - time (sec): 55.55 - samples/sec: 7924.65 - lr: 0.100000 2023-04-06 06:55:27,886 epoch 32 - iter 1060/2650 - loss 0.04718461 - time (sec): 73.96 - samples/sec: 7910.29 - lr: 0.100000 2023-04-06 06:55:46,524 epoch 32 - iter 1325/2650 - loss 0.04719569 - time (sec): 92.60 - samples/sec: 7898.73 - lr: 0.100000 2023-04-06 06:56:04,634 epoch 32 - iter 1590/2650 - loss 0.04753131 - time (sec): 110.71 - samples/sec: 7921.21 - lr: 0.100000 2023-04-06 06:56:23,284 epoch 32 - iter 1855/2650 - loss 0.04765527 - time (sec): 129.36 - samples/sec: 7919.74 - lr: 0.100000 2023-04-06 06:56:41,865 epoch 32 - iter 2120/2650 - loss 0.04793031 - time (sec): 147.94 - samples/sec: 7916.62 - lr: 0.100000 2023-04-06 06:56:59,968 epoch 32 - iter 2385/2650 - loss 0.04821548 - time (sec): 166.04 - samples/sec: 7927.02 - lr: 0.100000 2023-04-06 06:57:18,168 epoch 32 - iter 2650/2650 - loss 0.04816274 - time (sec): 184.24 - samples/sec: 7937.15 - lr: 0.100000 2023-04-06 06:57:18,168 ---------------------------------------------------------------------------------------------------- 2023-04-06 06:57:18,168 EPOCH 32 done: loss 0.0482 - lr 0.100000 2023-04-06 06:57:18,168 BAD EPOCHS (no improvement): 1 2023-04-06 06:57:18,171 ---------------------------------------------------------------------------------------------------- 2023-04-06 06:57:36,384 epoch 33 - iter 265/2650 - loss 0.04424056 - time (sec): 18.21 - samples/sec: 8018.02 - lr: 0.100000 2023-04-06 06:57:54,422 epoch 33 - iter 530/2650 - loss 0.04530764 - time (sec): 36.25 - samples/sec: 8049.11 - lr: 0.100000 2023-04-06 06:58:12,579 epoch 33 - iter 795/2650 - loss 0.04567019 - time (sec): 54.41 - samples/sec: 8058.15 - lr: 0.100000 2023-04-06 06:58:30,928 epoch 33 - iter 1060/2650 - loss 0.04642559 - time (sec): 72.76 - samples/sec: 8033.54 - lr: 0.100000 2023-04-06 06:58:49,429 epoch 33 - iter 1325/2650 - loss 0.04693289 - time (sec): 91.26 - samples/sec: 8015.08 - lr: 0.100000 2023-04-06 06:59:08,303 epoch 33 - iter 1590/2650 - loss 0.04691168 - time (sec): 110.13 - samples/sec: 7982.37 - lr: 0.100000 2023-04-06 06:59:27,247 epoch 33 - iter 1855/2650 - loss 0.04704022 - time (sec): 129.08 - samples/sec: 7959.46 - lr: 0.100000 2023-04-06 06:59:45,253 epoch 33 - iter 2120/2650 - loss 0.04737948 - time (sec): 147.08 - samples/sec: 7976.38 - lr: 0.100000 2023-04-06 07:00:03,306 epoch 33 - iter 2385/2650 - loss 0.04751665 - time (sec): 165.13 - samples/sec: 7980.14 - lr: 0.100000 2023-04-06 07:00:21,612 epoch 33 - iter 2650/2650 - loss 0.04770536 - time (sec): 183.44 - samples/sec: 7971.86 - lr: 0.100000 2023-04-06 07:00:21,612 ---------------------------------------------------------------------------------------------------- 2023-04-06 07:00:21,612 EPOCH 33 done: loss 0.0477 - lr 0.100000 2023-04-06 07:00:21,612 BAD EPOCHS (no improvement): 0 2023-04-06 07:00:21,614 ---------------------------------------------------------------------------------------------------- 2023-04-06 07:00:40,059 epoch 34 - iter 265/2650 - loss 0.04786226 - time (sec): 18.44 - samples/sec: 7971.30 - lr: 0.100000 2023-04-06 07:00:58,550 epoch 34 - iter 530/2650 - loss 0.04734375 - time (sec): 36.94 - samples/sec: 7910.82 - lr: 0.100000 2023-04-06 07:01:16,946 epoch 34 - iter 795/2650 - loss 0.04746725 - time (sec): 55.33 - samples/sec: 7901.13 - lr: 0.100000 2023-04-06 07:01:35,519 epoch 34 - iter 1060/2650 - loss 0.04690567 - time (sec): 73.90 - samples/sec: 7909.33 - lr: 0.100000 2023-04-06 07:01:53,723 epoch 34 - iter 1325/2650 - loss 0.04702633 - time (sec): 92.11 - samples/sec: 7942.61 - lr: 0.100000 2023-04-06 07:02:12,263 epoch 34 - iter 1590/2650 - loss 0.04707414 - time (sec): 110.65 - samples/sec: 7944.70 - lr: 0.100000 2023-04-06 07:02:30,449 epoch 34 - iter 1855/2650 - loss 0.04727529 - time (sec): 128.83 - samples/sec: 7943.30 - lr: 0.100000 2023-04-06 07:02:49,169 epoch 34 - iter 2120/2650 - loss 0.04724277 - time (sec): 147.55 - samples/sec: 7934.68 - lr: 0.100000 2023-04-06 07:03:07,211 epoch 34 - iter 2385/2650 - loss 0.04728943 - time (sec): 165.60 - samples/sec: 7948.28 - lr: 0.100000 2023-04-06 07:03:25,444 epoch 34 - iter 2650/2650 - loss 0.04738759 - time (sec): 183.83 - samples/sec: 7954.98 - lr: 0.100000 2023-04-06 07:03:25,445 ---------------------------------------------------------------------------------------------------- 2023-04-06 07:03:25,445 EPOCH 34 done: loss 0.0474 - lr 0.100000 2023-04-06 07:03:25,445 BAD EPOCHS (no improvement): 0 2023-04-06 07:03:25,449 ---------------------------------------------------------------------------------------------------- 2023-04-06 07:03:44,016 epoch 35 - iter 265/2650 - loss 0.04745213 - time (sec): 18.57 - samples/sec: 7880.71 - lr: 0.100000 2023-04-06 07:04:02,313 epoch 35 - iter 530/2650 - loss 0.04702722 - time (sec): 36.86 - samples/sec: 7876.79 - lr: 0.100000 2023-04-06 07:04:20,514 epoch 35 - iter 795/2650 - loss 0.04707895 - time (sec): 55.07 - samples/sec: 7927.28 - lr: 0.100000 2023-04-06 07:04:38,685 epoch 35 - iter 1060/2650 - loss 0.04737276 - time (sec): 73.24 - samples/sec: 7946.87 - lr: 0.100000 2023-04-06 07:04:57,113 epoch 35 - iter 1325/2650 - loss 0.04730952 - time (sec): 91.66 - samples/sec: 7960.96 - lr: 0.100000 2023-04-06 07:05:15,392 epoch 35 - iter 1590/2650 - loss 0.04753383 - time (sec): 109.94 - samples/sec: 7957.15 - lr: 0.100000 2023-04-06 07:05:34,028 epoch 35 - iter 1855/2650 - loss 0.04761081 - time (sec): 128.58 - samples/sec: 7961.40 - lr: 0.100000 2023-04-06 07:05:52,207 epoch 35 - iter 2120/2650 - loss 0.04765235 - time (sec): 146.76 - samples/sec: 7973.05 - lr: 0.100000 2023-04-06 07:06:10,771 epoch 35 - iter 2385/2650 - loss 0.04750009 - time (sec): 165.32 - samples/sec: 7973.12 - lr: 0.100000 2023-04-06 07:06:28,721 epoch 35 - iter 2650/2650 - loss 0.04771099 - time (sec): 183.27 - samples/sec: 7979.17 - lr: 0.100000 2023-04-06 07:06:28,722 ---------------------------------------------------------------------------------------------------- 2023-04-06 07:06:28,722 EPOCH 35 done: loss 0.0477 - lr 0.100000 2023-04-06 07:06:28,722 BAD EPOCHS (no improvement): 1 2023-04-06 07:06:28,725 ---------------------------------------------------------------------------------------------------- 2023-04-06 07:06:46,997 epoch 36 - iter 265/2650 - loss 0.04686789 - time (sec): 18.27 - samples/sec: 8001.64 - lr: 0.100000 2023-04-06 07:07:05,341 epoch 36 - iter 530/2650 - loss 0.04686082 - time (sec): 36.62 - samples/sec: 7997.44 - lr: 0.100000 2023-04-06 07:07:23,776 epoch 36 - iter 795/2650 - loss 0.04672167 - time (sec): 55.05 - samples/sec: 7965.71 - lr: 0.100000 2023-04-06 07:07:42,313 epoch 36 - iter 1060/2650 - loss 0.04705424 - time (sec): 73.59 - samples/sec: 7964.61 - lr: 0.100000 2023-04-06 07:08:00,519 epoch 36 - iter 1325/2650 - loss 0.04685178 - time (sec): 91.79 - samples/sec: 7977.32 - lr: 0.100000 2023-04-06 07:08:18,911 epoch 36 - iter 1590/2650 - loss 0.04689855 - time (sec): 110.19 - samples/sec: 7973.80 - lr: 0.100000 2023-04-06 07:08:37,541 epoch 36 - iter 1855/2650 - loss 0.04690350 - time (sec): 128.82 - samples/sec: 7957.66 - lr: 0.100000 2023-04-06 07:08:55,610 epoch 36 - iter 2120/2650 - loss 0.04711047 - time (sec): 146.88 - samples/sec: 7971.12 - lr: 0.100000 2023-04-06 07:09:13,629 epoch 36 - iter 2385/2650 - loss 0.04738291 - time (sec): 164.90 - samples/sec: 7981.10 - lr: 0.100000 2023-04-06 07:09:32,058 epoch 36 - iter 2650/2650 - loss 0.04752748 - time (sec): 183.33 - samples/sec: 7976.52 - lr: 0.100000 2023-04-06 07:09:32,059 ---------------------------------------------------------------------------------------------------- 2023-04-06 07:09:32,059 EPOCH 36 done: loss 0.0475 - lr 0.100000 2023-04-06 07:09:32,059 BAD EPOCHS (no improvement): 2 2023-04-06 07:09:32,062 ---------------------------------------------------------------------------------------------------- 2023-04-06 07:09:50,514 epoch 37 - iter 265/2650 - loss 0.04462592 - time (sec): 18.45 - samples/sec: 7988.93 - lr: 0.100000 2023-04-06 07:10:08,778 epoch 37 - iter 530/2650 - loss 0.04566629 - time (sec): 36.72 - samples/sec: 7993.73 - lr: 0.100000 2023-04-06 07:10:27,377 epoch 37 - iter 795/2650 - loss 0.04673274 - time (sec): 55.31 - samples/sec: 7909.90 - lr: 0.100000 2023-04-06 07:10:46,310 epoch 37 - iter 1060/2650 - loss 0.04659566 - time (sec): 74.25 - samples/sec: 7857.25 - lr: 0.100000 2023-04-06 07:11:04,698 epoch 37 - iter 1325/2650 - loss 0.04668428 - time (sec): 92.64 - samples/sec: 7877.89 - lr: 0.100000 2023-04-06 07:11:22,942 epoch 37 - iter 1590/2650 - loss 0.04687260 - time (sec): 110.88 - samples/sec: 7880.20 - lr: 0.100000 2023-04-06 07:11:41,860 epoch 37 - iter 1855/2650 - loss 0.04696027 - time (sec): 129.80 - samples/sec: 7870.53 - lr: 0.100000 2023-04-06 07:12:00,074 epoch 37 - iter 2120/2650 - loss 0.04698397 - time (sec): 148.01 - samples/sec: 7889.61 - lr: 0.100000 2023-04-06 07:12:19,189 epoch 37 - iter 2385/2650 - loss 0.04694635 - time (sec): 167.13 - samples/sec: 7868.31 - lr: 0.100000 2023-04-06 07:12:37,797 epoch 37 - iter 2650/2650 - loss 0.04712440 - time (sec): 185.73 - samples/sec: 7873.39 - lr: 0.100000 2023-04-06 07:12:37,797 ---------------------------------------------------------------------------------------------------- 2023-04-06 07:12:37,797 EPOCH 37 done: loss 0.0471 - lr 0.100000 2023-04-06 07:12:37,798 BAD EPOCHS (no improvement): 0 2023-04-06 07:12:37,800 ---------------------------------------------------------------------------------------------------- 2023-04-06 07:12:55,610 epoch 38 - iter 265/2650 - loss 0.04444841 - time (sec): 17.81 - samples/sec: 8050.87 - lr: 0.100000 2023-04-06 07:13:13,723 epoch 38 - iter 530/2650 - loss 0.04521853 - time (sec): 35.92 - samples/sec: 7993.66 - lr: 0.100000 2023-04-06 07:13:31,745 epoch 38 - iter 795/2650 - loss 0.04548371 - time (sec): 53.94 - samples/sec: 8021.89 - lr: 0.100000 2023-04-06 07:13:50,092 epoch 38 - iter 1060/2650 - loss 0.04575076 - time (sec): 72.29 - samples/sec: 8018.57 - lr: 0.100000 2023-04-06 07:14:08,998 epoch 38 - iter 1325/2650 - loss 0.04595887 - time (sec): 91.20 - samples/sec: 7977.88 - lr: 0.100000 2023-04-06 07:14:27,641 epoch 38 - iter 1590/2650 - loss 0.04596051 - time (sec): 109.84 - samples/sec: 7975.79 - lr: 0.100000 2023-04-06 07:14:46,377 epoch 38 - iter 1855/2650 - loss 0.04609244 - time (sec): 128.58 - samples/sec: 7958.18 - lr: 0.100000 2023-04-06 07:15:04,780 epoch 38 - iter 2120/2650 - loss 0.04620948 - time (sec): 146.98 - samples/sec: 7958.29 - lr: 0.100000 2023-04-06 07:15:23,114 epoch 38 - iter 2385/2650 - loss 0.04631248 - time (sec): 165.31 - samples/sec: 7961.35 - lr: 0.100000 2023-04-06 07:15:41,348 epoch 38 - iter 2650/2650 - loss 0.04660947 - time (sec): 183.55 - samples/sec: 7967.20 - lr: 0.100000 2023-04-06 07:15:41,348 ---------------------------------------------------------------------------------------------------- 2023-04-06 07:15:41,348 EPOCH 38 done: loss 0.0466 - lr 0.100000 2023-04-06 07:15:41,348 BAD EPOCHS (no improvement): 0 2023-04-06 07:15:41,351 ---------------------------------------------------------------------------------------------------- 2023-04-06 07:15:59,902 epoch 39 - iter 265/2650 - loss 0.04667561 - time (sec): 18.55 - samples/sec: 7921.22 - lr: 0.100000 2023-04-06 07:16:18,101 epoch 39 - iter 530/2650 - loss 0.04593290 - time (sec): 36.75 - samples/sec: 7966.24 - lr: 0.100000 2023-04-06 07:16:37,023 epoch 39 - iter 795/2650 - loss 0.04575068 - time (sec): 55.67 - samples/sec: 7918.96 - lr: 0.100000 2023-04-06 07:16:54,826 epoch 39 - iter 1060/2650 - loss 0.04585644 - time (sec): 73.47 - samples/sec: 7966.28 - lr: 0.100000 2023-04-06 07:17:13,344 epoch 39 - iter 1325/2650 - loss 0.04603577 - time (sec): 91.99 - samples/sec: 7954.65 - lr: 0.100000 2023-04-06 07:17:31,125 epoch 39 - iter 1590/2650 - loss 0.04626398 - time (sec): 109.77 - samples/sec: 7980.41 - lr: 0.100000 2023-04-06 07:17:49,466 epoch 39 - iter 1855/2650 - loss 0.04654012 - time (sec): 128.11 - samples/sec: 7979.86 - lr: 0.100000 2023-04-06 07:18:08,508 epoch 39 - iter 2120/2650 - loss 0.04661497 - time (sec): 147.16 - samples/sec: 7955.43 - lr: 0.100000 2023-04-06 07:18:26,740 epoch 39 - iter 2385/2650 - loss 0.04666364 - time (sec): 165.39 - samples/sec: 7964.94 - lr: 0.100000 2023-04-06 07:18:45,060 epoch 39 - iter 2650/2650 - loss 0.04685336 - time (sec): 183.71 - samples/sec: 7960.23 - lr: 0.100000 2023-04-06 07:18:45,060 ---------------------------------------------------------------------------------------------------- 2023-04-06 07:18:45,060 EPOCH 39 done: loss 0.0469 - lr 0.100000 2023-04-06 07:18:45,060 BAD EPOCHS (no improvement): 1 2023-04-06 07:18:45,063 ---------------------------------------------------------------------------------------------------- 2023-04-06 07:19:03,462 epoch 40 - iter 265/2650 - loss 0.04626657 - time (sec): 18.40 - samples/sec: 7877.29 - lr: 0.100000 2023-04-06 07:19:21,973 epoch 40 - iter 530/2650 - loss 0.04505035 - time (sec): 36.91 - samples/sec: 7955.81 - lr: 0.100000 2023-04-06 07:19:40,467 epoch 40 - iter 795/2650 - loss 0.04528049 - time (sec): 55.40 - samples/sec: 7927.54 - lr: 0.100000 2023-04-06 07:19:58,789 epoch 40 - iter 1060/2650 - loss 0.04580529 - time (sec): 73.73 - samples/sec: 7943.51 - lr: 0.100000 2023-04-06 07:20:17,246 epoch 40 - iter 1325/2650 - loss 0.04567707 - time (sec): 92.18 - samples/sec: 7940.63 - lr: 0.100000 2023-04-06 07:20:35,733 epoch 40 - iter 1590/2650 - loss 0.04600558 - time (sec): 110.67 - samples/sec: 7935.37 - lr: 0.100000 2023-04-06 07:20:54,316 epoch 40 - iter 1855/2650 - loss 0.04593860 - time (sec): 129.25 - samples/sec: 7931.53 - lr: 0.100000 2023-04-06 07:21:12,658 epoch 40 - iter 2120/2650 - loss 0.04607121 - time (sec): 147.59 - samples/sec: 7935.42 - lr: 0.100000 2023-04-06 07:21:31,153 epoch 40 - iter 2385/2650 - loss 0.04620532 - time (sec): 166.09 - samples/sec: 7929.63 - lr: 0.100000 2023-04-06 07:21:49,098 epoch 40 - iter 2650/2650 - loss 0.04641797 - time (sec): 184.03 - samples/sec: 7946.14 - lr: 0.100000 2023-04-06 07:21:49,098 ---------------------------------------------------------------------------------------------------- 2023-04-06 07:21:49,098 EPOCH 40 done: loss 0.0464 - lr 0.100000 2023-04-06 07:21:49,098 BAD EPOCHS (no improvement): 0 2023-04-06 07:21:49,101 ---------------------------------------------------------------------------------------------------- 2023-04-06 07:22:07,453 epoch 41 - iter 265/2650 - loss 0.04548783 - time (sec): 18.35 - samples/sec: 7932.22 - lr: 0.100000 2023-04-06 07:22:25,713 epoch 41 - iter 530/2650 - loss 0.04636131 - time (sec): 36.61 - samples/sec: 7978.45 - lr: 0.100000 2023-04-06 07:22:44,432 epoch 41 - iter 795/2650 - loss 0.04576686 - time (sec): 55.33 - samples/sec: 7961.41 - lr: 0.100000 2023-04-06 07:23:02,968 epoch 41 - iter 1060/2650 - loss 0.04577877 - time (sec): 73.87 - samples/sec: 7961.59 - lr: 0.100000 2023-04-06 07:23:21,550 epoch 41 - iter 1325/2650 - loss 0.04581047 - time (sec): 92.45 - samples/sec: 7953.07 - lr: 0.100000 2023-04-06 07:23:39,171 epoch 41 - iter 1590/2650 - loss 0.04604941 - time (sec): 110.07 - samples/sec: 7979.30 - lr: 0.100000 2023-04-06 07:23:57,355 epoch 41 - iter 1855/2650 - loss 0.04591811 - time (sec): 128.25 - samples/sec: 7983.37 - lr: 0.100000 2023-04-06 07:24:15,804 epoch 41 - iter 2120/2650 - loss 0.04595454 - time (sec): 146.70 - samples/sec: 7979.58 - lr: 0.100000 2023-04-06 07:24:33,888 epoch 41 - iter 2385/2650 - loss 0.04607900 - time (sec): 164.79 - samples/sec: 7983.46 - lr: 0.100000 2023-04-06 07:24:52,344 epoch 41 - iter 2650/2650 - loss 0.04626977 - time (sec): 183.24 - samples/sec: 7980.47 - lr: 0.100000 2023-04-06 07:24:52,344 ---------------------------------------------------------------------------------------------------- 2023-04-06 07:24:52,344 EPOCH 41 done: loss 0.0463 - lr 0.100000 2023-04-06 07:24:52,344 BAD EPOCHS (no improvement): 0 2023-04-06 07:24:52,347 ---------------------------------------------------------------------------------------------------- 2023-04-06 07:25:10,938 epoch 42 - iter 265/2650 - loss 0.04452100 - time (sec): 18.59 - samples/sec: 7862.93 - lr: 0.100000 2023-04-06 07:25:29,191 epoch 42 - iter 530/2650 - loss 0.04513252 - time (sec): 36.84 - samples/sec: 7954.45 - lr: 0.100000 2023-04-06 07:25:47,218 epoch 42 - iter 795/2650 - loss 0.04509194 - time (sec): 54.87 - samples/sec: 7974.77 - lr: 0.100000 2023-04-06 07:26:05,720 epoch 42 - iter 1060/2650 - loss 0.04531120 - time (sec): 73.37 - samples/sec: 7981.41 - lr: 0.100000 2023-04-06 07:26:24,002 epoch 42 - iter 1325/2650 - loss 0.04531895 - time (sec): 91.66 - samples/sec: 7975.45 - lr: 0.100000 2023-04-06 07:26:42,277 epoch 42 - iter 1590/2650 - loss 0.04541402 - time (sec): 109.93 - samples/sec: 7977.73 - lr: 0.100000 2023-04-06 07:27:00,784 epoch 42 - iter 1855/2650 - loss 0.04556688 - time (sec): 128.44 - samples/sec: 7964.56 - lr: 0.100000 2023-04-06 07:27:19,716 epoch 42 - iter 2120/2650 - loss 0.04578374 - time (sec): 147.37 - samples/sec: 7943.75 - lr: 0.100000 2023-04-06 07:27:38,029 epoch 42 - iter 2385/2650 - loss 0.04588951 - time (sec): 165.68 - samples/sec: 7946.59 - lr: 0.100000 2023-04-06 07:27:56,733 epoch 42 - iter 2650/2650 - loss 0.04606499 - time (sec): 184.39 - samples/sec: 7930.99 - lr: 0.100000 2023-04-06 07:27:56,733 ---------------------------------------------------------------------------------------------------- 2023-04-06 07:27:56,733 EPOCH 42 done: loss 0.0461 - lr 0.100000 2023-04-06 07:27:56,733 BAD EPOCHS (no improvement): 0 2023-04-06 07:27:56,736 ---------------------------------------------------------------------------------------------------- 2023-04-06 07:28:15,353 epoch 43 - iter 265/2650 - loss 0.04506543 - time (sec): 18.62 - samples/sec: 7804.52 - lr: 0.100000 2023-04-06 07:28:34,012 epoch 43 - iter 530/2650 - loss 0.04450566 - time (sec): 37.28 - samples/sec: 7806.17 - lr: 0.100000 2023-04-06 07:28:53,066 epoch 43 - iter 795/2650 - loss 0.04490488 - time (sec): 56.33 - samples/sec: 7804.36 - lr: 0.100000 2023-04-06 07:29:11,482 epoch 43 - iter 1060/2650 - loss 0.04519482 - time (sec): 74.75 - samples/sec: 7839.28 - lr: 0.100000 2023-04-06 07:29:30,507 epoch 43 - iter 1325/2650 - loss 0.04557753 - time (sec): 93.77 - samples/sec: 7805.30 - lr: 0.100000 2023-04-06 07:29:48,821 epoch 43 - iter 1590/2650 - loss 0.04560729 - time (sec): 112.08 - samples/sec: 7826.79 - lr: 0.100000 2023-04-06 07:30:06,869 epoch 43 - iter 1855/2650 - loss 0.04565619 - time (sec): 130.13 - samples/sec: 7864.87 - lr: 0.100000 2023-04-06 07:30:25,158 epoch 43 - iter 2120/2650 - loss 0.04572466 - time (sec): 148.42 - samples/sec: 7881.91 - lr: 0.100000 2023-04-06 07:30:43,844 epoch 43 - iter 2385/2650 - loss 0.04579375 - time (sec): 167.11 - samples/sec: 7878.62 - lr: 0.100000 2023-04-06 07:31:02,357 epoch 43 - iter 2650/2650 - loss 0.04606939 - time (sec): 185.62 - samples/sec: 7878.25 - lr: 0.100000 2023-04-06 07:31:02,357 ---------------------------------------------------------------------------------------------------- 2023-04-06 07:31:02,357 EPOCH 43 done: loss 0.0461 - lr 0.100000 2023-04-06 07:31:02,357 BAD EPOCHS (no improvement): 1 2023-04-06 07:31:02,363 ---------------------------------------------------------------------------------------------------- 2023-04-06 07:31:21,192 epoch 44 - iter 265/2650 - loss 0.04406182 - time (sec): 18.83 - samples/sec: 7873.33 - lr: 0.100000 2023-04-06 07:31:39,819 epoch 44 - iter 530/2650 - loss 0.04367286 - time (sec): 37.46 - samples/sec: 7875.74 - lr: 0.100000 2023-04-06 07:31:58,246 epoch 44 - iter 795/2650 - loss 0.04459361 - time (sec): 55.88 - samples/sec: 7883.92 - lr: 0.100000 2023-04-06 07:32:16,676 epoch 44 - iter 1060/2650 - loss 0.04466473 - time (sec): 74.31 - samples/sec: 7886.74 - lr: 0.100000 2023-04-06 07:32:35,246 epoch 44 - iter 1325/2650 - loss 0.04489351 - time (sec): 92.88 - samples/sec: 7879.54 - lr: 0.100000 2023-04-06 07:32:54,012 epoch 44 - iter 1590/2650 - loss 0.04521088 - time (sec): 111.65 - samples/sec: 7870.36 - lr: 0.100000 2023-04-06 07:33:12,363 epoch 44 - iter 1855/2650 - loss 0.04519873 - time (sec): 130.00 - samples/sec: 7878.81 - lr: 0.100000 2023-04-06 07:33:30,625 epoch 44 - iter 2120/2650 - loss 0.04549340 - time (sec): 148.26 - samples/sec: 7879.53 - lr: 0.100000 2023-04-06 07:33:49,805 epoch 44 - iter 2385/2650 - loss 0.04551792 - time (sec): 167.44 - samples/sec: 7853.02 - lr: 0.100000 2023-04-06 07:34:08,582 epoch 44 - iter 2650/2650 - loss 0.04566768 - time (sec): 186.22 - samples/sec: 7852.93 - lr: 0.100000 2023-04-06 07:34:08,582 ---------------------------------------------------------------------------------------------------- 2023-04-06 07:34:08,582 EPOCH 44 done: loss 0.0457 - lr 0.100000 2023-04-06 07:34:08,582 BAD EPOCHS (no improvement): 0 2023-04-06 07:34:08,586 ---------------------------------------------------------------------------------------------------- 2023-04-06 07:34:27,387 epoch 45 - iter 265/2650 - loss 0.04334286 - time (sec): 18.80 - samples/sec: 7781.76 - lr: 0.100000 2023-04-06 07:34:45,888 epoch 45 - iter 530/2650 - loss 0.04355469 - time (sec): 37.30 - samples/sec: 7849.33 - lr: 0.100000 2023-04-06 07:35:04,661 epoch 45 - iter 795/2650 - loss 0.04392615 - time (sec): 56.08 - samples/sec: 7841.36 - lr: 0.100000 2023-04-06 07:35:23,073 epoch 45 - iter 1060/2650 - loss 0.04390119 - time (sec): 74.49 - samples/sec: 7857.12 - lr: 0.100000 2023-04-06 07:35:41,336 epoch 45 - iter 1325/2650 - loss 0.04434738 - time (sec): 92.75 - samples/sec: 7887.08 - lr: 0.100000 2023-04-06 07:35:59,849 epoch 45 - iter 1590/2650 - loss 0.04457737 - time (sec): 111.26 - samples/sec: 7881.70 - lr: 0.100000 2023-04-06 07:36:18,132 epoch 45 - iter 1855/2650 - loss 0.04472227 - time (sec): 129.55 - samples/sec: 7899.93 - lr: 0.100000 2023-04-06 07:36:36,165 epoch 45 - iter 2120/2650 - loss 0.04484784 - time (sec): 147.58 - samples/sec: 7914.93 - lr: 0.100000 2023-04-06 07:36:54,541 epoch 45 - iter 2385/2650 - loss 0.04497065 - time (sec): 165.96 - samples/sec: 7920.10 - lr: 0.100000 2023-04-06 07:37:13,541 epoch 45 - iter 2650/2650 - loss 0.04514677 - time (sec): 184.96 - samples/sec: 7906.55 - lr: 0.100000 2023-04-06 07:37:13,542 ---------------------------------------------------------------------------------------------------- 2023-04-06 07:37:13,542 EPOCH 45 done: loss 0.0451 - lr 0.100000 2023-04-06 07:37:13,542 BAD EPOCHS (no improvement): 0 2023-04-06 07:37:13,545 ---------------------------------------------------------------------------------------------------- 2023-04-06 07:37:32,034 epoch 46 - iter 265/2650 - loss 0.04534815 - time (sec): 18.49 - samples/sec: 7949.91 - lr: 0.100000 2023-04-06 07:37:51,014 epoch 46 - iter 530/2650 - loss 0.04528589 - time (sec): 37.47 - samples/sec: 7833.59 - lr: 0.100000 2023-04-06 07:38:09,823 epoch 46 - iter 795/2650 - loss 0.04548214 - time (sec): 56.28 - samples/sec: 7809.77 - lr: 0.100000 2023-04-06 07:38:28,272 epoch 46 - iter 1060/2650 - loss 0.04556463 - time (sec): 74.73 - samples/sec: 7818.50 - lr: 0.100000 2023-04-06 07:38:47,295 epoch 46 - iter 1325/2650 - loss 0.04514534 - time (sec): 93.75 - samples/sec: 7799.93 - lr: 0.100000 2023-04-06 07:39:06,436 epoch 46 - iter 1590/2650 - loss 0.04551431 - time (sec): 112.89 - samples/sec: 7772.98 - lr: 0.100000 2023-04-06 07:39:25,206 epoch 46 - iter 1855/2650 - loss 0.04553915 - time (sec): 131.66 - samples/sec: 7773.55 - lr: 0.100000 2023-04-06 07:39:43,463 epoch 46 - iter 2120/2650 - loss 0.04560963 - time (sec): 149.92 - samples/sec: 7793.75 - lr: 0.100000 2023-04-06 07:40:01,825 epoch 46 - iter 2385/2650 - loss 0.04562946 - time (sec): 168.28 - samples/sec: 7816.31 - lr: 0.100000 2023-04-06 07:40:20,695 epoch 46 - iter 2650/2650 - loss 0.04572817 - time (sec): 187.15 - samples/sec: 7813.87 - lr: 0.100000 2023-04-06 07:40:20,695 ---------------------------------------------------------------------------------------------------- 2023-04-06 07:40:20,695 EPOCH 46 done: loss 0.0457 - lr 0.100000 2023-04-06 07:40:20,695 BAD EPOCHS (no improvement): 1 2023-04-06 07:40:20,698 ---------------------------------------------------------------------------------------------------- 2023-04-06 07:40:39,183 epoch 47 - iter 265/2650 - loss 0.04133594 - time (sec): 18.48 - samples/sec: 7854.31 - lr: 0.100000 2023-04-06 07:40:57,904 epoch 47 - iter 530/2650 - loss 0.04262400 - time (sec): 37.21 - samples/sec: 7824.54 - lr: 0.100000 2023-04-06 07:41:15,973 epoch 47 - iter 795/2650 - loss 0.04353134 - time (sec): 55.27 - samples/sec: 7880.48 - lr: 0.100000 2023-04-06 07:41:34,852 epoch 47 - iter 1060/2650 - loss 0.04393743 - time (sec): 74.15 - samples/sec: 7864.98 - lr: 0.100000 2023-04-06 07:41:53,964 epoch 47 - iter 1325/2650 - loss 0.04431883 - time (sec): 93.27 - samples/sec: 7821.93 - lr: 0.100000 2023-04-06 07:42:12,803 epoch 47 - iter 1590/2650 - loss 0.04435540 - time (sec): 112.10 - samples/sec: 7811.32 - lr: 0.100000 2023-04-06 07:42:31,639 epoch 47 - iter 1855/2650 - loss 0.04463048 - time (sec): 130.94 - samples/sec: 7816.97 - lr: 0.100000 2023-04-06 07:42:50,300 epoch 47 - iter 2120/2650 - loss 0.04484304 - time (sec): 149.60 - samples/sec: 7819.86 - lr: 0.100000 2023-04-06 07:43:09,317 epoch 47 - iter 2385/2650 - loss 0.04490130 - time (sec): 168.62 - samples/sec: 7804.51 - lr: 0.100000 2023-04-06 07:43:28,060 epoch 47 - iter 2650/2650 - loss 0.04492588 - time (sec): 187.36 - samples/sec: 7805.02 - lr: 0.100000 2023-04-06 07:43:28,060 ---------------------------------------------------------------------------------------------------- 2023-04-06 07:43:28,060 EPOCH 47 done: loss 0.0449 - lr 0.100000 2023-04-06 07:43:28,060 BAD EPOCHS (no improvement): 0 2023-04-06 07:43:28,064 ---------------------------------------------------------------------------------------------------- 2023-04-06 07:43:46,996 epoch 48 - iter 265/2650 - loss 0.04406631 - time (sec): 18.93 - samples/sec: 7695.00 - lr: 0.100000 2023-04-06 07:44:05,835 epoch 48 - iter 530/2650 - loss 0.04450037 - time (sec): 37.77 - samples/sec: 7765.58 - lr: 0.100000 2023-04-06 07:44:24,646 epoch 48 - iter 795/2650 - loss 0.04473852 - time (sec): 56.58 - samples/sec: 7770.27 - lr: 0.100000 2023-04-06 07:44:43,704 epoch 48 - iter 1060/2650 - loss 0.04479096 - time (sec): 75.64 - samples/sec: 7764.59 - lr: 0.100000 2023-04-06 07:45:02,341 epoch 48 - iter 1325/2650 - loss 0.04484832 - time (sec): 94.28 - samples/sec: 7768.38 - lr: 0.100000 2023-04-06 07:45:20,667 epoch 48 - iter 1590/2650 - loss 0.04479779 - time (sec): 112.60 - samples/sec: 7809.17 - lr: 0.100000 2023-04-06 07:45:38,539 epoch 48 - iter 1855/2650 - loss 0.04493183 - time (sec): 130.48 - samples/sec: 7853.41 - lr: 0.100000 2023-04-06 07:45:56,563 epoch 48 - iter 2120/2650 - loss 0.04501677 - time (sec): 148.50 - samples/sec: 7874.30 - lr: 0.100000 2023-04-06 07:46:14,798 epoch 48 - iter 2385/2650 - loss 0.04508084 - time (sec): 166.73 - samples/sec: 7894.91 - lr: 0.100000 2023-04-06 07:46:33,352 epoch 48 - iter 2650/2650 - loss 0.04502742 - time (sec): 185.29 - samples/sec: 7892.38 - lr: 0.100000 2023-04-06 07:46:33,352 ---------------------------------------------------------------------------------------------------- 2023-04-06 07:46:33,352 EPOCH 48 done: loss 0.0450 - lr 0.100000 2023-04-06 07:46:33,352 BAD EPOCHS (no improvement): 1 2023-04-06 07:46:33,356 ---------------------------------------------------------------------------------------------------- 2023-04-06 07:46:51,796 epoch 49 - iter 265/2650 - loss 0.04341949 - time (sec): 18.44 - samples/sec: 7914.22 - lr: 0.100000 2023-04-06 07:47:10,447 epoch 49 - iter 530/2650 - loss 0.04487760 - time (sec): 37.09 - samples/sec: 7882.13 - lr: 0.100000 2023-04-06 07:47:28,719 epoch 49 - iter 795/2650 - loss 0.04452396 - time (sec): 55.36 - samples/sec: 7922.98 - lr: 0.100000 2023-04-06 07:47:47,409 epoch 49 - iter 1060/2650 - loss 0.04489697 - time (sec): 74.05 - samples/sec: 7900.90 - lr: 0.100000 2023-04-06 07:48:06,370 epoch 49 - iter 1325/2650 - loss 0.04495540 - time (sec): 93.01 - samples/sec: 7883.62 - lr: 0.100000 2023-04-06 07:48:25,267 epoch 49 - iter 1590/2650 - loss 0.04495202 - time (sec): 111.91 - samples/sec: 7860.57 - lr: 0.100000 2023-04-06 07:48:44,075 epoch 49 - iter 1855/2650 - loss 0.04497125 - time (sec): 130.72 - samples/sec: 7845.06 - lr: 0.100000 2023-04-06 07:49:02,783 epoch 49 - iter 2120/2650 - loss 0.04517330 - time (sec): 149.43 - samples/sec: 7845.13 - lr: 0.100000 2023-04-06 07:49:21,491 epoch 49 - iter 2385/2650 - loss 0.04523375 - time (sec): 168.13 - samples/sec: 7839.58 - lr: 0.100000 2023-04-06 07:49:39,682 epoch 49 - iter 2650/2650 - loss 0.04525814 - time (sec): 186.33 - samples/sec: 7848.43 - lr: 0.100000 2023-04-06 07:49:39,682 ---------------------------------------------------------------------------------------------------- 2023-04-06 07:49:39,682 EPOCH 49 done: loss 0.0453 - lr 0.100000 2023-04-06 07:49:39,682 BAD EPOCHS (no improvement): 2 2023-04-06 07:49:39,685 ---------------------------------------------------------------------------------------------------- 2023-04-06 07:49:58,383 epoch 50 - iter 265/2650 - loss 0.04394068 - time (sec): 18.70 - samples/sec: 7753.77 - lr: 0.100000 2023-04-06 07:50:16,960 epoch 50 - iter 530/2650 - loss 0.04406703 - time (sec): 37.27 - samples/sec: 7813.22 - lr: 0.100000 2023-04-06 07:50:35,787 epoch 50 - iter 795/2650 - loss 0.04387915 - time (sec): 56.10 - samples/sec: 7849.94 - lr: 0.100000 2023-04-06 07:50:54,503 epoch 50 - iter 1060/2650 - loss 0.04409459 - time (sec): 74.82 - samples/sec: 7860.43 - lr: 0.100000 2023-04-06 07:51:12,973 epoch 50 - iter 1325/2650 - loss 0.04426199 - time (sec): 93.29 - samples/sec: 7851.60 - lr: 0.100000 2023-04-06 07:51:30,993 epoch 50 - iter 1590/2650 - loss 0.04445126 - time (sec): 111.31 - samples/sec: 7886.98 - lr: 0.100000 2023-04-06 07:51:49,798 epoch 50 - iter 1855/2650 - loss 0.04487967 - time (sec): 130.11 - samples/sec: 7875.74 - lr: 0.100000 2023-04-06 07:52:08,304 epoch 50 - iter 2120/2650 - loss 0.04488535 - time (sec): 148.62 - samples/sec: 7880.26 - lr: 0.100000 2023-04-06 07:52:26,470 epoch 50 - iter 2385/2650 - loss 0.04511402 - time (sec): 166.78 - samples/sec: 7891.84 - lr: 0.100000 2023-04-06 07:52:44,703 epoch 50 - iter 2650/2650 - loss 0.04526537 - time (sec): 185.02 - samples/sec: 7903.93 - lr: 0.100000 2023-04-06 07:52:44,703 ---------------------------------------------------------------------------------------------------- 2023-04-06 07:52:44,703 EPOCH 50 done: loss 0.0453 - lr 0.100000 2023-04-06 07:52:44,703 BAD EPOCHS (no improvement): 3 2023-04-06 07:52:44,706 ---------------------------------------------------------------------------------------------------- 2023-04-06 07:53:02,893 epoch 51 - iter 265/2650 - loss 0.04307510 - time (sec): 18.19 - samples/sec: 7930.23 - lr: 0.100000 2023-04-06 07:53:21,148 epoch 51 - iter 530/2650 - loss 0.04394790 - time (sec): 36.44 - samples/sec: 7930.12 - lr: 0.100000 2023-04-06 07:53:39,656 epoch 51 - iter 795/2650 - loss 0.04453353 - time (sec): 54.95 - samples/sec: 7894.47 - lr: 0.100000 2023-04-06 07:53:58,343 epoch 51 - iter 1060/2650 - loss 0.04446002 - time (sec): 73.64 - samples/sec: 7892.98 - lr: 0.100000 2023-04-06 07:54:16,727 epoch 51 - iter 1325/2650 - loss 0.04447675 - time (sec): 92.02 - samples/sec: 7915.51 - lr: 0.100000 2023-04-06 07:54:34,819 epoch 51 - iter 1590/2650 - loss 0.04455558 - time (sec): 110.11 - samples/sec: 7933.96 - lr: 0.100000 2023-04-06 07:54:53,257 epoch 51 - iter 1855/2650 - loss 0.04486059 - time (sec): 128.55 - samples/sec: 7933.51 - lr: 0.100000 2023-04-06 07:55:11,940 epoch 51 - iter 2120/2650 - loss 0.04497491 - time (sec): 147.23 - samples/sec: 7922.53 - lr: 0.100000 2023-04-06 07:55:30,500 epoch 51 - iter 2385/2650 - loss 0.04514164 - time (sec): 165.79 - samples/sec: 7927.71 - lr: 0.100000 2023-04-06 07:55:49,198 epoch 51 - iter 2650/2650 - loss 0.04511150 - time (sec): 184.49 - samples/sec: 7926.41 - lr: 0.100000 2023-04-06 07:55:49,199 ---------------------------------------------------------------------------------------------------- 2023-04-06 07:55:49,199 EPOCH 51 done: loss 0.0451 - lr 0.100000 2023-04-06 07:55:49,199 Epoch 51: reducing learning rate of group 0 to 5.0000e-02. 2023-04-06 07:55:49,199 BAD EPOCHS (no improvement): 4 2023-04-06 07:55:49,202 ---------------------------------------------------------------------------------------------------- 2023-04-06 07:56:07,448 epoch 52 - iter 265/2650 - loss 0.04317669 - time (sec): 18.25 - samples/sec: 7901.61 - lr: 0.050000 2023-04-06 07:56:36,017 epoch 52 - iter 530/2650 - loss 0.04212095 - time (sec): 46.81 - samples/sec: 6222.89 - lr: 0.050000 2023-04-06 07:56:54,699 epoch 52 - iter 795/2650 - loss 0.04206812 - time (sec): 65.50 - samples/sec: 6691.32 - lr: 0.050000 2023-04-06 07:57:12,794 epoch 52 - iter 1060/2650 - loss 0.04208970 - time (sec): 83.59 - samples/sec: 6957.54 - lr: 0.050000 2023-04-06 07:57:31,397 epoch 52 - iter 1325/2650 - loss 0.04226703 - time (sec): 102.19 - samples/sec: 7136.30 - lr: 0.050000 2023-04-06 07:57:49,888 epoch 52 - iter 1590/2650 - loss 0.04226864 - time (sec): 120.69 - samples/sec: 7255.92 - lr: 0.050000 2023-04-06 07:58:08,228 epoch 52 - iter 1855/2650 - loss 0.04204409 - time (sec): 139.03 - samples/sec: 7355.59 - lr: 0.050000 2023-04-06 07:58:26,862 epoch 52 - iter 2120/2650 - loss 0.04222261 - time (sec): 157.66 - samples/sec: 7420.62 - lr: 0.050000 2023-04-06 07:58:45,200 epoch 52 - iter 2385/2650 - loss 0.04215198 - time (sec): 176.00 - samples/sec: 7481.94 - lr: 0.050000 2023-04-06 07:59:03,769 epoch 52 - iter 2650/2650 - loss 0.04230955 - time (sec): 194.57 - samples/sec: 7515.99 - lr: 0.050000 2023-04-06 07:59:03,769 ---------------------------------------------------------------------------------------------------- 2023-04-06 07:59:03,769 EPOCH 52 done: loss 0.0423 - lr 0.050000 2023-04-06 07:59:03,769 BAD EPOCHS (no improvement): 0 2023-04-06 07:59:03,773 ---------------------------------------------------------------------------------------------------- 2023-04-06 07:59:22,388 epoch 53 - iter 265/2650 - loss 0.03949641 - time (sec): 18.62 - samples/sec: 7907.80 - lr: 0.050000 2023-04-06 07:59:40,905 epoch 53 - iter 530/2650 - loss 0.04017371 - time (sec): 37.13 - samples/sec: 7915.70 - lr: 0.050000 2023-04-06 07:59:59,600 epoch 53 - iter 795/2650 - loss 0.04024837 - time (sec): 55.83 - samples/sec: 7883.39 - lr: 0.050000 2023-04-06 08:00:18,199 epoch 53 - iter 1060/2650 - loss 0.04041977 - time (sec): 74.43 - samples/sec: 7891.57 - lr: 0.050000 2023-04-06 08:00:36,905 epoch 53 - iter 1325/2650 - loss 0.04070280 - time (sec): 93.13 - samples/sec: 7893.25 - lr: 0.050000 2023-04-06 08:00:55,111 epoch 53 - iter 1590/2650 - loss 0.04062021 - time (sec): 111.34 - samples/sec: 7903.27 - lr: 0.050000 2023-04-06 08:01:13,500 epoch 53 - iter 1855/2650 - loss 0.04059717 - time (sec): 129.73 - samples/sec: 7894.84 - lr: 0.050000 2023-04-06 08:01:31,815 epoch 53 - iter 2120/2650 - loss 0.04082940 - time (sec): 148.04 - samples/sec: 7895.80 - lr: 0.050000 2023-04-06 08:01:50,155 epoch 53 - iter 2385/2650 - loss 0.04090555 - time (sec): 166.38 - samples/sec: 7904.76 - lr: 0.050000 2023-04-06 08:02:08,739 epoch 53 - iter 2650/2650 - loss 0.04084780 - time (sec): 184.97 - samples/sec: 7906.13 - lr: 0.050000 2023-04-06 08:02:08,739 ---------------------------------------------------------------------------------------------------- 2023-04-06 08:02:08,739 EPOCH 53 done: loss 0.0408 - lr 0.050000 2023-04-06 08:02:08,739 BAD EPOCHS (no improvement): 0 2023-04-06 08:02:08,743 ---------------------------------------------------------------------------------------------------- 2023-04-06 08:02:27,417 epoch 54 - iter 265/2650 - loss 0.03990547 - time (sec): 18.67 - samples/sec: 7938.85 - lr: 0.050000 2023-04-06 08:02:45,886 epoch 54 - iter 530/2650 - loss 0.04022015 - time (sec): 37.14 - samples/sec: 7878.84 - lr: 0.050000 2023-04-06 08:03:04,541 epoch 54 - iter 795/2650 - loss 0.04047452 - time (sec): 55.80 - samples/sec: 7880.92 - lr: 0.050000 2023-04-06 08:03:22,990 epoch 54 - iter 1060/2650 - loss 0.03989706 - time (sec): 74.25 - samples/sec: 7880.85 - lr: 0.050000 2023-04-06 08:03:41,141 epoch 54 - iter 1325/2650 - loss 0.03989365 - time (sec): 92.40 - samples/sec: 7902.12 - lr: 0.050000 2023-04-06 08:03:59,680 epoch 54 - iter 1590/2650 - loss 0.03987238 - time (sec): 110.94 - samples/sec: 7897.25 - lr: 0.050000 2023-04-06 08:04:18,393 epoch 54 - iter 1855/2650 - loss 0.04000256 - time (sec): 129.65 - samples/sec: 7886.29 - lr: 0.050000 2023-04-06 08:04:36,873 epoch 54 - iter 2120/2650 - loss 0.04006729 - time (sec): 148.13 - samples/sec: 7888.67 - lr: 0.050000 2023-04-06 08:04:55,632 epoch 54 - iter 2385/2650 - loss 0.04015447 - time (sec): 166.89 - samples/sec: 7891.01 - lr: 0.050000 2023-04-06 08:05:14,016 epoch 54 - iter 2650/2650 - loss 0.04023065 - time (sec): 185.27 - samples/sec: 7893.03 - lr: 0.050000 2023-04-06 08:05:14,017 ---------------------------------------------------------------------------------------------------- 2023-04-06 08:05:14,017 EPOCH 54 done: loss 0.0402 - lr 0.050000 2023-04-06 08:05:14,017 BAD EPOCHS (no improvement): 0 2023-04-06 08:05:14,020 ---------------------------------------------------------------------------------------------------- 2023-04-06 08:05:32,283 epoch 55 - iter 265/2650 - loss 0.04017155 - time (sec): 18.26 - samples/sec: 7986.01 - lr: 0.050000 2023-04-06 08:05:51,066 epoch 55 - iter 530/2650 - loss 0.03952158 - time (sec): 37.05 - samples/sec: 7888.44 - lr: 0.050000 2023-04-06 08:06:09,891 epoch 55 - iter 795/2650 - loss 0.03928623 - time (sec): 55.87 - samples/sec: 7869.57 - lr: 0.050000 2023-04-06 08:06:28,238 epoch 55 - iter 1060/2650 - loss 0.03923005 - time (sec): 74.22 - samples/sec: 7879.00 - lr: 0.050000 2023-04-06 08:06:46,407 epoch 55 - iter 1325/2650 - loss 0.03924633 - time (sec): 92.39 - samples/sec: 7898.76 - lr: 0.050000 2023-04-06 08:07:05,036 epoch 55 - iter 1590/2650 - loss 0.03939704 - time (sec): 111.02 - samples/sec: 7907.97 - lr: 0.050000 2023-04-06 08:07:23,577 epoch 55 - iter 1855/2650 - loss 0.03962985 - time (sec): 129.56 - samples/sec: 7902.99 - lr: 0.050000 2023-04-06 08:07:41,743 epoch 55 - iter 2120/2650 - loss 0.03968707 - time (sec): 147.72 - samples/sec: 7916.66 - lr: 0.050000 2023-04-06 08:08:00,280 epoch 55 - iter 2385/2650 - loss 0.03947652 - time (sec): 166.26 - samples/sec: 7918.39 - lr: 0.050000 2023-04-06 08:08:18,778 epoch 55 - iter 2650/2650 - loss 0.03942933 - time (sec): 184.76 - samples/sec: 7915.03 - lr: 0.050000 2023-04-06 08:08:18,778 ---------------------------------------------------------------------------------------------------- 2023-04-06 08:08:18,778 EPOCH 55 done: loss 0.0394 - lr 0.050000 2023-04-06 08:08:18,778 BAD EPOCHS (no improvement): 0 2023-04-06 08:08:18,781 ---------------------------------------------------------------------------------------------------- 2023-04-06 08:08:37,147 epoch 56 - iter 265/2650 - loss 0.03780974 - time (sec): 18.37 - samples/sec: 8005.77 - lr: 0.050000 2023-04-06 08:08:55,591 epoch 56 - iter 530/2650 - loss 0.03840103 - time (sec): 36.81 - samples/sec: 7956.35 - lr: 0.050000 2023-04-06 08:09:14,100 epoch 56 - iter 795/2650 - loss 0.03848071 - time (sec): 55.32 - samples/sec: 7927.45 - lr: 0.050000 2023-04-06 08:09:32,852 epoch 56 - iter 1060/2650 - loss 0.03867367 - time (sec): 74.07 - samples/sec: 7893.78 - lr: 0.050000 2023-04-06 08:09:51,126 epoch 56 - iter 1325/2650 - loss 0.03870451 - time (sec): 92.34 - samples/sec: 7910.51 - lr: 0.050000 2023-04-06 08:10:09,144 epoch 56 - iter 1590/2650 - loss 0.03860717 - time (sec): 110.36 - samples/sec: 7923.26 - lr: 0.050000 2023-04-06 08:10:27,602 epoch 56 - iter 1855/2650 - loss 0.03849407 - time (sec): 128.82 - samples/sec: 7929.91 - lr: 0.050000 2023-04-06 08:10:46,326 epoch 56 - iter 2120/2650 - loss 0.03871938 - time (sec): 147.55 - samples/sec: 7924.08 - lr: 0.050000 2023-04-06 08:11:04,864 epoch 56 - iter 2385/2650 - loss 0.03875875 - time (sec): 166.08 - samples/sec: 7913.49 - lr: 0.050000 2023-04-06 08:11:23,271 epoch 56 - iter 2650/2650 - loss 0.03894808 - time (sec): 184.49 - samples/sec: 7926.51 - lr: 0.050000 2023-04-06 08:11:23,271 ---------------------------------------------------------------------------------------------------- 2023-04-06 08:11:23,271 EPOCH 56 done: loss 0.0389 - lr 0.050000 2023-04-06 08:11:23,271 BAD EPOCHS (no improvement): 0 2023-04-06 08:11:23,274 ---------------------------------------------------------------------------------------------------- 2023-04-06 08:11:41,587 epoch 57 - iter 265/2650 - loss 0.03771233 - time (sec): 18.31 - samples/sec: 7958.56 - lr: 0.050000 2023-04-06 08:12:00,159 epoch 57 - iter 530/2650 - loss 0.03758825 - time (sec): 36.88 - samples/sec: 7890.89 - lr: 0.050000 2023-04-06 08:12:18,531 epoch 57 - iter 795/2650 - loss 0.03756020 - time (sec): 55.26 - samples/sec: 7891.80 - lr: 0.050000 2023-04-06 08:12:37,146 epoch 57 - iter 1060/2650 - loss 0.03829720 - time (sec): 73.87 - samples/sec: 7875.68 - lr: 0.050000 2023-04-06 08:12:55,638 epoch 57 - iter 1325/2650 - loss 0.03836526 - time (sec): 92.36 - samples/sec: 7882.03 - lr: 0.050000 2023-04-06 08:13:14,070 epoch 57 - iter 1590/2650 - loss 0.03856784 - time (sec): 110.80 - samples/sec: 7903.01 - lr: 0.050000 2023-04-06 08:13:32,690 epoch 57 - iter 1855/2650 - loss 0.03864894 - time (sec): 129.42 - samples/sec: 7912.21 - lr: 0.050000 2023-04-06 08:13:50,942 epoch 57 - iter 2120/2650 - loss 0.03859804 - time (sec): 147.67 - samples/sec: 7915.05 - lr: 0.050000 2023-04-06 08:14:09,891 epoch 57 - iter 2385/2650 - loss 0.03852514 - time (sec): 166.62 - samples/sec: 7904.95 - lr: 0.050000 2023-04-06 08:14:28,044 epoch 57 - iter 2650/2650 - loss 0.03855941 - time (sec): 184.77 - samples/sec: 7914.52 - lr: 0.050000 2023-04-06 08:14:28,044 ---------------------------------------------------------------------------------------------------- 2023-04-06 08:14:28,044 EPOCH 57 done: loss 0.0386 - lr 0.050000 2023-04-06 08:14:28,044 BAD EPOCHS (no improvement): 0 2023-04-06 08:14:28,047 ---------------------------------------------------------------------------------------------------- 2023-04-06 08:14:46,566 epoch 58 - iter 265/2650 - loss 0.03736573 - time (sec): 18.52 - samples/sec: 7870.13 - lr: 0.050000 2023-04-06 08:15:04,807 epoch 58 - iter 530/2650 - loss 0.03696909 - time (sec): 36.76 - samples/sec: 7948.33 - lr: 0.050000 2023-04-06 08:15:23,158 epoch 58 - iter 795/2650 - loss 0.03726252 - time (sec): 55.11 - samples/sec: 7932.98 - lr: 0.050000 2023-04-06 08:15:41,545 epoch 58 - iter 1060/2650 - loss 0.03739019 - time (sec): 73.50 - samples/sec: 7941.96 - lr: 0.050000 2023-04-06 08:16:00,035 epoch 58 - iter 1325/2650 - loss 0.03781461 - time (sec): 91.99 - samples/sec: 7925.80 - lr: 0.050000 2023-04-06 08:16:18,375 epoch 58 - iter 1590/2650 - loss 0.03778357 - time (sec): 110.33 - samples/sec: 7941.57 - lr: 0.050000 2023-04-06 08:16:37,059 epoch 58 - iter 1855/2650 - loss 0.03778931 - time (sec): 129.01 - samples/sec: 7935.86 - lr: 0.050000 2023-04-06 08:16:56,731 epoch 58 - iter 2120/2650 - loss 0.03780422 - time (sec): 148.68 - samples/sec: 7864.28 - lr: 0.050000 2023-04-06 08:17:15,593 epoch 58 - iter 2385/2650 - loss 0.03788493 - time (sec): 167.55 - samples/sec: 7862.84 - lr: 0.050000 2023-04-06 08:17:35,134 epoch 58 - iter 2650/2650 - loss 0.03796442 - time (sec): 187.09 - samples/sec: 7816.48 - lr: 0.050000 2023-04-06 08:17:35,135 ---------------------------------------------------------------------------------------------------- 2023-04-06 08:17:35,135 EPOCH 58 done: loss 0.0380 - lr 0.050000 2023-04-06 08:17:35,135 BAD EPOCHS (no improvement): 0 2023-04-06 08:17:35,137 ---------------------------------------------------------------------------------------------------- 2023-04-06 08:17:54,328 epoch 59 - iter 265/2650 - loss 0.03931499 - time (sec): 19.19 - samples/sec: 7597.10 - lr: 0.050000 2023-04-06 08:18:13,436 epoch 59 - iter 530/2650 - loss 0.03895720 - time (sec): 38.30 - samples/sec: 7671.49 - lr: 0.050000 2023-04-06 08:18:32,125 epoch 59 - iter 795/2650 - loss 0.03816632 - time (sec): 56.99 - samples/sec: 7756.61 - lr: 0.050000 2023-04-06 08:18:50,376 epoch 59 - iter 1060/2650 - loss 0.03828851 - time (sec): 75.24 - samples/sec: 7797.49 - lr: 0.050000 2023-04-06 08:19:08,591 epoch 59 - iter 1325/2650 - loss 0.03821523 - time (sec): 93.45 - samples/sec: 7835.54 - lr: 0.050000 2023-04-06 08:19:27,269 epoch 59 - iter 1590/2650 - loss 0.03822281 - time (sec): 112.13 - samples/sec: 7837.13 - lr: 0.050000 2023-04-06 08:19:46,439 epoch 59 - iter 1855/2650 - loss 0.03828985 - time (sec): 131.30 - samples/sec: 7816.09 - lr: 0.050000 2023-04-06 08:20:05,216 epoch 59 - iter 2120/2650 - loss 0.03821431 - time (sec): 150.08 - samples/sec: 7805.09 - lr: 0.050000 2023-04-06 08:20:23,926 epoch 59 - iter 2385/2650 - loss 0.03809873 - time (sec): 168.79 - samples/sec: 7805.61 - lr: 0.050000 2023-04-06 08:20:42,263 epoch 59 - iter 2650/2650 - loss 0.03810763 - time (sec): 187.13 - samples/sec: 7814.88 - lr: 0.050000 2023-04-06 08:20:42,263 ---------------------------------------------------------------------------------------------------- 2023-04-06 08:20:42,263 EPOCH 59 done: loss 0.0381 - lr 0.050000 2023-04-06 08:20:42,263 BAD EPOCHS (no improvement): 1 2023-04-06 08:20:42,266 ---------------------------------------------------------------------------------------------------- 2023-04-06 08:21:00,791 epoch 60 - iter 265/2650 - loss 0.03800331 - time (sec): 18.53 - samples/sec: 7850.79 - lr: 0.050000 2023-04-06 08:21:19,166 epoch 60 - iter 530/2650 - loss 0.03756505 - time (sec): 36.90 - samples/sec: 7924.36 - lr: 0.050000 2023-04-06 08:21:37,572 epoch 60 - iter 795/2650 - loss 0.03776981 - time (sec): 55.31 - samples/sec: 7950.23 - lr: 0.050000 2023-04-06 08:21:56,468 epoch 60 - iter 1060/2650 - loss 0.03747141 - time (sec): 74.20 - samples/sec: 7923.14 - lr: 0.050000 2023-04-06 08:22:15,089 epoch 60 - iter 1325/2650 - loss 0.03734002 - time (sec): 92.82 - samples/sec: 7912.39 - lr: 0.050000 2023-04-06 08:22:33,971 epoch 60 - iter 1590/2650 - loss 0.03725013 - time (sec): 111.71 - samples/sec: 7880.75 - lr: 0.050000 2023-04-06 08:22:52,191 epoch 60 - iter 1855/2650 - loss 0.03738627 - time (sec): 129.93 - samples/sec: 7895.21 - lr: 0.050000 2023-04-06 08:23:10,507 epoch 60 - iter 2120/2650 - loss 0.03745933 - time (sec): 148.24 - samples/sec: 7900.67 - lr: 0.050000 2023-04-06 08:23:29,567 epoch 60 - iter 2385/2650 - loss 0.03757382 - time (sec): 167.30 - samples/sec: 7871.50 - lr: 0.050000 2023-04-06 08:23:47,929 epoch 60 - iter 2650/2650 - loss 0.03772311 - time (sec): 185.66 - samples/sec: 7876.42 - lr: 0.050000 2023-04-06 08:23:47,930 ---------------------------------------------------------------------------------------------------- 2023-04-06 08:23:47,930 EPOCH 60 done: loss 0.0377 - lr 0.050000 2023-04-06 08:23:47,930 BAD EPOCHS (no improvement): 0 2023-04-06 08:23:47,932 ---------------------------------------------------------------------------------------------------- 2023-04-06 08:24:05,975 epoch 61 - iter 265/2650 - loss 0.03582226 - time (sec): 18.04 - samples/sec: 8093.14 - lr: 0.050000 2023-04-06 08:24:24,343 epoch 61 - iter 530/2650 - loss 0.03584937 - time (sec): 36.41 - samples/sec: 8025.32 - lr: 0.050000 2023-04-06 08:24:42,834 epoch 61 - iter 795/2650 - loss 0.03647279 - time (sec): 54.90 - samples/sec: 7964.88 - lr: 0.050000 2023-04-06 08:25:01,342 epoch 61 - iter 1060/2650 - loss 0.03685953 - time (sec): 73.41 - samples/sec: 7943.07 - lr: 0.050000 2023-04-06 08:25:20,052 epoch 61 - iter 1325/2650 - loss 0.03728120 - time (sec): 92.12 - samples/sec: 7919.43 - lr: 0.050000 2023-04-06 08:25:38,253 epoch 61 - iter 1590/2650 - loss 0.03717444 - time (sec): 110.32 - samples/sec: 7925.12 - lr: 0.050000 2023-04-06 08:25:56,382 epoch 61 - iter 1855/2650 - loss 0.03704189 - time (sec): 128.45 - samples/sec: 7935.55 - lr: 0.050000 2023-04-06 08:26:14,831 epoch 61 - iter 2120/2650 - loss 0.03720451 - time (sec): 146.90 - samples/sec: 7944.41 - lr: 0.050000 2023-04-06 08:26:33,422 epoch 61 - iter 2385/2650 - loss 0.03740467 - time (sec): 165.49 - samples/sec: 7942.47 - lr: 0.050000 2023-04-06 08:26:52,171 epoch 61 - iter 2650/2650 - loss 0.03748947 - time (sec): 184.24 - samples/sec: 7937.35 - lr: 0.050000 2023-04-06 08:26:52,171 ---------------------------------------------------------------------------------------------------- 2023-04-06 08:26:52,171 EPOCH 61 done: loss 0.0375 - lr 0.050000 2023-04-06 08:26:52,171 BAD EPOCHS (no improvement): 0 2023-04-06 08:26:52,174 ---------------------------------------------------------------------------------------------------- 2023-04-06 08:27:10,460 epoch 62 - iter 265/2650 - loss 0.03648005 - time (sec): 18.29 - samples/sec: 7974.39 - lr: 0.050000 2023-04-06 08:27:29,255 epoch 62 - iter 530/2650 - loss 0.03649473 - time (sec): 37.08 - samples/sec: 7902.28 - lr: 0.050000 2023-04-06 08:27:47,552 epoch 62 - iter 795/2650 - loss 0.03668905 - time (sec): 55.38 - samples/sec: 7928.94 - lr: 0.050000 2023-04-06 08:28:05,978 epoch 62 - iter 1060/2650 - loss 0.03664085 - time (sec): 73.80 - samples/sec: 7945.28 - lr: 0.050000 2023-04-06 08:28:24,582 epoch 62 - iter 1325/2650 - loss 0.03661351 - time (sec): 92.41 - samples/sec: 7938.56 - lr: 0.050000 2023-04-06 08:28:42,983 epoch 62 - iter 1590/2650 - loss 0.03664443 - time (sec): 110.81 - samples/sec: 7927.07 - lr: 0.050000 2023-04-06 08:29:01,280 epoch 62 - iter 1855/2650 - loss 0.03660260 - time (sec): 129.11 - samples/sec: 7931.89 - lr: 0.050000 2023-04-06 08:29:19,943 epoch 62 - iter 2120/2650 - loss 0.03670882 - time (sec): 147.77 - samples/sec: 7926.28 - lr: 0.050000 2023-04-06 08:29:38,227 epoch 62 - iter 2385/2650 - loss 0.03663047 - time (sec): 166.05 - samples/sec: 7933.11 - lr: 0.050000 2023-04-06 08:29:56,708 epoch 62 - iter 2650/2650 - loss 0.03676576 - time (sec): 184.53 - samples/sec: 7924.59 - lr: 0.050000 2023-04-06 08:29:56,709 ---------------------------------------------------------------------------------------------------- 2023-04-06 08:29:56,709 EPOCH 62 done: loss 0.0368 - lr 0.050000 2023-04-06 08:29:56,709 BAD EPOCHS (no improvement): 0 2023-04-06 08:29:56,712 ---------------------------------------------------------------------------------------------------- 2023-04-06 08:30:15,059 epoch 63 - iter 265/2650 - loss 0.03587844 - time (sec): 18.35 - samples/sec: 7958.13 - lr: 0.050000 2023-04-06 08:30:33,899 epoch 63 - iter 530/2650 - loss 0.03604145 - time (sec): 37.19 - samples/sec: 7902.06 - lr: 0.050000 2023-04-06 08:30:52,360 epoch 63 - iter 795/2650 - loss 0.03626954 - time (sec): 55.65 - samples/sec: 7919.57 - lr: 0.050000 2023-04-06 08:31:10,983 epoch 63 - iter 1060/2650 - loss 0.03616809 - time (sec): 74.27 - samples/sec: 7905.43 - lr: 0.050000 2023-04-06 08:31:29,493 epoch 63 - iter 1325/2650 - loss 0.03643307 - time (sec): 92.78 - samples/sec: 7907.63 - lr: 0.050000 2023-04-06 08:31:48,546 epoch 63 - iter 1590/2650 - loss 0.03668294 - time (sec): 111.83 - samples/sec: 7869.96 - lr: 0.050000 2023-04-06 08:32:06,751 epoch 63 - iter 1855/2650 - loss 0.03650228 - time (sec): 130.04 - samples/sec: 7886.77 - lr: 0.050000 2023-04-06 08:32:24,852 epoch 63 - iter 2120/2650 - loss 0.03643091 - time (sec): 148.14 - samples/sec: 7908.74 - lr: 0.050000 2023-04-06 08:32:43,235 epoch 63 - iter 2385/2650 - loss 0.03649633 - time (sec): 166.52 - samples/sec: 7905.78 - lr: 0.050000 2023-04-06 08:33:01,366 epoch 63 - iter 2650/2650 - loss 0.03657186 - time (sec): 184.65 - samples/sec: 7919.45 - lr: 0.050000 2023-04-06 08:33:01,367 ---------------------------------------------------------------------------------------------------- 2023-04-06 08:33:01,367 EPOCH 63 done: loss 0.0366 - lr 0.050000 2023-04-06 08:33:01,367 BAD EPOCHS (no improvement): 0 2023-04-06 08:33:01,374 ---------------------------------------------------------------------------------------------------- 2023-04-06 08:33:19,691 epoch 64 - iter 265/2650 - loss 0.03706833 - time (sec): 18.32 - samples/sec: 7936.82 - lr: 0.050000 2023-04-06 08:33:37,811 epoch 64 - iter 530/2650 - loss 0.03755918 - time (sec): 36.44 - samples/sec: 7963.78 - lr: 0.050000 2023-04-06 08:33:55,889 epoch 64 - iter 795/2650 - loss 0.03718159 - time (sec): 54.51 - samples/sec: 7976.12 - lr: 0.050000 2023-04-06 08:34:14,385 epoch 64 - iter 1060/2650 - loss 0.03682237 - time (sec): 73.01 - samples/sec: 7962.33 - lr: 0.050000 2023-04-06 08:34:33,002 epoch 64 - iter 1325/2650 - loss 0.03690525 - time (sec): 91.63 - samples/sec: 7953.32 - lr: 0.050000 2023-04-06 08:34:51,409 epoch 64 - iter 1590/2650 - loss 0.03674728 - time (sec): 110.03 - samples/sec: 7945.46 - lr: 0.050000 2023-04-06 08:35:09,722 epoch 64 - iter 1855/2650 - loss 0.03670028 - time (sec): 128.35 - samples/sec: 7965.19 - lr: 0.050000 2023-04-06 08:35:27,737 epoch 64 - iter 2120/2650 - loss 0.03669037 - time (sec): 146.36 - samples/sec: 7978.80 - lr: 0.050000 2023-04-06 08:35:46,763 epoch 64 - iter 2385/2650 - loss 0.03673285 - time (sec): 165.39 - samples/sec: 7948.64 - lr: 0.050000 2023-04-06 08:36:05,516 epoch 64 - iter 2650/2650 - loss 0.03684009 - time (sec): 184.14 - samples/sec: 7941.53 - lr: 0.050000 2023-04-06 08:36:05,516 ---------------------------------------------------------------------------------------------------- 2023-04-06 08:36:05,516 EPOCH 64 done: loss 0.0368 - lr 0.050000 2023-04-06 08:36:05,516 BAD EPOCHS (no improvement): 1 2023-04-06 08:36:05,519 ---------------------------------------------------------------------------------------------------- 2023-04-06 08:36:23,629 epoch 65 - iter 265/2650 - loss 0.03635001 - time (sec): 18.11 - samples/sec: 8051.20 - lr: 0.050000 2023-04-06 08:36:42,094 epoch 65 - iter 530/2650 - loss 0.03614812 - time (sec): 36.58 - samples/sec: 8000.43 - lr: 0.050000 2023-04-06 08:37:00,547 epoch 65 - iter 795/2650 - loss 0.03601744 - time (sec): 55.03 - samples/sec: 7986.94 - lr: 0.050000 2023-04-06 08:37:19,438 epoch 65 - iter 1060/2650 - loss 0.03609651 - time (sec): 73.92 - samples/sec: 7946.31 - lr: 0.050000 2023-04-06 08:37:38,020 epoch 65 - iter 1325/2650 - loss 0.03624251 - time (sec): 92.50 - samples/sec: 7936.37 - lr: 0.050000 2023-04-06 08:37:56,802 epoch 65 - iter 1590/2650 - loss 0.03620736 - time (sec): 111.28 - samples/sec: 7922.74 - lr: 0.050000 2023-04-06 08:38:15,079 epoch 65 - iter 1855/2650 - loss 0.03621544 - time (sec): 129.56 - samples/sec: 7917.46 - lr: 0.050000 2023-04-06 08:38:34,581 epoch 65 - iter 2120/2650 - loss 0.03631300 - time (sec): 149.06 - samples/sec: 7870.26 - lr: 0.050000 2023-04-06 08:38:53,809 epoch 65 - iter 2385/2650 - loss 0.03625454 - time (sec): 168.29 - samples/sec: 7830.65 - lr: 0.050000 2023-04-06 08:39:13,232 epoch 65 - iter 2650/2650 - loss 0.03635050 - time (sec): 187.71 - samples/sec: 7790.43 - lr: 0.050000 2023-04-06 08:39:13,232 ---------------------------------------------------------------------------------------------------- 2023-04-06 08:39:13,232 EPOCH 65 done: loss 0.0364 - lr 0.050000 2023-04-06 08:39:13,232 BAD EPOCHS (no improvement): 0 2023-04-06 08:39:13,235 ---------------------------------------------------------------------------------------------------- 2023-04-06 08:39:32,497 epoch 66 - iter 265/2650 - loss 0.03544732 - time (sec): 19.26 - samples/sec: 7574.47 - lr: 0.050000 2023-04-06 08:39:51,228 epoch 66 - iter 530/2650 - loss 0.03572977 - time (sec): 37.99 - samples/sec: 7723.05 - lr: 0.050000 2023-04-06 08:40:09,612 epoch 66 - iter 795/2650 - loss 0.03545739 - time (sec): 56.38 - samples/sec: 7822.59 - lr: 0.050000 2023-04-06 08:40:28,007 epoch 66 - iter 1060/2650 - loss 0.03546893 - time (sec): 74.77 - samples/sec: 7863.52 - lr: 0.050000 2023-04-06 08:40:46,551 epoch 66 - iter 1325/2650 - loss 0.03556712 - time (sec): 93.32 - samples/sec: 7840.41 - lr: 0.050000 2023-04-06 08:41:05,016 epoch 66 - iter 1590/2650 - loss 0.03547528 - time (sec): 111.78 - samples/sec: 7852.76 - lr: 0.050000 2023-04-06 08:41:24,252 epoch 66 - iter 1855/2650 - loss 0.03565681 - time (sec): 131.02 - samples/sec: 7812.36 - lr: 0.050000 2023-04-06 08:41:42,985 epoch 66 - iter 2120/2650 - loss 0.03574899 - time (sec): 149.75 - samples/sec: 7810.74 - lr: 0.050000 2023-04-06 08:42:01,278 epoch 66 - iter 2385/2650 - loss 0.03590109 - time (sec): 168.04 - samples/sec: 7828.77 - lr: 0.050000 2023-04-06 08:42:20,211 epoch 66 - iter 2650/2650 - loss 0.03607360 - time (sec): 186.98 - samples/sec: 7821.14 - lr: 0.050000 2023-04-06 08:42:20,211 ---------------------------------------------------------------------------------------------------- 2023-04-06 08:42:20,211 EPOCH 66 done: loss 0.0361 - lr 0.050000 2023-04-06 08:42:20,211 BAD EPOCHS (no improvement): 0 2023-04-06 08:42:20,214 ---------------------------------------------------------------------------------------------------- 2023-04-06 08:42:39,650 epoch 67 - iter 265/2650 - loss 0.03502976 - time (sec): 19.44 - samples/sec: 7541.18 - lr: 0.050000 2023-04-06 08:42:58,016 epoch 67 - iter 530/2650 - loss 0.03555262 - time (sec): 37.80 - samples/sec: 7741.18 - lr: 0.050000 2023-04-06 08:43:16,294 epoch 67 - iter 795/2650 - loss 0.03544218 - time (sec): 56.08 - samples/sec: 7810.51 - lr: 0.050000 2023-04-06 08:43:35,071 epoch 67 - iter 1060/2650 - loss 0.03546233 - time (sec): 74.86 - samples/sec: 7817.66 - lr: 0.050000 2023-04-06 08:43:53,338 epoch 67 - iter 1325/2650 - loss 0.03529041 - time (sec): 93.12 - samples/sec: 7843.31 - lr: 0.050000 2023-04-06 08:44:11,435 epoch 67 - iter 1590/2650 - loss 0.03556601 - time (sec): 111.22 - samples/sec: 7870.09 - lr: 0.050000 2023-04-06 08:44:30,285 epoch 67 - iter 1855/2650 - loss 0.03578601 - time (sec): 130.07 - samples/sec: 7871.65 - lr: 0.050000 2023-04-06 08:44:48,925 epoch 67 - iter 2120/2650 - loss 0.03568597 - time (sec): 148.71 - samples/sec: 7865.92 - lr: 0.050000 2023-04-06 08:45:07,150 epoch 67 - iter 2385/2650 - loss 0.03583832 - time (sec): 166.94 - samples/sec: 7883.25 - lr: 0.050000 2023-04-06 08:45:25,245 epoch 67 - iter 2650/2650 - loss 0.03584341 - time (sec): 185.03 - samples/sec: 7903.38 - lr: 0.050000 2023-04-06 08:45:25,245 ---------------------------------------------------------------------------------------------------- 2023-04-06 08:45:25,245 EPOCH 67 done: loss 0.0358 - lr 0.050000 2023-04-06 08:45:25,245 BAD EPOCHS (no improvement): 0 2023-04-06 08:45:25,249 ---------------------------------------------------------------------------------------------------- 2023-04-06 08:45:43,583 epoch 68 - iter 265/2650 - loss 0.03412143 - time (sec): 18.33 - samples/sec: 7969.54 - lr: 0.050000 2023-04-06 08:46:01,902 epoch 68 - iter 530/2650 - loss 0.03546672 - time (sec): 36.65 - samples/sec: 7958.85 - lr: 0.050000 2023-04-06 08:46:20,608 epoch 68 - iter 795/2650 - loss 0.03524105 - time (sec): 55.36 - samples/sec: 7934.37 - lr: 0.050000 2023-04-06 08:46:38,732 epoch 68 - iter 1060/2650 - loss 0.03511720 - time (sec): 73.48 - samples/sec: 7979.51 - lr: 0.050000 2023-04-06 08:46:57,514 epoch 68 - iter 1325/2650 - loss 0.03534079 - time (sec): 92.27 - samples/sec: 7967.23 - lr: 0.050000 2023-04-06 08:47:16,225 epoch 68 - iter 1590/2650 - loss 0.03555314 - time (sec): 110.98 - samples/sec: 7940.46 - lr: 0.050000 2023-04-06 08:47:34,143 epoch 68 - iter 1855/2650 - loss 0.03543341 - time (sec): 128.89 - samples/sec: 7959.52 - lr: 0.050000 2023-04-06 08:47:52,051 epoch 68 - iter 2120/2650 - loss 0.03564632 - time (sec): 146.80 - samples/sec: 7978.36 - lr: 0.050000 2023-04-06 08:48:10,428 epoch 68 - iter 2385/2650 - loss 0.03545061 - time (sec): 165.18 - samples/sec: 7973.33 - lr: 0.050000 2023-04-06 08:48:28,694 epoch 68 - iter 2650/2650 - loss 0.03547201 - time (sec): 183.44 - samples/sec: 7971.68 - lr: 0.050000 2023-04-06 08:48:28,694 ---------------------------------------------------------------------------------------------------- 2023-04-06 08:48:28,694 EPOCH 68 done: loss 0.0355 - lr 0.050000 2023-04-06 08:48:28,694 BAD EPOCHS (no improvement): 0 2023-04-06 08:48:28,698 ---------------------------------------------------------------------------------------------------- 2023-04-06 08:48:47,206 epoch 69 - iter 265/2650 - loss 0.03508737 - time (sec): 18.51 - samples/sec: 7901.33 - lr: 0.050000 2023-04-06 08:49:06,144 epoch 69 - iter 530/2650 - loss 0.03481490 - time (sec): 37.45 - samples/sec: 7834.41 - lr: 0.050000 2023-04-06 08:49:24,304 epoch 69 - iter 795/2650 - loss 0.03518114 - time (sec): 55.61 - samples/sec: 7921.99 - lr: 0.050000 2023-04-06 08:49:42,605 epoch 69 - iter 1060/2650 - loss 0.03517075 - time (sec): 73.91 - samples/sec: 7938.92 - lr: 0.050000 2023-04-06 08:50:00,679 epoch 69 - iter 1325/2650 - loss 0.03541748 - time (sec): 91.98 - samples/sec: 7970.09 - lr: 0.050000 2023-04-06 08:50:19,092 epoch 69 - iter 1590/2650 - loss 0.03532515 - time (sec): 110.39 - samples/sec: 7976.91 - lr: 0.050000 2023-04-06 08:50:36,871 epoch 69 - iter 1855/2650 - loss 0.03537381 - time (sec): 128.17 - samples/sec: 7995.46 - lr: 0.050000 2023-04-06 08:50:54,942 epoch 69 - iter 2120/2650 - loss 0.03525958 - time (sec): 146.24 - samples/sec: 8002.67 - lr: 0.050000 2023-04-06 08:51:13,154 epoch 69 - iter 2385/2650 - loss 0.03524298 - time (sec): 164.46 - samples/sec: 8013.36 - lr: 0.050000 2023-04-06 08:51:31,149 epoch 69 - iter 2650/2650 - loss 0.03515726 - time (sec): 182.45 - samples/sec: 8015.07 - lr: 0.050000 2023-04-06 08:51:31,150 ---------------------------------------------------------------------------------------------------- 2023-04-06 08:51:31,150 EPOCH 69 done: loss 0.0352 - lr 0.050000 2023-04-06 08:51:31,150 BAD EPOCHS (no improvement): 0 2023-04-06 08:51:31,153 ---------------------------------------------------------------------------------------------------- 2023-04-06 08:51:49,305 epoch 70 - iter 265/2650 - loss 0.03476939 - time (sec): 18.15 - samples/sec: 8119.74 - lr: 0.050000 2023-04-06 08:52:07,600 epoch 70 - iter 530/2650 - loss 0.03473854 - time (sec): 36.45 - samples/sec: 8082.29 - lr: 0.050000 2023-04-06 08:52:25,887 epoch 70 - iter 795/2650 - loss 0.03473109 - time (sec): 54.73 - samples/sec: 8066.52 - lr: 0.050000 2023-04-06 08:52:44,384 epoch 70 - iter 1060/2650 - loss 0.03484515 - time (sec): 73.23 - samples/sec: 8026.41 - lr: 0.050000 2023-04-06 08:53:02,237 epoch 70 - iter 1325/2650 - loss 0.03469703 - time (sec): 91.08 - samples/sec: 8040.34 - lr: 0.050000 2023-04-06 08:53:20,283 epoch 70 - iter 1590/2650 - loss 0.03489442 - time (sec): 109.13 - samples/sec: 8051.13 - lr: 0.050000 2023-04-06 08:53:38,566 epoch 70 - iter 1855/2650 - loss 0.03490723 - time (sec): 127.41 - samples/sec: 8054.37 - lr: 0.050000 2023-04-06 08:53:56,418 epoch 70 - iter 2120/2650 - loss 0.03480659 - time (sec): 145.26 - samples/sec: 8068.93 - lr: 0.050000 2023-04-06 08:54:14,515 epoch 70 - iter 2385/2650 - loss 0.03490544 - time (sec): 163.36 - samples/sec: 8057.86 - lr: 0.050000 2023-04-06 08:54:33,488 epoch 70 - iter 2650/2650 - loss 0.03485682 - time (sec): 182.34 - samples/sec: 8020.19 - lr: 0.050000 2023-04-06 08:54:33,488 ---------------------------------------------------------------------------------------------------- 2023-04-06 08:54:33,488 EPOCH 70 done: loss 0.0349 - lr 0.050000 2023-04-06 08:54:33,488 BAD EPOCHS (no improvement): 0 2023-04-06 08:54:33,492 ---------------------------------------------------------------------------------------------------- 2023-04-06 08:54:51,849 epoch 71 - iter 265/2650 - loss 0.03522536 - time (sec): 18.36 - samples/sec: 7989.59 - lr: 0.050000 2023-04-06 08:55:10,136 epoch 71 - iter 530/2650 - loss 0.03473894 - time (sec): 36.64 - samples/sec: 7977.37 - lr: 0.050000 2023-04-06 08:55:27,926 epoch 71 - iter 795/2650 - loss 0.03482672 - time (sec): 54.43 - samples/sec: 8057.75 - lr: 0.050000 2023-04-06 08:55:46,217 epoch 71 - iter 1060/2650 - loss 0.03525877 - time (sec): 72.72 - samples/sec: 8042.05 - lr: 0.050000 2023-04-06 08:56:04,886 epoch 71 - iter 1325/2650 - loss 0.03526755 - time (sec): 91.39 - samples/sec: 7995.04 - lr: 0.050000 2023-04-06 08:56:23,712 epoch 71 - iter 1590/2650 - loss 0.03548702 - time (sec): 110.22 - samples/sec: 7956.58 - lr: 0.050000 2023-04-06 08:56:42,091 epoch 71 - iter 1855/2650 - loss 0.03521969 - time (sec): 128.60 - samples/sec: 7954.16 - lr: 0.050000 2023-04-06 08:57:00,413 epoch 71 - iter 2120/2650 - loss 0.03530901 - time (sec): 146.92 - samples/sec: 7953.94 - lr: 0.050000 2023-04-06 08:57:19,322 epoch 71 - iter 2385/2650 - loss 0.03532913 - time (sec): 165.83 - samples/sec: 7938.03 - lr: 0.050000 2023-04-06 08:57:38,675 epoch 71 - iter 2650/2650 - loss 0.03527969 - time (sec): 185.18 - samples/sec: 7896.86 - lr: 0.050000 2023-04-06 08:57:38,675 ---------------------------------------------------------------------------------------------------- 2023-04-06 08:57:38,675 EPOCH 71 done: loss 0.0353 - lr 0.050000 2023-04-06 08:57:38,675 BAD EPOCHS (no improvement): 1 2023-04-06 08:57:38,678 ---------------------------------------------------------------------------------------------------- 2023-04-06 08:57:57,553 epoch 72 - iter 265/2650 - loss 0.03345234 - time (sec): 18.87 - samples/sec: 7833.67 - lr: 0.050000 2023-04-06 08:58:15,858 epoch 72 - iter 530/2650 - loss 0.03379768 - time (sec): 37.18 - samples/sec: 7884.81 - lr: 0.050000 2023-04-06 08:58:34,237 epoch 72 - iter 795/2650 - loss 0.03408444 - time (sec): 55.56 - samples/sec: 7933.06 - lr: 0.050000 2023-04-06 08:58:52,637 epoch 72 - iter 1060/2650 - loss 0.03423549 - time (sec): 73.96 - samples/sec: 7938.81 - lr: 0.050000 2023-04-06 08:59:10,866 epoch 72 - iter 1325/2650 - loss 0.03458923 - time (sec): 92.19 - samples/sec: 7956.19 - lr: 0.050000 2023-04-06 08:59:29,334 epoch 72 - iter 1590/2650 - loss 0.03460055 - time (sec): 110.65 - samples/sec: 7954.93 - lr: 0.050000 2023-04-06 08:59:47,826 epoch 72 - iter 1855/2650 - loss 0.03475157 - time (sec): 129.15 - samples/sec: 7939.87 - lr: 0.050000 2023-04-06 09:00:05,885 epoch 72 - iter 2120/2650 - loss 0.03473314 - time (sec): 147.21 - samples/sec: 7949.17 - lr: 0.050000 2023-04-06 09:00:24,434 epoch 72 - iter 2385/2650 - loss 0.03476465 - time (sec): 165.76 - samples/sec: 7941.15 - lr: 0.050000 2023-04-06 09:00:42,342 epoch 72 - iter 2650/2650 - loss 0.03482905 - time (sec): 183.66 - samples/sec: 7962.19 - lr: 0.050000 2023-04-06 09:00:42,342 ---------------------------------------------------------------------------------------------------- 2023-04-06 09:00:42,342 EPOCH 72 done: loss 0.0348 - lr 0.050000 2023-04-06 09:00:42,342 BAD EPOCHS (no improvement): 0 2023-04-06 09:00:42,346 ---------------------------------------------------------------------------------------------------- 2023-04-06 09:01:00,594 epoch 73 - iter 265/2650 - loss 0.03402814 - time (sec): 18.25 - samples/sec: 7998.24 - lr: 0.050000 2023-04-06 09:01:18,968 epoch 73 - iter 530/2650 - loss 0.03313738 - time (sec): 36.62 - samples/sec: 8006.18 - lr: 0.050000 2023-04-06 09:01:37,329 epoch 73 - iter 795/2650 - loss 0.03313602 - time (sec): 54.98 - samples/sec: 7982.36 - lr: 0.050000 2023-04-06 09:01:55,820 epoch 73 - iter 1060/2650 - loss 0.03369596 - time (sec): 73.47 - samples/sec: 7958.29 - lr: 0.050000 2023-04-06 09:02:14,131 epoch 73 - iter 1325/2650 - loss 0.03344482 - time (sec): 91.79 - samples/sec: 7972.26 - lr: 0.050000 2023-04-06 09:02:32,443 epoch 73 - iter 1590/2650 - loss 0.03361579 - time (sec): 110.10 - samples/sec: 7973.73 - lr: 0.050000 2023-04-06 09:02:51,032 epoch 73 - iter 1855/2650 - loss 0.03396642 - time (sec): 128.69 - samples/sec: 7955.15 - lr: 0.050000 2023-04-06 09:03:09,746 epoch 73 - iter 2120/2650 - loss 0.03400095 - time (sec): 147.40 - samples/sec: 7940.02 - lr: 0.050000 2023-04-06 09:03:28,887 epoch 73 - iter 2385/2650 - loss 0.03394283 - time (sec): 166.54 - samples/sec: 7905.35 - lr: 0.050000 2023-04-06 09:03:47,442 epoch 73 - iter 2650/2650 - loss 0.03408357 - time (sec): 185.10 - samples/sec: 7900.55 - lr: 0.050000 2023-04-06 09:03:47,442 ---------------------------------------------------------------------------------------------------- 2023-04-06 09:03:47,442 EPOCH 73 done: loss 0.0341 - lr 0.050000 2023-04-06 09:03:47,443 BAD EPOCHS (no improvement): 0 2023-04-06 09:03:47,447 ---------------------------------------------------------------------------------------------------- 2023-04-06 09:04:06,213 epoch 74 - iter 265/2650 - loss 0.03443911 - time (sec): 18.77 - samples/sec: 7900.30 - lr: 0.050000 2023-04-06 09:04:24,235 epoch 74 - iter 530/2650 - loss 0.03503425 - time (sec): 36.79 - samples/sec: 7932.38 - lr: 0.050000 2023-04-06 09:04:42,658 epoch 74 - iter 795/2650 - loss 0.03462991 - time (sec): 55.21 - samples/sec: 7961.50 - lr: 0.050000 2023-04-06 09:05:01,504 epoch 74 - iter 1060/2650 - loss 0.03433172 - time (sec): 74.06 - samples/sec: 7914.64 - lr: 0.050000 2023-04-06 09:05:20,197 epoch 74 - iter 1325/2650 - loss 0.03435563 - time (sec): 92.75 - samples/sec: 7885.00 - lr: 0.050000 2023-04-06 09:05:39,063 epoch 74 - iter 1590/2650 - loss 0.03431365 - time (sec): 111.62 - samples/sec: 7850.98 - lr: 0.050000 2023-04-06 09:05:59,828 epoch 74 - iter 1855/2650 - loss 0.03409831 - time (sec): 132.38 - samples/sec: 7737.78 - lr: 0.050000 2023-04-06 09:06:18,884 epoch 74 - iter 2120/2650 - loss 0.03423670 - time (sec): 151.44 - samples/sec: 7715.28 - lr: 0.050000 2023-04-06 09:06:37,590 epoch 74 - iter 2385/2650 - loss 0.03446479 - time (sec): 170.14 - samples/sec: 7728.32 - lr: 0.050000 2023-04-06 09:06:56,613 epoch 74 - iter 2650/2650 - loss 0.03450510 - time (sec): 189.17 - samples/sec: 7730.59 - lr: 0.050000 2023-04-06 09:06:56,613 ---------------------------------------------------------------------------------------------------- 2023-04-06 09:06:56,613 EPOCH 74 done: loss 0.0345 - lr 0.050000 2023-04-06 09:06:56,613 BAD EPOCHS (no improvement): 1 2023-04-06 09:06:56,617 ---------------------------------------------------------------------------------------------------- 2023-04-06 09:07:14,875 epoch 75 - iter 265/2650 - loss 0.03265029 - time (sec): 18.26 - samples/sec: 7985.09 - lr: 0.050000 2023-04-06 09:07:33,581 epoch 75 - iter 530/2650 - loss 0.03362194 - time (sec): 36.96 - samples/sec: 7959.17 - lr: 0.050000 2023-04-06 09:07:53,215 epoch 75 - iter 795/2650 - loss 0.03399037 - time (sec): 56.60 - samples/sec: 7764.31 - lr: 0.050000 2023-04-06 09:08:11,666 epoch 75 - iter 1060/2650 - loss 0.03392145 - time (sec): 75.05 - samples/sec: 7781.86 - lr: 0.050000 2023-04-06 09:08:31,387 epoch 75 - iter 1325/2650 - loss 0.03393355 - time (sec): 94.77 - samples/sec: 7692.54 - lr: 0.050000 2023-04-06 09:08:49,891 epoch 75 - iter 1590/2650 - loss 0.03402442 - time (sec): 113.27 - samples/sec: 7736.46 - lr: 0.050000 2023-04-06 09:09:09,738 epoch 75 - iter 1855/2650 - loss 0.03419250 - time (sec): 133.12 - samples/sec: 7683.74 - lr: 0.050000 2023-04-06 09:09:27,886 epoch 75 - iter 2120/2650 - loss 0.03431212 - time (sec): 151.27 - samples/sec: 7722.66 - lr: 0.050000 2023-04-06 09:09:47,733 epoch 75 - iter 2385/2650 - loss 0.03427570 - time (sec): 171.12 - samples/sec: 7685.52 - lr: 0.050000 2023-04-06 09:10:06,189 epoch 75 - iter 2650/2650 - loss 0.03447008 - time (sec): 189.57 - samples/sec: 7714.04 - lr: 0.050000 2023-04-06 09:10:06,189 ---------------------------------------------------------------------------------------------------- 2023-04-06 09:10:06,189 EPOCH 75 done: loss 0.0345 - lr 0.050000 2023-04-06 09:10:06,189 BAD EPOCHS (no improvement): 2 2023-04-06 09:10:06,192 ---------------------------------------------------------------------------------------------------- 2023-04-06 09:10:24,448 epoch 76 - iter 265/2650 - loss 0.03380093 - time (sec): 18.26 - samples/sec: 7970.67 - lr: 0.050000 2023-04-06 09:10:42,466 epoch 76 - iter 530/2650 - loss 0.03380657 - time (sec): 36.27 - samples/sec: 8018.75 - lr: 0.050000 2023-04-06 09:11:00,587 epoch 76 - iter 795/2650 - loss 0.03401981 - time (sec): 54.40 - samples/sec: 8061.68 - lr: 0.050000 2023-04-06 09:11:18,803 epoch 76 - iter 1060/2650 - loss 0.03405491 - time (sec): 72.61 - samples/sec: 8029.48 - lr: 0.050000 2023-04-06 09:11:37,339 epoch 76 - iter 1325/2650 - loss 0.03416638 - time (sec): 91.15 - samples/sec: 8017.83 - lr: 0.050000 2023-04-06 09:11:56,073 epoch 76 - iter 1590/2650 - loss 0.03439098 - time (sec): 109.88 - samples/sec: 8004.14 - lr: 0.050000 2023-04-06 09:12:14,677 epoch 76 - iter 1855/2650 - loss 0.03431106 - time (sec): 128.49 - samples/sec: 7994.08 - lr: 0.050000 2023-04-06 09:12:33,319 epoch 76 - iter 2120/2650 - loss 0.03433939 - time (sec): 147.13 - samples/sec: 7964.39 - lr: 0.050000 2023-04-06 09:12:51,987 epoch 76 - iter 2385/2650 - loss 0.03439578 - time (sec): 165.79 - samples/sec: 7941.39 - lr: 0.050000 2023-04-06 09:13:11,924 epoch 76 - iter 2650/2650 - loss 0.03452267 - time (sec): 185.73 - samples/sec: 7873.54 - lr: 0.050000 2023-04-06 09:13:11,924 ---------------------------------------------------------------------------------------------------- 2023-04-06 09:13:11,924 EPOCH 76 done: loss 0.0345 - lr 0.050000 2023-04-06 09:13:11,924 BAD EPOCHS (no improvement): 3 2023-04-06 09:13:11,927 ---------------------------------------------------------------------------------------------------- 2023-04-06 09:13:30,412 epoch 77 - iter 265/2650 - loss 0.03406618 - time (sec): 18.48 - samples/sec: 7832.87 - lr: 0.050000 2023-04-06 09:13:49,386 epoch 77 - iter 530/2650 - loss 0.03365991 - time (sec): 37.46 - samples/sec: 7782.33 - lr: 0.050000 2023-04-06 09:14:08,682 epoch 77 - iter 795/2650 - loss 0.03344780 - time (sec): 56.75 - samples/sec: 7723.00 - lr: 0.050000 2023-04-06 09:14:27,308 epoch 77 - iter 1060/2650 - loss 0.03346557 - time (sec): 75.38 - samples/sec: 7767.31 - lr: 0.050000 2023-04-06 09:14:46,103 epoch 77 - iter 1325/2650 - loss 0.03384522 - time (sec): 94.18 - samples/sec: 7770.79 - lr: 0.050000 2023-04-06 09:15:04,625 epoch 77 - iter 1590/2650 - loss 0.03394696 - time (sec): 112.70 - samples/sec: 7784.16 - lr: 0.050000 2023-04-06 09:15:23,293 epoch 77 - iter 1855/2650 - loss 0.03402563 - time (sec): 131.37 - samples/sec: 7798.42 - lr: 0.050000 2023-04-06 09:15:41,853 epoch 77 - iter 2120/2650 - loss 0.03403879 - time (sec): 149.93 - samples/sec: 7804.01 - lr: 0.050000 2023-04-06 09:15:59,904 epoch 77 - iter 2385/2650 - loss 0.03424196 - time (sec): 167.98 - samples/sec: 7829.91 - lr: 0.050000 2023-04-06 09:16:18,865 epoch 77 - iter 2650/2650 - loss 0.03408478 - time (sec): 186.94 - samples/sec: 7822.72 - lr: 0.050000 2023-04-06 09:16:18,866 ---------------------------------------------------------------------------------------------------- 2023-04-06 09:16:18,866 EPOCH 77 done: loss 0.0341 - lr 0.050000 2023-04-06 09:16:18,866 Epoch 77: reducing learning rate of group 0 to 2.5000e-02. 2023-04-06 09:16:18,866 BAD EPOCHS (no improvement): 4 2023-04-06 09:16:18,869 ---------------------------------------------------------------------------------------------------- 2023-04-06 09:16:37,446 epoch 78 - iter 265/2650 - loss 0.03447249 - time (sec): 18.58 - samples/sec: 7897.46 - lr: 0.025000 2023-04-06 09:16:55,852 epoch 78 - iter 530/2650 - loss 0.03400949 - time (sec): 36.98 - samples/sec: 7926.20 - lr: 0.025000 2023-04-06 09:17:14,294 epoch 78 - iter 795/2650 - loss 0.03400474 - time (sec): 55.43 - samples/sec: 7920.35 - lr: 0.025000 2023-04-06 09:17:32,913 epoch 78 - iter 1060/2650 - loss 0.03374163 - time (sec): 74.04 - samples/sec: 7903.69 - lr: 0.025000 2023-04-06 09:17:50,942 epoch 78 - iter 1325/2650 - loss 0.03372852 - time (sec): 92.07 - samples/sec: 7943.15 - lr: 0.025000 2023-04-06 09:18:09,372 epoch 78 - iter 1590/2650 - loss 0.03360985 - time (sec): 110.50 - samples/sec: 7941.66 - lr: 0.025000 2023-04-06 09:18:27,383 epoch 78 - iter 1855/2650 - loss 0.03345230 - time (sec): 128.51 - samples/sec: 7959.21 - lr: 0.025000 2023-04-06 09:18:46,329 epoch 78 - iter 2120/2650 - loss 0.03346955 - time (sec): 147.46 - samples/sec: 7939.64 - lr: 0.025000 2023-04-06 09:19:04,539 epoch 78 - iter 2385/2650 - loss 0.03344710 - time (sec): 165.67 - samples/sec: 7941.78 - lr: 0.025000 2023-04-06 09:19:22,697 epoch 78 - iter 2650/2650 - loss 0.03338020 - time (sec): 183.83 - samples/sec: 7955.06 - lr: 0.025000 2023-04-06 09:19:22,697 ---------------------------------------------------------------------------------------------------- 2023-04-06 09:19:22,697 EPOCH 78 done: loss 0.0334 - lr 0.025000 2023-04-06 09:19:22,697 BAD EPOCHS (no improvement): 0 2023-04-06 09:19:22,701 ---------------------------------------------------------------------------------------------------- 2023-04-06 09:19:40,703 epoch 79 - iter 265/2650 - loss 0.03348908 - time (sec): 18.00 - samples/sec: 8152.36 - lr: 0.025000 2023-04-06 09:19:58,520 epoch 79 - iter 530/2650 - loss 0.03287950 - time (sec): 35.82 - samples/sec: 8105.59 - lr: 0.025000 2023-04-06 09:20:17,141 epoch 79 - iter 795/2650 - loss 0.03280051 - time (sec): 54.44 - samples/sec: 8038.34 - lr: 0.025000 2023-04-06 09:20:35,279 epoch 79 - iter 1060/2650 - loss 0.03261065 - time (sec): 72.58 - samples/sec: 8032.22 - lr: 0.025000 2023-04-06 09:20:53,521 epoch 79 - iter 1325/2650 - loss 0.03241404 - time (sec): 90.82 - samples/sec: 8016.54 - lr: 0.025000 2023-04-06 09:21:11,864 epoch 79 - iter 1590/2650 - loss 0.03238196 - time (sec): 109.16 - samples/sec: 8006.42 - lr: 0.025000 2023-04-06 09:21:30,060 epoch 79 - iter 1855/2650 - loss 0.03229641 - time (sec): 127.36 - samples/sec: 8029.83 - lr: 0.025000 2023-04-06 09:21:47,811 epoch 79 - iter 2120/2650 - loss 0.03245474 - time (sec): 145.11 - samples/sec: 8045.32 - lr: 0.025000 2023-04-06 09:22:06,164 epoch 79 - iter 2385/2650 - loss 0.03244314 - time (sec): 163.46 - samples/sec: 8039.91 - lr: 0.025000 2023-04-06 09:22:24,280 epoch 79 - iter 2650/2650 - loss 0.03242759 - time (sec): 181.58 - samples/sec: 8053.57 - lr: 0.025000 2023-04-06 09:22:24,280 ---------------------------------------------------------------------------------------------------- 2023-04-06 09:22:24,280 EPOCH 79 done: loss 0.0324 - lr 0.025000 2023-04-06 09:22:24,281 BAD EPOCHS (no improvement): 0 2023-04-06 09:22:24,284 ---------------------------------------------------------------------------------------------------- 2023-04-06 09:22:42,772 epoch 80 - iter 265/2650 - loss 0.03294178 - time (sec): 18.49 - samples/sec: 7821.72 - lr: 0.025000 2023-04-06 09:23:00,727 epoch 80 - iter 530/2650 - loss 0.03265089 - time (sec): 36.44 - samples/sec: 7939.86 - lr: 0.025000 2023-04-06 09:23:19,120 epoch 80 - iter 795/2650 - loss 0.03186263 - time (sec): 54.84 - samples/sec: 7956.55 - lr: 0.025000 2023-04-06 09:23:37,507 epoch 80 - iter 1060/2650 - loss 0.03216290 - time (sec): 73.22 - samples/sec: 7965.32 - lr: 0.025000 2023-04-06 09:23:55,759 epoch 80 - iter 1325/2650 - loss 0.03241497 - time (sec): 91.48 - samples/sec: 7965.63 - lr: 0.025000 2023-04-06 09:24:14,427 epoch 80 - iter 1590/2650 - loss 0.03238240 - time (sec): 110.14 - samples/sec: 7942.77 - lr: 0.025000 2023-04-06 09:24:33,108 epoch 80 - iter 1855/2650 - loss 0.03237858 - time (sec): 128.82 - samples/sec: 7926.13 - lr: 0.025000 2023-04-06 09:24:51,608 epoch 80 - iter 2120/2650 - loss 0.03224237 - time (sec): 147.32 - samples/sec: 7924.30 - lr: 0.025000 2023-04-06 09:25:10,702 epoch 80 - iter 2385/2650 - loss 0.03235970 - time (sec): 166.42 - samples/sec: 7907.10 - lr: 0.025000 2023-04-06 09:25:29,047 epoch 80 - iter 2650/2650 - loss 0.03232011 - time (sec): 184.76 - samples/sec: 7914.81 - lr: 0.025000 2023-04-06 09:25:29,047 ---------------------------------------------------------------------------------------------------- 2023-04-06 09:25:29,047 EPOCH 80 done: loss 0.0323 - lr 0.025000 2023-04-06 09:25:29,047 BAD EPOCHS (no improvement): 0 2023-04-06 09:25:29,051 ---------------------------------------------------------------------------------------------------- 2023-04-06 09:25:47,280 epoch 81 - iter 265/2650 - loss 0.03092679 - time (sec): 18.23 - samples/sec: 8012.03 - lr: 0.025000 2023-04-06 09:26:05,378 epoch 81 - iter 530/2650 - loss 0.03160487 - time (sec): 36.33 - samples/sec: 8088.69 - lr: 0.025000 2023-04-06 09:26:23,598 epoch 81 - iter 795/2650 - loss 0.03161915 - time (sec): 54.55 - samples/sec: 8082.48 - lr: 0.025000 2023-04-06 09:26:41,676 epoch 81 - iter 1060/2650 - loss 0.03179431 - time (sec): 72.62 - samples/sec: 8081.20 - lr: 0.025000 2023-04-06 09:27:00,204 epoch 81 - iter 1325/2650 - loss 0.03178664 - time (sec): 91.15 - samples/sec: 8063.40 - lr: 0.025000 2023-04-06 09:27:18,841 epoch 81 - iter 1590/2650 - loss 0.03158232 - time (sec): 109.79 - samples/sec: 8033.13 - lr: 0.025000 2023-04-06 09:27:37,198 epoch 81 - iter 1855/2650 - loss 0.03174525 - time (sec): 128.15 - samples/sec: 8014.85 - lr: 0.025000 2023-04-06 09:27:55,852 epoch 81 - iter 2120/2650 - loss 0.03169177 - time (sec): 146.80 - samples/sec: 7990.08 - lr: 0.025000 2023-04-06 09:28:14,231 epoch 81 - iter 2385/2650 - loss 0.03183817 - time (sec): 165.18 - samples/sec: 7973.62 - lr: 0.025000 2023-04-06 09:28:32,808 epoch 81 - iter 2650/2650 - loss 0.03168019 - time (sec): 183.76 - samples/sec: 7958.13 - lr: 0.025000 2023-04-06 09:28:32,808 ---------------------------------------------------------------------------------------------------- 2023-04-06 09:28:32,808 EPOCH 81 done: loss 0.0317 - lr 0.025000 2023-04-06 09:28:32,808 BAD EPOCHS (no improvement): 0 2023-04-06 09:28:32,811 ---------------------------------------------------------------------------------------------------- 2023-04-06 09:28:51,578 epoch 82 - iter 265/2650 - loss 0.03154434 - time (sec): 18.77 - samples/sec: 7739.73 - lr: 0.025000 2023-04-06 09:29:09,901 epoch 82 - iter 530/2650 - loss 0.03166951 - time (sec): 37.09 - samples/sec: 7843.02 - lr: 0.025000 2023-04-06 09:29:28,648 epoch 82 - iter 795/2650 - loss 0.03118984 - time (sec): 55.84 - samples/sec: 7866.50 - lr: 0.025000 2023-04-06 09:29:47,133 epoch 82 - iter 1060/2650 - loss 0.03093947 - time (sec): 74.32 - samples/sec: 7874.65 - lr: 0.025000 2023-04-06 09:30:05,307 epoch 82 - iter 1325/2650 - loss 0.03106307 - time (sec): 92.50 - samples/sec: 7885.18 - lr: 0.025000 2023-04-06 09:30:23,999 epoch 82 - iter 1590/2650 - loss 0.03124692 - time (sec): 111.19 - samples/sec: 7870.67 - lr: 0.025000 2023-04-06 09:30:43,238 epoch 82 - iter 1855/2650 - loss 0.03136789 - time (sec): 130.43 - samples/sec: 7853.14 - lr: 0.025000 2023-04-06 09:31:01,484 epoch 82 - iter 2120/2650 - loss 0.03143721 - time (sec): 148.67 - samples/sec: 7845.00 - lr: 0.025000 2023-04-06 09:31:20,520 epoch 82 - iter 2385/2650 - loss 0.03163465 - time (sec): 167.71 - samples/sec: 7850.59 - lr: 0.025000 2023-04-06 09:31:38,701 epoch 82 - iter 2650/2650 - loss 0.03163665 - time (sec): 185.89 - samples/sec: 7866.79 - lr: 0.025000 2023-04-06 09:31:38,702 ---------------------------------------------------------------------------------------------------- 2023-04-06 09:31:38,702 EPOCH 82 done: loss 0.0316 - lr 0.025000 2023-04-06 09:31:38,702 BAD EPOCHS (no improvement): 0 2023-04-06 09:31:38,705 ---------------------------------------------------------------------------------------------------- 2023-04-06 09:31:56,901 epoch 83 - iter 265/2650 - loss 0.03082133 - time (sec): 18.20 - samples/sec: 8070.46 - lr: 0.025000 2023-04-06 09:32:15,197 epoch 83 - iter 530/2650 - loss 0.03184642 - time (sec): 36.49 - samples/sec: 8030.09 - lr: 0.025000 2023-04-06 09:32:33,681 epoch 83 - iter 795/2650 - loss 0.03171315 - time (sec): 54.98 - samples/sec: 8012.28 - lr: 0.025000 2023-04-06 09:32:51,964 epoch 83 - iter 1060/2650 - loss 0.03159024 - time (sec): 73.26 - samples/sec: 8040.34 - lr: 0.025000 2023-04-06 09:33:10,135 epoch 83 - iter 1325/2650 - loss 0.03163975 - time (sec): 91.43 - samples/sec: 8035.06 - lr: 0.025000 2023-04-06 09:33:28,569 epoch 83 - iter 1590/2650 - loss 0.03142994 - time (sec): 109.86 - samples/sec: 8016.87 - lr: 0.025000 2023-04-06 09:33:47,055 epoch 83 - iter 1855/2650 - loss 0.03144163 - time (sec): 128.35 - samples/sec: 8005.47 - lr: 0.025000 2023-04-06 09:34:04,669 epoch 83 - iter 2120/2650 - loss 0.03142107 - time (sec): 145.96 - samples/sec: 8024.73 - lr: 0.025000 2023-04-06 09:34:22,886 epoch 83 - iter 2385/2650 - loss 0.03151275 - time (sec): 164.18 - samples/sec: 8013.49 - lr: 0.025000 2023-04-06 09:34:41,641 epoch 83 - iter 2650/2650 - loss 0.03147094 - time (sec): 182.94 - samples/sec: 7993.85 - lr: 0.025000 2023-04-06 09:34:41,641 ---------------------------------------------------------------------------------------------------- 2023-04-06 09:34:41,641 EPOCH 83 done: loss 0.0315 - lr 0.025000 2023-04-06 09:34:41,641 BAD EPOCHS (no improvement): 0 2023-04-06 09:34:41,644 ---------------------------------------------------------------------------------------------------- 2023-04-06 09:35:00,060 epoch 84 - iter 265/2650 - loss 0.03032183 - time (sec): 18.42 - samples/sec: 7931.04 - lr: 0.025000 2023-04-06 09:35:18,703 epoch 84 - iter 530/2650 - loss 0.03136716 - time (sec): 37.06 - samples/sec: 7899.65 - lr: 0.025000 2023-04-06 09:35:37,174 epoch 84 - iter 795/2650 - loss 0.03128783 - time (sec): 55.53 - samples/sec: 7945.84 - lr: 0.025000 2023-04-06 09:35:55,558 epoch 84 - iter 1060/2650 - loss 0.03123678 - time (sec): 73.91 - samples/sec: 7964.68 - lr: 0.025000 2023-04-06 09:36:14,386 epoch 84 - iter 1325/2650 - loss 0.03122624 - time (sec): 92.74 - samples/sec: 7926.55 - lr: 0.025000 2023-04-06 09:36:32,247 epoch 84 - iter 1590/2650 - loss 0.03104300 - time (sec): 110.60 - samples/sec: 7954.32 - lr: 0.025000 2023-04-06 09:36:50,583 epoch 84 - iter 1855/2650 - loss 0.03107409 - time (sec): 128.94 - samples/sec: 7963.12 - lr: 0.025000 2023-04-06 09:37:08,983 epoch 84 - iter 2120/2650 - loss 0.03117841 - time (sec): 147.34 - samples/sec: 7962.72 - lr: 0.025000 2023-04-06 09:37:27,179 epoch 84 - iter 2385/2650 - loss 0.03117395 - time (sec): 165.54 - samples/sec: 7958.67 - lr: 0.025000 2023-04-06 09:37:45,465 epoch 84 - iter 2650/2650 - loss 0.03111473 - time (sec): 183.82 - samples/sec: 7955.35 - lr: 0.025000 2023-04-06 09:37:45,465 ---------------------------------------------------------------------------------------------------- 2023-04-06 09:37:45,465 EPOCH 84 done: loss 0.0311 - lr 0.025000 2023-04-06 09:37:45,465 BAD EPOCHS (no improvement): 0 2023-04-06 09:37:45,468 ---------------------------------------------------------------------------------------------------- 2023-04-06 09:38:04,353 epoch 85 - iter 265/2650 - loss 0.03018813 - time (sec): 18.88 - samples/sec: 7844.65 - lr: 0.025000 2023-04-06 09:38:22,388 epoch 85 - iter 530/2650 - loss 0.03047344 - time (sec): 36.92 - samples/sec: 7988.52 - lr: 0.025000 2023-04-06 09:38:40,509 epoch 85 - iter 795/2650 - loss 0.03089773 - time (sec): 55.04 - samples/sec: 7998.20 - lr: 0.025000 2023-04-06 09:38:58,892 epoch 85 - iter 1060/2650 - loss 0.03080456 - time (sec): 73.42 - samples/sec: 7990.87 - lr: 0.025000 2023-04-06 09:39:17,209 epoch 85 - iter 1325/2650 - loss 0.03094171 - time (sec): 91.74 - samples/sec: 7974.39 - lr: 0.025000 2023-04-06 09:39:35,881 epoch 85 - iter 1590/2650 - loss 0.03090591 - time (sec): 110.41 - samples/sec: 7956.40 - lr: 0.025000 2023-04-06 09:39:54,724 epoch 85 - iter 1855/2650 - loss 0.03100766 - time (sec): 129.26 - samples/sec: 7935.30 - lr: 0.025000 2023-04-06 09:40:12,952 epoch 85 - iter 2120/2650 - loss 0.03112836 - time (sec): 147.48 - samples/sec: 7947.44 - lr: 0.025000 2023-04-06 09:40:31,066 epoch 85 - iter 2385/2650 - loss 0.03119593 - time (sec): 165.60 - samples/sec: 7955.34 - lr: 0.025000 2023-04-06 09:40:49,165 epoch 85 - iter 2650/2650 - loss 0.03130380 - time (sec): 183.70 - samples/sec: 7960.71 - lr: 0.025000 2023-04-06 09:40:49,166 ---------------------------------------------------------------------------------------------------- 2023-04-06 09:40:49,166 EPOCH 85 done: loss 0.0313 - lr 0.025000 2023-04-06 09:40:49,166 BAD EPOCHS (no improvement): 1 2023-04-06 09:40:49,169 ---------------------------------------------------------------------------------------------------- 2023-04-06 09:41:07,113 epoch 86 - iter 265/2650 - loss 0.03069412 - time (sec): 17.94 - samples/sec: 8062.32 - lr: 0.025000 2023-04-06 09:41:25,573 epoch 86 - iter 530/2650 - loss 0.03035511 - time (sec): 36.40 - samples/sec: 8012.60 - lr: 0.025000 2023-04-06 09:41:43,728 epoch 86 - iter 795/2650 - loss 0.03044105 - time (sec): 54.56 - samples/sec: 8025.08 - lr: 0.025000 2023-04-06 09:42:02,134 epoch 86 - iter 1060/2650 - loss 0.03041735 - time (sec): 72.97 - samples/sec: 8009.60 - lr: 0.025000 2023-04-06 09:42:20,336 epoch 86 - iter 1325/2650 - loss 0.03070927 - time (sec): 91.17 - samples/sec: 7999.27 - lr: 0.025000 2023-04-06 09:42:38,860 epoch 86 - iter 1590/2650 - loss 0.03062484 - time (sec): 109.69 - samples/sec: 7994.11 - lr: 0.025000 2023-04-06 09:42:56,806 epoch 86 - iter 1855/2650 - loss 0.03059316 - time (sec): 127.64 - samples/sec: 8011.38 - lr: 0.025000 2023-04-06 09:43:15,111 epoch 86 - iter 2120/2650 - loss 0.03073718 - time (sec): 145.94 - samples/sec: 8016.34 - lr: 0.025000 2023-04-06 09:43:33,146 epoch 86 - iter 2385/2650 - loss 0.03092875 - time (sec): 163.98 - samples/sec: 8026.55 - lr: 0.025000 2023-04-06 09:43:51,828 epoch 86 - iter 2650/2650 - loss 0.03099146 - time (sec): 182.66 - samples/sec: 8005.94 - lr: 0.025000 2023-04-06 09:43:51,829 ---------------------------------------------------------------------------------------------------- 2023-04-06 09:43:51,829 EPOCH 86 done: loss 0.0310 - lr 0.025000 2023-04-06 09:43:51,829 BAD EPOCHS (no improvement): 0 2023-04-06 09:43:51,832 ---------------------------------------------------------------------------------------------------- 2023-04-06 09:44:10,378 epoch 87 - iter 265/2650 - loss 0.03203710 - time (sec): 18.55 - samples/sec: 7828.45 - lr: 0.025000 2023-04-06 09:44:29,226 epoch 87 - iter 530/2650 - loss 0.03149645 - time (sec): 37.39 - samples/sec: 7817.08 - lr: 0.025000 2023-04-06 09:44:47,819 epoch 87 - iter 795/2650 - loss 0.03087534 - time (sec): 55.99 - samples/sec: 7836.83 - lr: 0.025000 2023-04-06 09:45:06,463 epoch 87 - iter 1060/2650 - loss 0.03114811 - time (sec): 74.63 - samples/sec: 7865.44 - lr: 0.025000 2023-04-06 09:45:24,345 epoch 87 - iter 1325/2650 - loss 0.03122279 - time (sec): 92.51 - samples/sec: 7898.80 - lr: 0.025000 2023-04-06 09:45:42,345 epoch 87 - iter 1590/2650 - loss 0.03132279 - time (sec): 110.51 - samples/sec: 7922.61 - lr: 0.025000 2023-04-06 09:46:00,763 epoch 87 - iter 1855/2650 - loss 0.03132212 - time (sec): 128.93 - samples/sec: 7932.51 - lr: 0.025000 2023-04-06 09:46:19,543 epoch 87 - iter 2120/2650 - loss 0.03117344 - time (sec): 147.71 - samples/sec: 7927.24 - lr: 0.025000 2023-04-06 09:46:37,947 epoch 87 - iter 2385/2650 - loss 0.03109555 - time (sec): 166.12 - samples/sec: 7920.89 - lr: 0.025000 2023-04-06 09:46:56,430 epoch 87 - iter 2650/2650 - loss 0.03109712 - time (sec): 184.60 - samples/sec: 7921.89 - lr: 0.025000 2023-04-06 09:46:56,430 ---------------------------------------------------------------------------------------------------- 2023-04-06 09:46:56,430 EPOCH 87 done: loss 0.0311 - lr 0.025000 2023-04-06 09:46:56,430 BAD EPOCHS (no improvement): 1 2023-04-06 09:46:56,433 ---------------------------------------------------------------------------------------------------- 2023-04-06 09:47:14,270 epoch 88 - iter 265/2650 - loss 0.03025468 - time (sec): 17.84 - samples/sec: 8079.90 - lr: 0.025000 2023-04-06 09:47:32,646 epoch 88 - iter 530/2650 - loss 0.03041313 - time (sec): 36.21 - samples/sec: 8043.12 - lr: 0.025000 2023-04-06 09:47:51,344 epoch 88 - iter 795/2650 - loss 0.03060392 - time (sec): 54.91 - samples/sec: 7959.15 - lr: 0.025000 2023-04-06 09:48:10,211 epoch 88 - iter 1060/2650 - loss 0.03095930 - time (sec): 73.78 - samples/sec: 7905.49 - lr: 0.025000 2023-04-06 09:48:28,419 epoch 88 - iter 1325/2650 - loss 0.03080838 - time (sec): 91.99 - samples/sec: 7943.99 - lr: 0.025000 2023-04-06 09:48:46,683 epoch 88 - iter 1590/2650 - loss 0.03076990 - time (sec): 110.25 - samples/sec: 7940.11 - lr: 0.025000 2023-04-06 09:49:04,960 epoch 88 - iter 1855/2650 - loss 0.03068172 - time (sec): 128.53 - samples/sec: 7950.41 - lr: 0.025000 2023-04-06 09:49:23,237 epoch 88 - iter 2120/2650 - loss 0.03069846 - time (sec): 146.80 - samples/sec: 7955.06 - lr: 0.025000 2023-04-06 09:49:41,379 epoch 88 - iter 2385/2650 - loss 0.03069917 - time (sec): 164.95 - samples/sec: 7973.33 - lr: 0.025000 2023-04-06 09:50:00,347 epoch 88 - iter 2650/2650 - loss 0.03076820 - time (sec): 183.91 - samples/sec: 7951.34 - lr: 0.025000 2023-04-06 09:50:00,347 ---------------------------------------------------------------------------------------------------- 2023-04-06 09:50:00,347 EPOCH 88 done: loss 0.0308 - lr 0.025000 2023-04-06 09:50:00,348 BAD EPOCHS (no improvement): 0 2023-04-06 09:50:00,350 ---------------------------------------------------------------------------------------------------- 2023-04-06 09:50:18,716 epoch 89 - iter 265/2650 - loss 0.03060707 - time (sec): 18.37 - samples/sec: 7897.30 - lr: 0.025000 2023-04-06 09:50:36,816 epoch 89 - iter 530/2650 - loss 0.03026629 - time (sec): 36.47 - samples/sec: 8017.26 - lr: 0.025000 2023-04-06 09:50:55,287 epoch 89 - iter 795/2650 - loss 0.03048645 - time (sec): 54.94 - samples/sec: 8002.36 - lr: 0.025000 2023-04-06 09:51:13,490 epoch 89 - iter 1060/2650 - loss 0.03042853 - time (sec): 73.14 - samples/sec: 8016.25 - lr: 0.025000 2023-04-06 09:51:31,422 epoch 89 - iter 1325/2650 - loss 0.03034908 - time (sec): 91.07 - samples/sec: 8037.67 - lr: 0.025000 2023-04-06 09:51:49,206 epoch 89 - iter 1590/2650 - loss 0.03016823 - time (sec): 108.86 - samples/sec: 8053.48 - lr: 0.025000 2023-04-06 09:52:07,773 epoch 89 - iter 1855/2650 - loss 0.03026907 - time (sec): 127.42 - samples/sec: 8044.02 - lr: 0.025000 2023-04-06 09:52:26,137 epoch 89 - iter 2120/2650 - loss 0.03031615 - time (sec): 145.79 - samples/sec: 8041.58 - lr: 0.025000 2023-04-06 09:52:44,268 epoch 89 - iter 2385/2650 - loss 0.03039435 - time (sec): 163.92 - samples/sec: 8035.62 - lr: 0.025000 2023-04-06 09:53:02,636 epoch 89 - iter 2650/2650 - loss 0.03048947 - time (sec): 182.29 - samples/sec: 8022.34 - lr: 0.025000 2023-04-06 09:53:02,637 ---------------------------------------------------------------------------------------------------- 2023-04-06 09:53:02,637 EPOCH 89 done: loss 0.0305 - lr 0.025000 2023-04-06 09:53:02,637 BAD EPOCHS (no improvement): 0 2023-04-06 09:53:02,641 ---------------------------------------------------------------------------------------------------- 2023-04-06 09:53:21,016 epoch 90 - iter 265/2650 - loss 0.02999573 - time (sec): 18.37 - samples/sec: 7931.17 - lr: 0.025000 2023-04-06 09:53:39,115 epoch 90 - iter 530/2650 - loss 0.02971504 - time (sec): 36.47 - samples/sec: 8008.21 - lr: 0.025000 2023-04-06 09:53:57,282 epoch 90 - iter 795/2650 - loss 0.02995024 - time (sec): 54.64 - samples/sec: 8019.35 - lr: 0.025000 2023-04-06 09:54:15,367 epoch 90 - iter 1060/2650 - loss 0.02994512 - time (sec): 72.73 - samples/sec: 8021.31 - lr: 0.025000 2023-04-06 09:54:33,980 epoch 90 - iter 1325/2650 - loss 0.02990897 - time (sec): 91.34 - samples/sec: 8008.57 - lr: 0.025000 2023-04-06 09:54:52,298 epoch 90 - iter 1590/2650 - loss 0.02992481 - time (sec): 109.66 - samples/sec: 8015.63 - lr: 0.025000 2023-04-06 09:55:10,450 epoch 90 - iter 1855/2650 - loss 0.03013628 - time (sec): 127.81 - samples/sec: 8020.79 - lr: 0.025000 2023-04-06 09:55:28,799 epoch 90 - iter 2120/2650 - loss 0.03016773 - time (sec): 146.16 - samples/sec: 8021.63 - lr: 0.025000 2023-04-06 09:55:46,759 epoch 90 - iter 2385/2650 - loss 0.03019036 - time (sec): 164.12 - samples/sec: 8035.35 - lr: 0.025000 2023-04-06 09:56:04,798 epoch 90 - iter 2650/2650 - loss 0.03030086 - time (sec): 182.16 - samples/sec: 8028.05 - lr: 0.025000 2023-04-06 09:56:04,798 ---------------------------------------------------------------------------------------------------- 2023-04-06 09:56:04,798 EPOCH 90 done: loss 0.0303 - lr 0.025000 2023-04-06 09:56:04,798 BAD EPOCHS (no improvement): 0 2023-04-06 09:56:04,801 ---------------------------------------------------------------------------------------------------- 2023-04-06 09:56:22,679 epoch 91 - iter 265/2650 - loss 0.02983226 - time (sec): 17.88 - samples/sec: 8146.59 - lr: 0.025000 2023-04-06 09:56:40,783 epoch 91 - iter 530/2650 - loss 0.03057380 - time (sec): 35.98 - samples/sec: 8106.28 - lr: 0.025000 2023-04-06 09:56:59,018 epoch 91 - iter 795/2650 - loss 0.03051211 - time (sec): 54.22 - samples/sec: 8068.80 - lr: 0.025000 2023-04-06 09:57:17,302 epoch 91 - iter 1060/2650 - loss 0.03053699 - time (sec): 72.50 - samples/sec: 8072.93 - lr: 0.025000 2023-04-06 09:57:35,470 epoch 91 - iter 1325/2650 - loss 0.03037614 - time (sec): 90.67 - samples/sec: 8073.76 - lr: 0.025000 2023-04-06 09:57:54,081 epoch 91 - iter 1590/2650 - loss 0.03031043 - time (sec): 109.28 - samples/sec: 8041.89 - lr: 0.025000 2023-04-06 09:58:12,620 epoch 91 - iter 1855/2650 - loss 0.03013952 - time (sec): 127.82 - samples/sec: 8021.87 - lr: 0.025000 2023-04-06 09:58:31,235 epoch 91 - iter 2120/2650 - loss 0.03016849 - time (sec): 146.43 - samples/sec: 8000.75 - lr: 0.025000 2023-04-06 09:59:01,159 epoch 91 - iter 2385/2650 - loss 0.03021219 - time (sec): 176.36 - samples/sec: 7468.65 - lr: 0.025000 2023-04-06 09:59:19,433 epoch 91 - iter 2650/2650 - loss 0.03016063 - time (sec): 194.63 - samples/sec: 7513.48 - lr: 0.025000 2023-04-06 09:59:19,433 ---------------------------------------------------------------------------------------------------- 2023-04-06 09:59:19,433 EPOCH 91 done: loss 0.0302 - lr 0.025000 2023-04-06 09:59:19,433 BAD EPOCHS (no improvement): 0 2023-04-06 09:59:19,437 ---------------------------------------------------------------------------------------------------- 2023-04-06 09:59:37,516 epoch 92 - iter 265/2650 - loss 0.02954097 - time (sec): 18.08 - samples/sec: 7988.23 - lr: 0.025000 2023-04-06 09:59:56,006 epoch 92 - iter 530/2650 - loss 0.02983538 - time (sec): 36.57 - samples/sec: 7984.00 - lr: 0.025000 2023-04-06 10:00:14,666 epoch 92 - iter 795/2650 - loss 0.02984469 - time (sec): 55.23 - samples/sec: 7941.00 - lr: 0.025000 2023-04-06 10:00:32,950 epoch 92 - iter 1060/2650 - loss 0.02976933 - time (sec): 73.51 - samples/sec: 7942.27 - lr: 0.025000 2023-04-06 10:00:50,902 epoch 92 - iter 1325/2650 - loss 0.02977028 - time (sec): 91.47 - samples/sec: 7964.68 - lr: 0.025000 2023-04-06 10:01:09,058 epoch 92 - iter 1590/2650 - loss 0.03002720 - time (sec): 109.62 - samples/sec: 7981.32 - lr: 0.025000 2023-04-06 10:01:27,553 epoch 92 - iter 1855/2650 - loss 0.03027696 - time (sec): 128.12 - samples/sec: 7983.95 - lr: 0.025000 2023-04-06 10:01:45,825 epoch 92 - iter 2120/2650 - loss 0.03023003 - time (sec): 146.39 - samples/sec: 7988.38 - lr: 0.025000 2023-04-06 10:02:04,069 epoch 92 - iter 2385/2650 - loss 0.03023304 - time (sec): 164.63 - samples/sec: 7991.31 - lr: 0.025000 2023-04-06 10:02:22,647 epoch 92 - iter 2650/2650 - loss 0.03020003 - time (sec): 183.21 - samples/sec: 7981.87 - lr: 0.025000 2023-04-06 10:02:22,648 ---------------------------------------------------------------------------------------------------- 2023-04-06 10:02:22,648 EPOCH 92 done: loss 0.0302 - lr 0.025000 2023-04-06 10:02:22,648 BAD EPOCHS (no improvement): 1 2023-04-06 10:02:22,653 ---------------------------------------------------------------------------------------------------- 2023-04-06 10:02:40,972 epoch 93 - iter 265/2650 - loss 0.02895471 - time (sec): 18.32 - samples/sec: 7998.13 - lr: 0.025000 2023-04-06 10:02:59,060 epoch 93 - iter 530/2650 - loss 0.02924961 - time (sec): 36.41 - samples/sec: 8024.49 - lr: 0.025000 2023-04-06 10:03:18,090 epoch 93 - iter 795/2650 - loss 0.02893645 - time (sec): 55.44 - samples/sec: 7934.94 - lr: 0.025000 2023-04-06 10:03:36,755 epoch 93 - iter 1060/2650 - loss 0.02904692 - time (sec): 74.10 - samples/sec: 7916.46 - lr: 0.025000 2023-04-06 10:03:55,208 epoch 93 - iter 1325/2650 - loss 0.02914444 - time (sec): 92.55 - samples/sec: 7929.22 - lr: 0.025000 2023-04-06 10:04:13,178 epoch 93 - iter 1590/2650 - loss 0.02930546 - time (sec): 110.52 - samples/sec: 7944.34 - lr: 0.025000 2023-04-06 10:04:31,325 epoch 93 - iter 1855/2650 - loss 0.02944732 - time (sec): 128.67 - samples/sec: 7946.61 - lr: 0.025000 2023-04-06 10:04:49,397 epoch 93 - iter 2120/2650 - loss 0.02948859 - time (sec): 146.74 - samples/sec: 7955.52 - lr: 0.025000 2023-04-06 10:05:07,870 epoch 93 - iter 2385/2650 - loss 0.02970329 - time (sec): 165.22 - samples/sec: 7963.62 - lr: 0.025000 2023-04-06 10:05:26,208 epoch 93 - iter 2650/2650 - loss 0.02976326 - time (sec): 183.56 - samples/sec: 7966.89 - lr: 0.025000 2023-04-06 10:05:26,209 ---------------------------------------------------------------------------------------------------- 2023-04-06 10:05:26,209 EPOCH 93 done: loss 0.0298 - lr 0.025000 2023-04-06 10:05:26,209 BAD EPOCHS (no improvement): 0 2023-04-06 10:05:26,217 ---------------------------------------------------------------------------------------------------- 2023-04-06 10:05:44,547 epoch 94 - iter 265/2650 - loss 0.02992291 - time (sec): 18.33 - samples/sec: 8050.79 - lr: 0.025000 2023-04-06 10:06:02,671 epoch 94 - iter 530/2650 - loss 0.02953093 - time (sec): 36.45 - samples/sec: 8069.44 - lr: 0.025000 2023-04-06 10:06:21,549 epoch 94 - iter 795/2650 - loss 0.02943924 - time (sec): 55.33 - samples/sec: 7967.73 - lr: 0.025000 2023-04-06 10:06:40,001 epoch 94 - iter 1060/2650 - loss 0.02923074 - time (sec): 73.78 - samples/sec: 7930.50 - lr: 0.025000 2023-04-06 10:06:58,870 epoch 94 - iter 1325/2650 - loss 0.02940616 - time (sec): 92.65 - samples/sec: 7921.60 - lr: 0.025000 2023-04-06 10:07:17,795 epoch 94 - iter 1590/2650 - loss 0.02951783 - time (sec): 111.58 - samples/sec: 7895.85 - lr: 0.025000 2023-04-06 10:07:36,282 epoch 94 - iter 1855/2650 - loss 0.02958460 - time (sec): 130.06 - samples/sec: 7908.48 - lr: 0.025000 2023-04-06 10:07:54,657 epoch 94 - iter 2120/2650 - loss 0.02964827 - time (sec): 148.44 - samples/sec: 7906.00 - lr: 0.025000 2023-04-06 10:08:12,880 epoch 94 - iter 2385/2650 - loss 0.02976769 - time (sec): 166.66 - samples/sec: 7911.16 - lr: 0.025000 2023-04-06 10:08:30,958 epoch 94 - iter 2650/2650 - loss 0.02971315 - time (sec): 184.74 - samples/sec: 7915.73 - lr: 0.025000 2023-04-06 10:08:30,958 ---------------------------------------------------------------------------------------------------- 2023-04-06 10:08:30,958 EPOCH 94 done: loss 0.0297 - lr 0.025000 2023-04-06 10:08:30,958 BAD EPOCHS (no improvement): 0 2023-04-06 10:08:30,962 ---------------------------------------------------------------------------------------------------- 2023-04-06 10:08:49,074 epoch 95 - iter 265/2650 - loss 0.03093760 - time (sec): 18.11 - samples/sec: 8022.81 - lr: 0.025000 2023-04-06 10:09:07,700 epoch 95 - iter 530/2650 - loss 0.03092172 - time (sec): 36.74 - samples/sec: 7974.79 - lr: 0.025000 2023-04-06 10:09:25,510 epoch 95 - iter 795/2650 - loss 0.03058677 - time (sec): 54.55 - samples/sec: 8033.07 - lr: 0.025000 2023-04-06 10:09:43,656 epoch 95 - iter 1060/2650 - loss 0.02970582 - time (sec): 72.69 - samples/sec: 8045.59 - lr: 0.025000 2023-04-06 10:10:02,711 epoch 95 - iter 1325/2650 - loss 0.02966750 - time (sec): 91.75 - samples/sec: 7986.52 - lr: 0.025000 2023-04-06 10:10:21,481 epoch 95 - iter 1590/2650 - loss 0.02936030 - time (sec): 110.52 - samples/sec: 7949.69 - lr: 0.025000 2023-04-06 10:10:40,272 epoch 95 - iter 1855/2650 - loss 0.02941326 - time (sec): 129.31 - samples/sec: 7923.26 - lr: 0.025000 2023-04-06 10:10:58,461 epoch 95 - iter 2120/2650 - loss 0.02959746 - time (sec): 147.50 - samples/sec: 7925.83 - lr: 0.025000 2023-04-06 10:11:16,588 epoch 95 - iter 2385/2650 - loss 0.02956152 - time (sec): 165.63 - samples/sec: 7941.11 - lr: 0.025000 2023-04-06 10:11:34,914 epoch 95 - iter 2650/2650 - loss 0.02964425 - time (sec): 183.95 - samples/sec: 7949.70 - lr: 0.025000 2023-04-06 10:11:34,914 ---------------------------------------------------------------------------------------------------- 2023-04-06 10:11:34,914 EPOCH 95 done: loss 0.0296 - lr 0.025000 2023-04-06 10:11:34,914 BAD EPOCHS (no improvement): 0 2023-04-06 10:11:34,918 ---------------------------------------------------------------------------------------------------- 2023-04-06 10:11:52,984 epoch 96 - iter 265/2650 - loss 0.02948807 - time (sec): 18.07 - samples/sec: 8111.15 - lr: 0.025000 2023-04-06 10:12:10,903 epoch 96 - iter 530/2650 - loss 0.02904995 - time (sec): 35.98 - samples/sec: 8099.26 - lr: 0.025000 2023-04-06 10:12:28,743 epoch 96 - iter 795/2650 - loss 0.02902849 - time (sec): 53.82 - samples/sec: 8120.20 - lr: 0.025000 2023-04-06 10:12:47,256 epoch 96 - iter 1060/2650 - loss 0.02922276 - time (sec): 72.34 - samples/sec: 8086.09 - lr: 0.025000 2023-04-06 10:13:05,361 epoch 96 - iter 1325/2650 - loss 0.02953263 - time (sec): 90.44 - samples/sec: 8066.64 - lr: 0.025000 2023-04-06 10:13:23,618 epoch 96 - iter 1590/2650 - loss 0.02930345 - time (sec): 108.70 - samples/sec: 8067.59 - lr: 0.025000 2023-04-06 10:13:41,713 epoch 96 - iter 1855/2650 - loss 0.02939696 - time (sec): 126.80 - samples/sec: 8076.08 - lr: 0.025000 2023-04-06 10:14:00,129 epoch 96 - iter 2120/2650 - loss 0.02945321 - time (sec): 145.21 - samples/sec: 8058.97 - lr: 0.025000 2023-04-06 10:14:18,495 epoch 96 - iter 2385/2650 - loss 0.02951025 - time (sec): 163.58 - samples/sec: 8051.70 - lr: 0.025000 2023-04-06 10:14:36,470 epoch 96 - iter 2650/2650 - loss 0.02955779 - time (sec): 181.55 - samples/sec: 8054.78 - lr: 0.025000 2023-04-06 10:14:36,470 ---------------------------------------------------------------------------------------------------- 2023-04-06 10:14:36,470 EPOCH 96 done: loss 0.0296 - lr 0.025000 2023-04-06 10:14:36,470 BAD EPOCHS (no improvement): 0 2023-04-06 10:14:36,474 ---------------------------------------------------------------------------------------------------- 2023-04-06 10:14:54,069 epoch 97 - iter 265/2650 - loss 0.02786645 - time (sec): 17.59 - samples/sec: 8174.84 - lr: 0.025000 2023-04-06 10:15:12,072 epoch 97 - iter 530/2650 - loss 0.02878904 - time (sec): 35.60 - samples/sec: 8168.21 - lr: 0.025000 2023-04-06 10:15:30,478 epoch 97 - iter 795/2650 - loss 0.02853451 - time (sec): 54.00 - samples/sec: 8127.11 - lr: 0.025000 2023-04-06 10:15:48,594 epoch 97 - iter 1060/2650 - loss 0.02881343 - time (sec): 72.12 - samples/sec: 8114.09 - lr: 0.025000 2023-04-06 10:16:07,234 epoch 97 - iter 1325/2650 - loss 0.02905169 - time (sec): 90.76 - samples/sec: 8076.09 - lr: 0.025000 2023-04-06 10:16:24,867 epoch 97 - iter 1590/2650 - loss 0.02897781 - time (sec): 108.39 - samples/sec: 8098.38 - lr: 0.025000 2023-04-06 10:16:42,956 epoch 97 - iter 1855/2650 - loss 0.02913988 - time (sec): 126.48 - samples/sec: 8096.68 - lr: 0.025000 2023-04-06 10:17:01,629 epoch 97 - iter 2120/2650 - loss 0.02927943 - time (sec): 145.16 - samples/sec: 8076.88 - lr: 0.025000 2023-04-06 10:17:20,111 epoch 97 - iter 2385/2650 - loss 0.02929977 - time (sec): 163.64 - samples/sec: 8056.10 - lr: 0.025000 2023-04-06 10:17:38,033 epoch 97 - iter 2650/2650 - loss 0.02944678 - time (sec): 181.56 - samples/sec: 8054.49 - lr: 0.025000 2023-04-06 10:17:38,033 ---------------------------------------------------------------------------------------------------- 2023-04-06 10:17:38,033 EPOCH 97 done: loss 0.0294 - lr 0.025000 2023-04-06 10:17:38,033 BAD EPOCHS (no improvement): 0 2023-04-06 10:17:38,037 ---------------------------------------------------------------------------------------------------- 2023-04-06 10:17:56,162 epoch 98 - iter 265/2650 - loss 0.03015602 - time (sec): 18.12 - samples/sec: 8105.35 - lr: 0.025000 2023-04-06 10:18:14,442 epoch 98 - iter 530/2650 - loss 0.02934542 - time (sec): 36.40 - samples/sec: 8049.26 - lr: 0.025000 2023-04-06 10:18:32,548 epoch 98 - iter 795/2650 - loss 0.02949456 - time (sec): 54.51 - samples/sec: 8033.02 - lr: 0.025000 2023-04-06 10:18:50,376 epoch 98 - iter 1060/2650 - loss 0.02923343 - time (sec): 72.34 - samples/sec: 8050.49 - lr: 0.025000 2023-04-06 10:19:08,369 epoch 98 - iter 1325/2650 - loss 0.02940113 - time (sec): 90.33 - samples/sec: 8053.20 - lr: 0.025000 2023-04-06 10:19:26,916 epoch 98 - iter 1590/2650 - loss 0.02931146 - time (sec): 108.88 - samples/sec: 8037.89 - lr: 0.025000 2023-04-06 10:19:45,195 epoch 98 - iter 1855/2650 - loss 0.02933681 - time (sec): 127.16 - samples/sec: 8041.35 - lr: 0.025000 2023-04-06 10:20:03,253 epoch 98 - iter 2120/2650 - loss 0.02940190 - time (sec): 145.22 - samples/sec: 8039.23 - lr: 0.025000 2023-04-06 10:20:21,390 epoch 98 - iter 2385/2650 - loss 0.02941209 - time (sec): 163.35 - samples/sec: 8046.18 - lr: 0.025000 2023-04-06 10:20:40,109 epoch 98 - iter 2650/2650 - loss 0.02947156 - time (sec): 182.07 - samples/sec: 8031.80 - lr: 0.025000 2023-04-06 10:20:40,109 ---------------------------------------------------------------------------------------------------- 2023-04-06 10:20:40,109 EPOCH 98 done: loss 0.0295 - lr 0.025000 2023-04-06 10:20:40,109 BAD EPOCHS (no improvement): 1 2023-04-06 10:20:40,113 ---------------------------------------------------------------------------------------------------- 2023-04-06 10:20:58,417 epoch 99 - iter 265/2650 - loss 0.02837117 - time (sec): 18.30 - samples/sec: 7841.33 - lr: 0.025000 2023-04-06 10:21:16,432 epoch 99 - iter 530/2650 - loss 0.02837371 - time (sec): 36.32 - samples/sec: 7943.71 - lr: 0.025000 2023-04-06 10:21:34,762 epoch 99 - iter 795/2650 - loss 0.02876587 - time (sec): 54.65 - samples/sec: 7954.97 - lr: 0.025000 2023-04-06 10:21:52,445 epoch 99 - iter 1060/2650 - loss 0.02883635 - time (sec): 72.33 - samples/sec: 8004.57 - lr: 0.025000 2023-04-06 10:22:11,170 epoch 99 - iter 1325/2650 - loss 0.02864805 - time (sec): 91.06 - samples/sec: 7980.49 - lr: 0.025000 2023-04-06 10:22:29,189 epoch 99 - iter 1590/2650 - loss 0.02872146 - time (sec): 109.08 - samples/sec: 8015.25 - lr: 0.025000 2023-04-06 10:22:47,379 epoch 99 - iter 1855/2650 - loss 0.02896624 - time (sec): 127.27 - samples/sec: 8012.85 - lr: 0.025000 2023-04-06 10:23:05,617 epoch 99 - iter 2120/2650 - loss 0.02897323 - time (sec): 145.50 - samples/sec: 8029.91 - lr: 0.025000 2023-04-06 10:23:23,765 epoch 99 - iter 2385/2650 - loss 0.02896029 - time (sec): 163.65 - samples/sec: 8036.46 - lr: 0.025000 2023-04-06 10:23:42,273 epoch 99 - iter 2650/2650 - loss 0.02897519 - time (sec): 182.16 - samples/sec: 8027.91 - lr: 0.025000 2023-04-06 10:23:42,273 ---------------------------------------------------------------------------------------------------- 2023-04-06 10:23:42,273 EPOCH 99 done: loss 0.0290 - lr 0.025000 2023-04-06 10:23:42,273 BAD EPOCHS (no improvement): 0 2023-04-06 10:23:42,277 ---------------------------------------------------------------------------------------------------- 2023-04-06 10:24:00,706 epoch 100 - iter 265/2650 - loss 0.02990085 - time (sec): 18.43 - samples/sec: 7988.61 - lr: 0.025000 2023-04-06 10:24:18,864 epoch 100 - iter 530/2650 - loss 0.02944842 - time (sec): 36.59 - samples/sec: 7984.25 - lr: 0.025000 2023-04-06 10:24:36,969 epoch 100 - iter 795/2650 - loss 0.02936896 - time (sec): 54.69 - samples/sec: 7999.33 - lr: 0.025000 2023-04-06 10:24:55,130 epoch 100 - iter 1060/2650 - loss 0.02917718 - time (sec): 72.85 - samples/sec: 8033.84 - lr: 0.025000 2023-04-06 10:25:12,861 epoch 100 - iter 1325/2650 - loss 0.02929608 - time (sec): 90.58 - samples/sec: 8055.79 - lr: 0.025000 2023-04-06 10:25:31,174 epoch 100 - iter 1590/2650 - loss 0.02903663 - time (sec): 108.90 - samples/sec: 8055.41 - lr: 0.025000 2023-04-06 10:25:49,564 epoch 100 - iter 1855/2650 - loss 0.02917733 - time (sec): 127.29 - samples/sec: 8042.48 - lr: 0.025000 2023-04-06 10:26:08,241 epoch 100 - iter 2120/2650 - loss 0.02923354 - time (sec): 145.96 - samples/sec: 8014.79 - lr: 0.025000 2023-04-06 10:26:27,336 epoch 100 - iter 2385/2650 - loss 0.02931082 - time (sec): 165.06 - samples/sec: 7972.54 - lr: 0.025000 2023-04-06 10:26:45,605 epoch 100 - iter 2650/2650 - loss 0.02923743 - time (sec): 183.33 - samples/sec: 7976.73 - lr: 0.025000 2023-04-06 10:26:45,605 ---------------------------------------------------------------------------------------------------- 2023-04-06 10:26:45,605 EPOCH 100 done: loss 0.0292 - lr 0.025000 2023-04-06 10:26:45,606 BAD EPOCHS (no improvement): 1 2023-04-06 10:26:45,612 ---------------------------------------------------------------------------------------------------- 2023-04-06 10:27:04,367 epoch 101 - iter 265/2650 - loss 0.02789438 - time (sec): 18.75 - samples/sec: 7747.40 - lr: 0.025000 2023-04-06 10:27:22,223 epoch 101 - iter 530/2650 - loss 0.02839336 - time (sec): 36.61 - samples/sec: 7934.40 - lr: 0.025000 2023-04-06 10:27:40,866 epoch 101 - iter 795/2650 - loss 0.02904500 - time (sec): 55.25 - samples/sec: 7933.18 - lr: 0.025000 2023-04-06 10:27:59,281 epoch 101 - iter 1060/2650 - loss 0.02913840 - time (sec): 73.67 - samples/sec: 7938.16 - lr: 0.025000 2023-04-06 10:28:17,467 epoch 101 - iter 1325/2650 - loss 0.02908760 - time (sec): 91.85 - samples/sec: 7949.14 - lr: 0.025000 2023-04-06 10:28:36,285 epoch 101 - iter 1590/2650 - loss 0.02914142 - time (sec): 110.67 - samples/sec: 7921.74 - lr: 0.025000 2023-04-06 10:28:54,458 epoch 101 - iter 1855/2650 - loss 0.02912699 - time (sec): 128.85 - samples/sec: 7940.91 - lr: 0.025000 2023-04-06 10:29:12,756 epoch 101 - iter 2120/2650 - loss 0.02918596 - time (sec): 147.14 - samples/sec: 7937.93 - lr: 0.025000 2023-04-06 10:29:31,592 epoch 101 - iter 2385/2650 - loss 0.02921213 - time (sec): 165.98 - samples/sec: 7930.64 - lr: 0.025000 2023-04-06 10:29:49,818 epoch 101 - iter 2650/2650 - loss 0.02921635 - time (sec): 184.21 - samples/sec: 7938.76 - lr: 0.025000 2023-04-06 10:29:49,818 ---------------------------------------------------------------------------------------------------- 2023-04-06 10:29:49,818 EPOCH 101 done: loss 0.0292 - lr 0.025000 2023-04-06 10:29:49,818 BAD EPOCHS (no improvement): 2 2023-04-06 10:29:49,821 ---------------------------------------------------------------------------------------------------- 2023-04-06 10:30:07,732 epoch 102 - iter 265/2650 - loss 0.02892950 - time (sec): 17.91 - samples/sec: 8112.17 - lr: 0.025000 2023-04-06 10:30:25,635 epoch 102 - iter 530/2650 - loss 0.02917550 - time (sec): 35.81 - samples/sec: 8110.43 - lr: 0.025000 2023-04-06 10:30:44,274 epoch 102 - iter 795/2650 - loss 0.02924241 - time (sec): 54.45 - samples/sec: 8046.97 - lr: 0.025000 2023-04-06 10:31:02,807 epoch 102 - iter 1060/2650 - loss 0.02909254 - time (sec): 72.99 - samples/sec: 8021.84 - lr: 0.025000 2023-04-06 10:31:21,652 epoch 102 - iter 1325/2650 - loss 0.02880245 - time (sec): 91.83 - samples/sec: 7964.19 - lr: 0.025000 2023-04-06 10:31:40,528 epoch 102 - iter 1590/2650 - loss 0.02883680 - time (sec): 110.71 - samples/sec: 7923.57 - lr: 0.025000 2023-04-06 10:31:59,540 epoch 102 - iter 1855/2650 - loss 0.02894956 - time (sec): 129.72 - samples/sec: 7894.59 - lr: 0.025000 2023-04-06 10:32:18,245 epoch 102 - iter 2120/2650 - loss 0.02909373 - time (sec): 148.42 - samples/sec: 7884.32 - lr: 0.025000 2023-04-06 10:32:36,873 epoch 102 - iter 2385/2650 - loss 0.02913656 - time (sec): 167.05 - samples/sec: 7880.86 - lr: 0.025000 2023-04-06 10:32:55,424 epoch 102 - iter 2650/2650 - loss 0.02910292 - time (sec): 185.60 - samples/sec: 7879.00 - lr: 0.025000 2023-04-06 10:32:55,424 ---------------------------------------------------------------------------------------------------- 2023-04-06 10:32:55,424 EPOCH 102 done: loss 0.0291 - lr 0.025000 2023-04-06 10:32:55,424 BAD EPOCHS (no improvement): 3 2023-04-06 10:32:55,427 ---------------------------------------------------------------------------------------------------- 2023-04-06 10:33:14,209 epoch 103 - iter 265/2650 - loss 0.02967606 - time (sec): 18.78 - samples/sec: 7795.58 - lr: 0.025000 2023-04-06 10:33:32,658 epoch 103 - iter 530/2650 - loss 0.02859386 - time (sec): 37.23 - samples/sec: 7854.78 - lr: 0.025000 2023-04-06 10:33:51,377 epoch 103 - iter 795/2650 - loss 0.02886302 - time (sec): 55.95 - samples/sec: 7856.93 - lr: 0.025000 2023-04-06 10:34:09,837 epoch 103 - iter 1060/2650 - loss 0.02882483 - time (sec): 74.41 - samples/sec: 7872.31 - lr: 0.025000 2023-04-06 10:34:28,185 epoch 103 - iter 1325/2650 - loss 0.02889446 - time (sec): 92.76 - samples/sec: 7899.98 - lr: 0.025000 2023-04-06 10:34:46,499 epoch 103 - iter 1590/2650 - loss 0.02891120 - time (sec): 111.07 - samples/sec: 7921.07 - lr: 0.025000 2023-04-06 10:35:04,737 epoch 103 - iter 1855/2650 - loss 0.02894634 - time (sec): 129.31 - samples/sec: 7929.77 - lr: 0.025000 2023-04-06 10:35:23,235 epoch 103 - iter 2120/2650 - loss 0.02891873 - time (sec): 147.81 - samples/sec: 7925.57 - lr: 0.025000 2023-04-06 10:35:41,538 epoch 103 - iter 2385/2650 - loss 0.02878494 - time (sec): 166.11 - samples/sec: 7936.15 - lr: 0.025000 2023-04-06 10:35:59,388 epoch 103 - iter 2650/2650 - loss 0.02882189 - time (sec): 183.96 - samples/sec: 7949.33 - lr: 0.025000 2023-04-06 10:35:59,388 ---------------------------------------------------------------------------------------------------- 2023-04-06 10:35:59,388 EPOCH 103 done: loss 0.0288 - lr 0.025000 2023-04-06 10:35:59,388 BAD EPOCHS (no improvement): 0 2023-04-06 10:35:59,391 ---------------------------------------------------------------------------------------------------- 2023-04-06 10:36:17,402 epoch 104 - iter 265/2650 - loss 0.02912064 - time (sec): 18.01 - samples/sec: 8114.92 - lr: 0.025000 2023-04-06 10:36:35,622 epoch 104 - iter 530/2650 - loss 0.02908643 - time (sec): 36.23 - samples/sec: 8094.73 - lr: 0.025000 2023-04-06 10:36:53,585 epoch 104 - iter 795/2650 - loss 0.02847266 - time (sec): 54.19 - samples/sec: 8082.99 - lr: 0.025000 2023-04-06 10:37:11,738 epoch 104 - iter 1060/2650 - loss 0.02853324 - time (sec): 72.35 - samples/sec: 8102.68 - lr: 0.025000 2023-04-06 10:37:30,117 epoch 104 - iter 1325/2650 - loss 0.02861226 - time (sec): 90.73 - samples/sec: 8085.87 - lr: 0.025000 2023-04-06 10:37:48,162 epoch 104 - iter 1590/2650 - loss 0.02871542 - time (sec): 108.77 - samples/sec: 8091.88 - lr: 0.025000 2023-04-06 10:38:06,226 epoch 104 - iter 1855/2650 - loss 0.02887285 - time (sec): 126.83 - samples/sec: 8085.55 - lr: 0.025000 2023-04-06 10:38:24,423 epoch 104 - iter 2120/2650 - loss 0.02888725 - time (sec): 145.03 - samples/sec: 8071.80 - lr: 0.025000 2023-04-06 10:38:42,449 epoch 104 - iter 2385/2650 - loss 0.02898914 - time (sec): 163.06 - samples/sec: 8072.28 - lr: 0.025000 2023-04-06 10:39:00,741 epoch 104 - iter 2650/2650 - loss 0.02894118 - time (sec): 181.35 - samples/sec: 8063.76 - lr: 0.025000 2023-04-06 10:39:00,742 ---------------------------------------------------------------------------------------------------- 2023-04-06 10:39:00,742 EPOCH 104 done: loss 0.0289 - lr 0.025000 2023-04-06 10:39:00,742 BAD EPOCHS (no improvement): 1 2023-04-06 10:39:00,745 ---------------------------------------------------------------------------------------------------- 2023-04-06 10:39:19,219 epoch 105 - iter 265/2650 - loss 0.02842705 - time (sec): 18.47 - samples/sec: 7934.05 - lr: 0.025000 2023-04-06 10:39:37,294 epoch 105 - iter 530/2650 - loss 0.02878726 - time (sec): 36.55 - samples/sec: 8004.35 - lr: 0.025000 2023-04-06 10:39:55,199 epoch 105 - iter 795/2650 - loss 0.02871867 - time (sec): 54.45 - samples/sec: 8040.42 - lr: 0.025000 2023-04-06 10:40:13,592 epoch 105 - iter 1060/2650 - loss 0.02879356 - time (sec): 72.85 - samples/sec: 8028.74 - lr: 0.025000 2023-04-06 10:40:31,942 epoch 105 - iter 1325/2650 - loss 0.02895523 - time (sec): 91.20 - samples/sec: 8019.34 - lr: 0.025000 2023-04-06 10:40:49,989 epoch 105 - iter 1590/2650 - loss 0.02863003 - time (sec): 109.24 - samples/sec: 8020.28 - lr: 0.025000 2023-04-06 10:41:07,872 epoch 105 - iter 1855/2650 - loss 0.02849800 - time (sec): 127.13 - samples/sec: 8032.82 - lr: 0.025000 2023-04-06 10:41:26,174 epoch 105 - iter 2120/2650 - loss 0.02853373 - time (sec): 145.43 - samples/sec: 8038.47 - lr: 0.025000 2023-04-06 10:41:44,536 epoch 105 - iter 2385/2650 - loss 0.02854717 - time (sec): 163.79 - samples/sec: 8030.03 - lr: 0.025000 2023-04-06 10:42:03,154 epoch 105 - iter 2650/2650 - loss 0.02869400 - time (sec): 182.41 - samples/sec: 8016.97 - lr: 0.025000 2023-04-06 10:42:03,154 ---------------------------------------------------------------------------------------------------- 2023-04-06 10:42:03,154 EPOCH 105 done: loss 0.0287 - lr 0.025000 2023-04-06 10:42:03,154 BAD EPOCHS (no improvement): 0 2023-04-06 10:42:03,158 ---------------------------------------------------------------------------------------------------- 2023-04-06 10:42:21,536 epoch 106 - iter 265/2650 - loss 0.02747638 - time (sec): 18.38 - samples/sec: 7895.74 - lr: 0.025000 2023-04-06 10:42:39,851 epoch 106 - iter 530/2650 - loss 0.02832522 - time (sec): 36.69 - samples/sec: 7932.77 - lr: 0.025000 2023-04-06 10:42:58,783 epoch 106 - iter 795/2650 - loss 0.02854966 - time (sec): 55.62 - samples/sec: 7903.00 - lr: 0.025000 2023-04-06 10:43:17,098 epoch 106 - iter 1060/2650 - loss 0.02855856 - time (sec): 73.94 - samples/sec: 7921.47 - lr: 0.025000 2023-04-06 10:43:35,752 epoch 106 - iter 1325/2650 - loss 0.02872708 - time (sec): 92.59 - samples/sec: 7902.97 - lr: 0.025000 2023-04-06 10:43:54,303 epoch 106 - iter 1590/2650 - loss 0.02858542 - time (sec): 111.14 - samples/sec: 7905.90 - lr: 0.025000 2023-04-06 10:44:12,897 epoch 106 - iter 1855/2650 - loss 0.02852546 - time (sec): 129.74 - samples/sec: 7899.35 - lr: 0.025000 2023-04-06 10:44:31,248 epoch 106 - iter 2120/2650 - loss 0.02859389 - time (sec): 148.09 - samples/sec: 7896.84 - lr: 0.025000 2023-04-06 10:44:49,337 epoch 106 - iter 2385/2650 - loss 0.02869341 - time (sec): 166.18 - samples/sec: 7912.69 - lr: 0.025000 2023-04-06 10:45:07,978 epoch 106 - iter 2650/2650 - loss 0.02854117 - time (sec): 184.82 - samples/sec: 7912.33 - lr: 0.025000 2023-04-06 10:45:07,979 ---------------------------------------------------------------------------------------------------- 2023-04-06 10:45:07,979 EPOCH 106 done: loss 0.0285 - lr 0.025000 2023-04-06 10:45:07,979 BAD EPOCHS (no improvement): 0 2023-04-06 10:45:07,982 ---------------------------------------------------------------------------------------------------- 2023-04-06 10:45:26,431 epoch 107 - iter 265/2650 - loss 0.02845543 - time (sec): 18.45 - samples/sec: 8000.34 - lr: 0.025000 2023-04-06 10:45:44,516 epoch 107 - iter 530/2650 - loss 0.02864783 - time (sec): 36.53 - samples/sec: 8035.59 - lr: 0.025000 2023-04-06 10:46:02,770 epoch 107 - iter 795/2650 - loss 0.02859257 - time (sec): 54.79 - samples/sec: 8001.38 - lr: 0.025000 2023-04-06 10:46:21,261 epoch 107 - iter 1060/2650 - loss 0.02856153 - time (sec): 73.28 - samples/sec: 7967.02 - lr: 0.025000 2023-04-06 10:46:39,603 epoch 107 - iter 1325/2650 - loss 0.02866236 - time (sec): 91.62 - samples/sec: 7963.76 - lr: 0.025000 2023-04-06 10:46:57,889 epoch 107 - iter 1590/2650 - loss 0.02874504 - time (sec): 109.91 - samples/sec: 7970.57 - lr: 0.025000 2023-04-06 10:47:16,241 epoch 107 - iter 1855/2650 - loss 0.02871136 - time (sec): 128.26 - samples/sec: 7971.64 - lr: 0.025000 2023-04-06 10:47:34,445 epoch 107 - iter 2120/2650 - loss 0.02884658 - time (sec): 146.46 - samples/sec: 7976.93 - lr: 0.025000 2023-04-06 10:47:52,746 epoch 107 - iter 2385/2650 - loss 0.02892844 - time (sec): 164.76 - samples/sec: 7980.84 - lr: 0.025000 2023-04-06 10:48:11,138 epoch 107 - iter 2650/2650 - loss 0.02880844 - time (sec): 183.16 - samples/sec: 7984.26 - lr: 0.025000 2023-04-06 10:48:11,138 ---------------------------------------------------------------------------------------------------- 2023-04-06 10:48:11,138 EPOCH 107 done: loss 0.0288 - lr 0.025000 2023-04-06 10:48:11,138 BAD EPOCHS (no improvement): 1 2023-04-06 10:48:11,141 ---------------------------------------------------------------------------------------------------- 2023-04-06 10:48:29,617 epoch 108 - iter 265/2650 - loss 0.02805338 - time (sec): 18.48 - samples/sec: 7909.93 - lr: 0.025000 2023-04-06 10:48:47,562 epoch 108 - iter 530/2650 - loss 0.02788342 - time (sec): 36.42 - samples/sec: 7999.65 - lr: 0.025000 2023-04-06 10:49:05,570 epoch 108 - iter 795/2650 - loss 0.02831989 - time (sec): 54.43 - samples/sec: 8020.85 - lr: 0.025000 2023-04-06 10:49:24,048 epoch 108 - iter 1060/2650 - loss 0.02824120 - time (sec): 72.91 - samples/sec: 7984.88 - lr: 0.025000 2023-04-06 10:49:42,321 epoch 108 - iter 1325/2650 - loss 0.02863601 - time (sec): 91.18 - samples/sec: 7976.00 - lr: 0.025000 2023-04-06 10:50:01,009 epoch 108 - iter 1590/2650 - loss 0.02876584 - time (sec): 109.87 - samples/sec: 7958.39 - lr: 0.025000 2023-04-06 10:50:19,536 epoch 108 - iter 1855/2650 - loss 0.02876441 - time (sec): 128.39 - samples/sec: 7958.59 - lr: 0.025000 2023-04-06 10:50:37,665 epoch 108 - iter 2120/2650 - loss 0.02874855 - time (sec): 146.52 - samples/sec: 7968.50 - lr: 0.025000 2023-04-06 10:50:56,480 epoch 108 - iter 2385/2650 - loss 0.02871840 - time (sec): 165.34 - samples/sec: 7961.85 - lr: 0.025000 2023-04-06 10:51:14,516 epoch 108 - iter 2650/2650 - loss 0.02876714 - time (sec): 183.37 - samples/sec: 7974.74 - lr: 0.025000 2023-04-06 10:51:14,516 ---------------------------------------------------------------------------------------------------- 2023-04-06 10:51:14,516 EPOCH 108 done: loss 0.0288 - lr 0.025000 2023-04-06 10:51:14,516 BAD EPOCHS (no improvement): 2 2023-04-06 10:51:14,519 ---------------------------------------------------------------------------------------------------- 2023-04-06 10:51:32,655 epoch 109 - iter 265/2650 - loss 0.02815426 - time (sec): 18.14 - samples/sec: 8070.39 - lr: 0.025000 2023-04-06 10:51:51,046 epoch 109 - iter 530/2650 - loss 0.02756844 - time (sec): 36.53 - samples/sec: 8032.04 - lr: 0.025000 2023-04-06 10:52:09,257 epoch 109 - iter 795/2650 - loss 0.02781535 - time (sec): 54.74 - samples/sec: 8034.73 - lr: 0.025000 2023-04-06 10:52:27,829 epoch 109 - iter 1060/2650 - loss 0.02787548 - time (sec): 73.31 - samples/sec: 7999.30 - lr: 0.025000 2023-04-06 10:52:45,864 epoch 109 - iter 1325/2650 - loss 0.02788362 - time (sec): 91.35 - samples/sec: 8011.41 - lr: 0.025000 2023-04-06 10:53:04,251 epoch 109 - iter 1590/2650 - loss 0.02793268 - time (sec): 109.73 - samples/sec: 8003.23 - lr: 0.025000 2023-04-06 10:53:22,761 epoch 109 - iter 1855/2650 - loss 0.02825866 - time (sec): 128.24 - samples/sec: 7995.09 - lr: 0.025000 2023-04-06 10:53:40,888 epoch 109 - iter 2120/2650 - loss 0.02822192 - time (sec): 146.37 - samples/sec: 7997.18 - lr: 0.025000 2023-04-06 10:53:59,401 epoch 109 - iter 2385/2650 - loss 0.02812970 - time (sec): 164.88 - samples/sec: 7993.10 - lr: 0.025000 2023-04-06 10:54:17,352 epoch 109 - iter 2650/2650 - loss 0.02816454 - time (sec): 182.83 - samples/sec: 7998.34 - lr: 0.025000 2023-04-06 10:54:17,352 ---------------------------------------------------------------------------------------------------- 2023-04-06 10:54:17,352 EPOCH 109 done: loss 0.0282 - lr 0.025000 2023-04-06 10:54:17,352 BAD EPOCHS (no improvement): 0 2023-04-06 10:54:17,356 ---------------------------------------------------------------------------------------------------- 2023-04-06 10:54:35,804 epoch 110 - iter 265/2650 - loss 0.02819089 - time (sec): 18.45 - samples/sec: 7983.71 - lr: 0.025000 2023-04-06 10:54:53,823 epoch 110 - iter 530/2650 - loss 0.02810899 - time (sec): 36.47 - samples/sec: 8039.44 - lr: 0.025000 2023-04-06 10:55:11,908 epoch 110 - iter 795/2650 - loss 0.02862162 - time (sec): 54.55 - samples/sec: 8028.92 - lr: 0.025000 2023-04-06 10:55:30,570 epoch 110 - iter 1060/2650 - loss 0.02866965 - time (sec): 73.21 - samples/sec: 8001.92 - lr: 0.025000 2023-04-06 10:55:48,963 epoch 110 - iter 1325/2650 - loss 0.02853119 - time (sec): 91.61 - samples/sec: 8001.01 - lr: 0.025000 2023-04-06 10:56:07,567 epoch 110 - iter 1590/2650 - loss 0.02840010 - time (sec): 110.21 - samples/sec: 7990.35 - lr: 0.025000 2023-04-06 10:56:25,750 epoch 110 - iter 1855/2650 - loss 0.02829936 - time (sec): 128.39 - samples/sec: 7989.72 - lr: 0.025000 2023-04-06 10:56:43,956 epoch 110 - iter 2120/2650 - loss 0.02819741 - time (sec): 146.60 - samples/sec: 7992.95 - lr: 0.025000 2023-04-06 10:57:02,122 epoch 110 - iter 2385/2650 - loss 0.02810957 - time (sec): 164.77 - samples/sec: 7996.54 - lr: 0.025000 2023-04-06 10:57:20,289 epoch 110 - iter 2650/2650 - loss 0.02812904 - time (sec): 182.93 - samples/sec: 7993.96 - lr: 0.025000 2023-04-06 10:57:20,289 ---------------------------------------------------------------------------------------------------- 2023-04-06 10:57:20,289 EPOCH 110 done: loss 0.0281 - lr 0.025000 2023-04-06 10:57:20,290 BAD EPOCHS (no improvement): 0 2023-04-06 10:57:20,292 ---------------------------------------------------------------------------------------------------- 2023-04-06 10:57:38,530 epoch 111 - iter 265/2650 - loss 0.02855048 - time (sec): 18.24 - samples/sec: 7975.23 - lr: 0.025000 2023-04-06 10:57:56,464 epoch 111 - iter 530/2650 - loss 0.02866215 - time (sec): 36.17 - samples/sec: 7999.78 - lr: 0.025000 2023-04-06 10:58:14,586 epoch 111 - iter 795/2650 - loss 0.02813147 - time (sec): 54.29 - samples/sec: 8018.65 - lr: 0.025000 2023-04-06 10:58:33,020 epoch 111 - iter 1060/2650 - loss 0.02819211 - time (sec): 72.73 - samples/sec: 8007.02 - lr: 0.025000 2023-04-06 10:58:51,370 epoch 111 - iter 1325/2650 - loss 0.02811289 - time (sec): 91.08 - samples/sec: 7992.03 - lr: 0.025000 2023-04-06 10:59:09,773 epoch 111 - iter 1590/2650 - loss 0.02803792 - time (sec): 109.48 - samples/sec: 7996.50 - lr: 0.025000 2023-04-06 10:59:27,729 epoch 111 - iter 1855/2650 - loss 0.02802278 - time (sec): 127.44 - samples/sec: 8017.73 - lr: 0.025000 2023-04-06 10:59:45,523 epoch 111 - iter 2120/2650 - loss 0.02800724 - time (sec): 145.23 - samples/sec: 8041.65 - lr: 0.025000 2023-04-06 11:00:04,234 epoch 111 - iter 2385/2650 - loss 0.02806012 - time (sec): 163.94 - samples/sec: 8022.13 - lr: 0.025000 2023-04-06 11:00:22,498 epoch 111 - iter 2650/2650 - loss 0.02808036 - time (sec): 182.21 - samples/sec: 8025.89 - lr: 0.025000 2023-04-06 11:00:22,498 ---------------------------------------------------------------------------------------------------- 2023-04-06 11:00:22,498 EPOCH 111 done: loss 0.0281 - lr 0.025000 2023-04-06 11:00:22,498 BAD EPOCHS (no improvement): 0 2023-04-06 11:00:22,500 ---------------------------------------------------------------------------------------------------- 2023-04-06 11:00:41,432 epoch 112 - iter 265/2650 - loss 0.02808795 - time (sec): 18.93 - samples/sec: 7794.32 - lr: 0.025000 2023-04-06 11:00:59,609 epoch 112 - iter 530/2650 - loss 0.02831789 - time (sec): 37.11 - samples/sec: 7894.26 - lr: 0.025000 2023-04-06 11:01:17,776 epoch 112 - iter 795/2650 - loss 0.02850330 - time (sec): 55.28 - samples/sec: 7945.23 - lr: 0.025000 2023-04-06 11:01:36,236 epoch 112 - iter 1060/2650 - loss 0.02843828 - time (sec): 73.74 - samples/sec: 7959.14 - lr: 0.025000 2023-04-06 11:01:54,266 epoch 112 - iter 1325/2650 - loss 0.02827085 - time (sec): 91.77 - samples/sec: 7979.62 - lr: 0.025000 2023-04-06 11:02:12,449 epoch 112 - iter 1590/2650 - loss 0.02806191 - time (sec): 109.95 - samples/sec: 7985.45 - lr: 0.025000 2023-04-06 11:02:30,007 epoch 112 - iter 1855/2650 - loss 0.02814577 - time (sec): 127.51 - samples/sec: 8011.80 - lr: 0.025000 2023-04-06 11:02:48,013 epoch 112 - iter 2120/2650 - loss 0.02815071 - time (sec): 145.51 - samples/sec: 8023.49 - lr: 0.025000 2023-04-06 11:03:06,820 epoch 112 - iter 2385/2650 - loss 0.02809798 - time (sec): 164.32 - samples/sec: 8001.24 - lr: 0.025000 2023-04-06 11:03:25,641 epoch 112 - iter 2650/2650 - loss 0.02820890 - time (sec): 183.14 - samples/sec: 7984.93 - lr: 0.025000 2023-04-06 11:03:25,641 ---------------------------------------------------------------------------------------------------- 2023-04-06 11:03:25,641 EPOCH 112 done: loss 0.0282 - lr 0.025000 2023-04-06 11:03:25,641 BAD EPOCHS (no improvement): 1 2023-04-06 11:03:25,644 ---------------------------------------------------------------------------------------------------- 2023-04-06 11:03:43,743 epoch 113 - iter 265/2650 - loss 0.02898648 - time (sec): 18.10 - samples/sec: 8009.77 - lr: 0.025000 2023-04-06 11:04:01,789 epoch 113 - iter 530/2650 - loss 0.02891836 - time (sec): 36.14 - samples/sec: 8049.98 - lr: 0.025000 2023-04-06 11:04:20,098 epoch 113 - iter 795/2650 - loss 0.02844602 - time (sec): 54.45 - samples/sec: 8039.41 - lr: 0.025000 2023-04-06 11:04:38,438 epoch 113 - iter 1060/2650 - loss 0.02833292 - time (sec): 72.79 - samples/sec: 8034.37 - lr: 0.025000 2023-04-06 11:04:57,036 epoch 113 - iter 1325/2650 - loss 0.02816762 - time (sec): 91.39 - samples/sec: 8009.38 - lr: 0.025000 2023-04-06 11:05:14,956 epoch 113 - iter 1590/2650 - loss 0.02802957 - time (sec): 109.31 - samples/sec: 8027.39 - lr: 0.025000 2023-04-06 11:05:33,653 epoch 113 - iter 1855/2650 - loss 0.02811505 - time (sec): 128.01 - samples/sec: 8005.99 - lr: 0.025000 2023-04-06 11:05:51,957 epoch 113 - iter 2120/2650 - loss 0.02822114 - time (sec): 146.31 - samples/sec: 8001.12 - lr: 0.025000 2023-04-06 11:06:10,453 epoch 113 - iter 2385/2650 - loss 0.02823519 - time (sec): 164.81 - samples/sec: 7990.13 - lr: 0.025000 2023-04-06 11:06:28,417 epoch 113 - iter 2650/2650 - loss 0.02815070 - time (sec): 182.77 - samples/sec: 8000.99 - lr: 0.025000 2023-04-06 11:06:28,417 ---------------------------------------------------------------------------------------------------- 2023-04-06 11:06:28,417 EPOCH 113 done: loss 0.0282 - lr 0.025000 2023-04-06 11:06:28,417 BAD EPOCHS (no improvement): 2 2023-04-06 11:06:28,421 ---------------------------------------------------------------------------------------------------- 2023-04-06 11:06:46,642 epoch 114 - iter 265/2650 - loss 0.02741600 - time (sec): 18.22 - samples/sec: 8015.99 - lr: 0.025000 2023-04-06 11:07:05,139 epoch 114 - iter 530/2650 - loss 0.02760826 - time (sec): 36.72 - samples/sec: 7996.71 - lr: 0.025000 2023-04-06 11:07:23,068 epoch 114 - iter 795/2650 - loss 0.02764054 - time (sec): 54.65 - samples/sec: 8050.61 - lr: 0.025000 2023-04-06 11:07:41,206 epoch 114 - iter 1060/2650 - loss 0.02786044 - time (sec): 72.79 - samples/sec: 8056.92 - lr: 0.025000 2023-04-06 11:07:59,240 epoch 114 - iter 1325/2650 - loss 0.02799111 - time (sec): 90.82 - samples/sec: 8045.08 - lr: 0.025000 2023-04-06 11:08:17,297 epoch 114 - iter 1590/2650 - loss 0.02809860 - time (sec): 108.88 - samples/sec: 8047.92 - lr: 0.025000 2023-04-06 11:08:35,490 epoch 114 - iter 1855/2650 - loss 0.02803108 - time (sec): 127.07 - samples/sec: 8048.95 - lr: 0.025000 2023-04-06 11:08:53,772 epoch 114 - iter 2120/2650 - loss 0.02816532 - time (sec): 145.35 - samples/sec: 8043.45 - lr: 0.025000 2023-04-06 11:09:12,143 epoch 114 - iter 2385/2650 - loss 0.02810536 - time (sec): 163.72 - samples/sec: 8044.44 - lr: 0.025000 2023-04-06 11:09:30,193 epoch 114 - iter 2650/2650 - loss 0.02812922 - time (sec): 181.77 - samples/sec: 8045.04 - lr: 0.025000 2023-04-06 11:09:30,193 ---------------------------------------------------------------------------------------------------- 2023-04-06 11:09:30,193 EPOCH 114 done: loss 0.0281 - lr 0.025000 2023-04-06 11:09:30,193 BAD EPOCHS (no improvement): 3 2023-04-06 11:09:30,196 ---------------------------------------------------------------------------------------------------- 2023-04-06 11:09:48,332 epoch 115 - iter 265/2650 - loss 0.02642194 - time (sec): 18.14 - samples/sec: 8056.22 - lr: 0.025000 2023-04-06 11:10:06,143 epoch 115 - iter 530/2650 - loss 0.02724488 - time (sec): 35.95 - samples/sec: 8095.78 - lr: 0.025000 2023-04-06 11:10:24,195 epoch 115 - iter 795/2650 - loss 0.02775477 - time (sec): 54.00 - samples/sec: 8099.44 - lr: 0.025000 2023-04-06 11:10:42,864 epoch 115 - iter 1060/2650 - loss 0.02758944 - time (sec): 72.67 - samples/sec: 8057.12 - lr: 0.025000 2023-04-06 11:11:01,183 epoch 115 - iter 1325/2650 - loss 0.02776471 - time (sec): 90.99 - samples/sec: 8052.54 - lr: 0.025000 2023-04-06 11:11:20,119 epoch 115 - iter 1590/2650 - loss 0.02767564 - time (sec): 109.92 - samples/sec: 8002.77 - lr: 0.025000 2023-04-06 11:11:38,689 epoch 115 - iter 1855/2650 - loss 0.02762188 - time (sec): 128.49 - samples/sec: 7983.78 - lr: 0.025000 2023-04-06 11:11:56,871 epoch 115 - iter 2120/2650 - loss 0.02761635 - time (sec): 146.68 - samples/sec: 7991.74 - lr: 0.025000 2023-04-06 11:12:15,478 epoch 115 - iter 2385/2650 - loss 0.02763510 - time (sec): 165.28 - samples/sec: 7978.50 - lr: 0.025000 2023-04-06 11:12:33,710 epoch 115 - iter 2650/2650 - loss 0.02761561 - time (sec): 183.51 - samples/sec: 7968.67 - lr: 0.025000 2023-04-06 11:12:33,710 ---------------------------------------------------------------------------------------------------- 2023-04-06 11:12:33,710 EPOCH 115 done: loss 0.0276 - lr 0.025000 2023-04-06 11:12:33,710 BAD EPOCHS (no improvement): 0 2023-04-06 11:12:33,713 ---------------------------------------------------------------------------------------------------- 2023-04-06 11:12:51,819 epoch 116 - iter 265/2650 - loss 0.02617465 - time (sec): 18.11 - samples/sec: 8163.29 - lr: 0.025000 2023-04-06 11:13:10,372 epoch 116 - iter 530/2650 - loss 0.02674889 - time (sec): 36.66 - samples/sec: 7997.60 - lr: 0.025000 2023-04-06 11:13:29,014 epoch 116 - iter 795/2650 - loss 0.02690283 - time (sec): 55.30 - samples/sec: 7927.93 - lr: 0.025000 2023-04-06 11:13:47,946 epoch 116 - iter 1060/2650 - loss 0.02693925 - time (sec): 74.23 - samples/sec: 7876.87 - lr: 0.025000 2023-04-06 11:14:06,545 epoch 116 - iter 1325/2650 - loss 0.02710702 - time (sec): 92.83 - samples/sec: 7865.81 - lr: 0.025000 2023-04-06 11:14:24,844 epoch 116 - iter 1590/2650 - loss 0.02732330 - time (sec): 111.13 - samples/sec: 7879.62 - lr: 0.025000 2023-04-06 11:14:43,377 epoch 116 - iter 1855/2650 - loss 0.02752942 - time (sec): 129.66 - samples/sec: 7876.93 - lr: 0.025000 2023-04-06 11:15:02,039 epoch 116 - iter 2120/2650 - loss 0.02749707 - time (sec): 148.33 - samples/sec: 7868.83 - lr: 0.025000 2023-04-06 11:15:20,693 epoch 116 - iter 2385/2650 - loss 0.02765499 - time (sec): 166.98 - samples/sec: 7873.75 - lr: 0.025000 2023-04-06 11:15:39,041 epoch 116 - iter 2650/2650 - loss 0.02767956 - time (sec): 185.33 - samples/sec: 7890.68 - lr: 0.025000 2023-04-06 11:15:39,041 ---------------------------------------------------------------------------------------------------- 2023-04-06 11:15:39,041 EPOCH 116 done: loss 0.0277 - lr 0.025000 2023-04-06 11:15:39,041 BAD EPOCHS (no improvement): 1 2023-04-06 11:15:39,047 ---------------------------------------------------------------------------------------------------- 2023-04-06 11:15:57,399 epoch 117 - iter 265/2650 - loss 0.02710422 - time (sec): 18.35 - samples/sec: 7940.88 - lr: 0.025000 2023-04-06 11:16:15,482 epoch 117 - iter 530/2650 - loss 0.02777508 - time (sec): 36.43 - samples/sec: 7965.48 - lr: 0.025000 2023-04-06 11:16:34,431 epoch 117 - iter 795/2650 - loss 0.02746774 - time (sec): 55.38 - samples/sec: 7907.87 - lr: 0.025000 2023-04-06 11:16:52,832 epoch 117 - iter 1060/2650 - loss 0.02734153 - time (sec): 73.78 - samples/sec: 7897.92 - lr: 0.025000 2023-04-06 11:17:10,960 epoch 117 - iter 1325/2650 - loss 0.02713756 - time (sec): 91.91 - samples/sec: 7925.78 - lr: 0.025000 2023-04-06 11:17:29,689 epoch 117 - iter 1590/2650 - loss 0.02742410 - time (sec): 110.64 - samples/sec: 7907.91 - lr: 0.025000 2023-04-06 11:17:47,677 epoch 117 - iter 1855/2650 - loss 0.02747933 - time (sec): 128.63 - samples/sec: 7942.46 - lr: 0.025000 2023-04-06 11:18:05,844 epoch 117 - iter 2120/2650 - loss 0.02756131 - time (sec): 146.80 - samples/sec: 7957.84 - lr: 0.025000 2023-04-06 11:18:24,363 epoch 117 - iter 2385/2650 - loss 0.02758241 - time (sec): 165.32 - samples/sec: 7955.57 - lr: 0.025000 2023-04-06 11:18:42,797 epoch 117 - iter 2650/2650 - loss 0.02770073 - time (sec): 183.75 - samples/sec: 7958.46 - lr: 0.025000 2023-04-06 11:18:42,797 ---------------------------------------------------------------------------------------------------- 2023-04-06 11:18:42,797 EPOCH 117 done: loss 0.0277 - lr 0.025000 2023-04-06 11:18:42,797 BAD EPOCHS (no improvement): 2 2023-04-06 11:18:42,800 ---------------------------------------------------------------------------------------------------- 2023-04-06 11:19:01,029 epoch 118 - iter 265/2650 - loss 0.02797587 - time (sec): 18.23 - samples/sec: 8037.77 - lr: 0.025000 2023-04-06 11:19:19,013 epoch 118 - iter 530/2650 - loss 0.02758550 - time (sec): 36.21 - samples/sec: 8051.72 - lr: 0.025000 2023-04-06 11:19:37,533 epoch 118 - iter 795/2650 - loss 0.02746952 - time (sec): 54.73 - samples/sec: 8018.84 - lr: 0.025000 2023-04-06 11:19:56,253 epoch 118 - iter 1060/2650 - loss 0.02749424 - time (sec): 73.45 - samples/sec: 7965.11 - lr: 0.025000 2023-04-06 11:20:14,416 epoch 118 - iter 1325/2650 - loss 0.02744679 - time (sec): 91.62 - samples/sec: 7971.53 - lr: 0.025000 2023-04-06 11:20:33,008 epoch 118 - iter 1590/2650 - loss 0.02748469 - time (sec): 110.21 - samples/sec: 7961.45 - lr: 0.025000 2023-04-06 11:20:51,488 epoch 118 - iter 1855/2650 - loss 0.02757637 - time (sec): 128.69 - samples/sec: 7960.08 - lr: 0.025000 2023-04-06 11:21:10,038 epoch 118 - iter 2120/2650 - loss 0.02767669 - time (sec): 147.24 - samples/sec: 7953.23 - lr: 0.025000 2023-04-06 11:21:28,463 epoch 118 - iter 2385/2650 - loss 0.02760613 - time (sec): 165.66 - samples/sec: 7950.46 - lr: 0.025000 2023-04-06 11:21:46,499 epoch 118 - iter 2650/2650 - loss 0.02763548 - time (sec): 183.70 - samples/sec: 7960.66 - lr: 0.025000 2023-04-06 11:21:46,499 ---------------------------------------------------------------------------------------------------- 2023-04-06 11:21:46,499 EPOCH 118 done: loss 0.0276 - lr 0.025000 2023-04-06 11:21:46,499 BAD EPOCHS (no improvement): 3 2023-04-06 11:21:46,502 ---------------------------------------------------------------------------------------------------- 2023-04-06 11:22:04,802 epoch 119 - iter 265/2650 - loss 0.02752910 - time (sec): 18.30 - samples/sec: 7938.03 - lr: 0.025000 2023-04-06 11:22:23,137 epoch 119 - iter 530/2650 - loss 0.02707297 - time (sec): 36.64 - samples/sec: 7955.01 - lr: 0.025000 2023-04-06 11:22:41,775 epoch 119 - iter 795/2650 - loss 0.02731471 - time (sec): 55.27 - samples/sec: 7931.66 - lr: 0.025000 2023-04-06 11:22:59,851 epoch 119 - iter 1060/2650 - loss 0.02744191 - time (sec): 73.35 - samples/sec: 7950.94 - lr: 0.025000 2023-04-06 11:23:17,941 epoch 119 - iter 1325/2650 - loss 0.02747602 - time (sec): 91.44 - samples/sec: 7989.23 - lr: 0.025000 2023-04-06 11:23:36,001 epoch 119 - iter 1590/2650 - loss 0.02734308 - time (sec): 109.50 - samples/sec: 8003.02 - lr: 0.025000 2023-04-06 11:23:54,682 epoch 119 - iter 1855/2650 - loss 0.02731932 - time (sec): 128.18 - samples/sec: 7985.70 - lr: 0.025000 2023-04-06 11:24:12,521 epoch 119 - iter 2120/2650 - loss 0.02726512 - time (sec): 146.02 - samples/sec: 8000.03 - lr: 0.025000 2023-04-06 11:24:31,099 epoch 119 - iter 2385/2650 - loss 0.02742967 - time (sec): 164.60 - samples/sec: 7987.77 - lr: 0.025000 2023-04-06 11:24:49,657 epoch 119 - iter 2650/2650 - loss 0.02742113 - time (sec): 183.15 - samples/sec: 7984.30 - lr: 0.025000 2023-04-06 11:24:49,657 ---------------------------------------------------------------------------------------------------- 2023-04-06 11:24:49,657 EPOCH 119 done: loss 0.0274 - lr 0.025000 2023-04-06 11:24:49,657 BAD EPOCHS (no improvement): 0 2023-04-06 11:24:49,660 ---------------------------------------------------------------------------------------------------- 2023-04-06 11:25:08,231 epoch 120 - iter 265/2650 - loss 0.02703443 - time (sec): 18.57 - samples/sec: 7933.66 - lr: 0.025000 2023-04-06 11:25:26,301 epoch 120 - iter 530/2650 - loss 0.02777405 - time (sec): 36.64 - samples/sec: 7953.95 - lr: 0.025000 2023-04-06 11:25:44,655 epoch 120 - iter 795/2650 - loss 0.02756376 - time (sec): 55.00 - samples/sec: 7961.56 - lr: 0.025000 2023-04-06 11:26:03,159 epoch 120 - iter 1060/2650 - loss 0.02705291 - time (sec): 73.50 - samples/sec: 7950.27 - lr: 0.025000 2023-04-06 11:26:21,930 epoch 120 - iter 1325/2650 - loss 0.02714080 - time (sec): 92.27 - samples/sec: 7922.87 - lr: 0.025000 2023-04-06 11:26:40,399 epoch 120 - iter 1590/2650 - loss 0.02719704 - time (sec): 110.74 - samples/sec: 7922.52 - lr: 0.025000 2023-04-06 11:26:58,456 epoch 120 - iter 1855/2650 - loss 0.02704220 - time (sec): 128.80 - samples/sec: 7941.13 - lr: 0.025000 2023-04-06 11:27:16,936 epoch 120 - iter 2120/2650 - loss 0.02702062 - time (sec): 147.28 - samples/sec: 7940.86 - lr: 0.025000 2023-04-06 11:27:34,925 epoch 120 - iter 2385/2650 - loss 0.02705738 - time (sec): 165.27 - samples/sec: 7956.80 - lr: 0.025000 2023-04-06 11:27:53,588 epoch 120 - iter 2650/2650 - loss 0.02706860 - time (sec): 183.93 - samples/sec: 7950.73 - lr: 0.025000 2023-04-06 11:27:53,588 ---------------------------------------------------------------------------------------------------- 2023-04-06 11:27:53,588 EPOCH 120 done: loss 0.0271 - lr 0.025000 2023-04-06 11:27:53,589 BAD EPOCHS (no improvement): 0 2023-04-06 11:27:53,592 ---------------------------------------------------------------------------------------------------- 2023-04-06 11:28:12,051 epoch 121 - iter 265/2650 - loss 0.02663318 - time (sec): 18.46 - samples/sec: 7915.37 - lr: 0.025000 2023-04-06 11:28:30,166 epoch 121 - iter 530/2650 - loss 0.02774453 - time (sec): 36.57 - samples/sec: 7996.20 - lr: 0.025000 2023-04-06 11:28:48,465 epoch 121 - iter 795/2650 - loss 0.02736845 - time (sec): 54.87 - samples/sec: 7978.36 - lr: 0.025000 2023-04-06 11:29:06,875 epoch 121 - iter 1060/2650 - loss 0.02723638 - time (sec): 73.28 - samples/sec: 7978.76 - lr: 0.025000 2023-04-06 11:29:24,909 epoch 121 - iter 1325/2650 - loss 0.02701161 - time (sec): 91.32 - samples/sec: 7984.78 - lr: 0.025000 2023-04-06 11:29:43,261 epoch 121 - iter 1590/2650 - loss 0.02705523 - time (sec): 109.67 - samples/sec: 7986.11 - lr: 0.025000 2023-04-06 11:30:01,878 epoch 121 - iter 1855/2650 - loss 0.02727763 - time (sec): 128.29 - samples/sec: 7966.00 - lr: 0.025000 2023-04-06 11:30:20,311 epoch 121 - iter 2120/2650 - loss 0.02735824 - time (sec): 146.72 - samples/sec: 7966.56 - lr: 0.025000 2023-04-06 11:30:38,958 epoch 121 - iter 2385/2650 - loss 0.02740457 - time (sec): 165.37 - samples/sec: 7950.48 - lr: 0.025000 2023-04-06 11:30:57,536 epoch 121 - iter 2650/2650 - loss 0.02743156 - time (sec): 183.94 - samples/sec: 7950.05 - lr: 0.025000 2023-04-06 11:30:57,536 ---------------------------------------------------------------------------------------------------- 2023-04-06 11:30:57,536 EPOCH 121 done: loss 0.0274 - lr 0.025000 2023-04-06 11:30:57,536 BAD EPOCHS (no improvement): 1 2023-04-06 11:30:57,540 ---------------------------------------------------------------------------------------------------- 2023-04-06 11:31:15,835 epoch 122 - iter 265/2650 - loss 0.02648890 - time (sec): 18.30 - samples/sec: 7960.00 - lr: 0.025000 2023-04-06 11:31:34,129 epoch 122 - iter 530/2650 - loss 0.02701858 - time (sec): 36.59 - samples/sec: 7965.50 - lr: 0.025000 2023-04-06 11:31:52,557 epoch 122 - iter 795/2650 - loss 0.02679734 - time (sec): 55.02 - samples/sec: 7955.97 - lr: 0.025000 2023-04-06 11:32:10,478 epoch 122 - iter 1060/2650 - loss 0.02698865 - time (sec): 72.94 - samples/sec: 7989.96 - lr: 0.025000 2023-04-06 11:32:28,863 epoch 122 - iter 1325/2650 - loss 0.02706848 - time (sec): 91.32 - samples/sec: 7986.76 - lr: 0.025000 2023-04-06 11:32:47,491 epoch 122 - iter 1590/2650 - loss 0.02729401 - time (sec): 109.95 - samples/sec: 7963.55 - lr: 0.025000 2023-04-06 11:33:06,528 epoch 122 - iter 1855/2650 - loss 0.02734037 - time (sec): 128.99 - samples/sec: 7930.76 - lr: 0.025000 2023-04-06 11:33:24,554 epoch 122 - iter 2120/2650 - loss 0.02724864 - time (sec): 147.01 - samples/sec: 7953.13 - lr: 0.025000 2023-04-06 11:33:42,906 epoch 122 - iter 2385/2650 - loss 0.02729012 - time (sec): 165.37 - samples/sec: 7969.13 - lr: 0.025000 2023-04-06 11:34:01,222 epoch 122 - iter 2650/2650 - loss 0.02736804 - time (sec): 183.68 - samples/sec: 7961.38 - lr: 0.025000 2023-04-06 11:34:01,222 ---------------------------------------------------------------------------------------------------- 2023-04-06 11:34:01,222 EPOCH 122 done: loss 0.0274 - lr 0.025000 2023-04-06 11:34:01,222 BAD EPOCHS (no improvement): 2 2023-04-06 11:34:01,225 ---------------------------------------------------------------------------------------------------- 2023-04-06 11:34:19,429 epoch 123 - iter 265/2650 - loss 0.02816273 - time (sec): 18.20 - samples/sec: 8017.71 - lr: 0.025000 2023-04-06 11:34:37,919 epoch 123 - iter 530/2650 - loss 0.02741931 - time (sec): 36.69 - samples/sec: 7937.68 - lr: 0.025000 2023-04-06 11:34:56,200 epoch 123 - iter 795/2650 - loss 0.02729499 - time (sec): 54.97 - samples/sec: 7969.17 - lr: 0.025000 2023-04-06 11:35:14,449 epoch 123 - iter 1060/2650 - loss 0.02723887 - time (sec): 73.22 - samples/sec: 8004.44 - lr: 0.025000 2023-04-06 11:35:32,732 epoch 123 - iter 1325/2650 - loss 0.02712680 - time (sec): 91.51 - samples/sec: 8008.83 - lr: 0.025000 2023-04-06 11:35:50,675 epoch 123 - iter 1590/2650 - loss 0.02735669 - time (sec): 109.45 - samples/sec: 8021.39 - lr: 0.025000 2023-04-06 11:36:09,295 epoch 123 - iter 1855/2650 - loss 0.02759161 - time (sec): 128.07 - samples/sec: 8008.45 - lr: 0.025000 2023-04-06 11:36:27,713 epoch 123 - iter 2120/2650 - loss 0.02748687 - time (sec): 146.49 - samples/sec: 7989.42 - lr: 0.025000 2023-04-06 11:36:46,527 epoch 123 - iter 2385/2650 - loss 0.02742551 - time (sec): 165.30 - samples/sec: 7970.08 - lr: 0.025000 2023-04-06 11:37:04,768 epoch 123 - iter 2650/2650 - loss 0.02741615 - time (sec): 183.54 - samples/sec: 7967.44 - lr: 0.025000 2023-04-06 11:37:04,768 ---------------------------------------------------------------------------------------------------- 2023-04-06 11:37:04,768 EPOCH 123 done: loss 0.0274 - lr 0.025000 2023-04-06 11:37:04,768 BAD EPOCHS (no improvement): 3 2023-04-06 11:37:04,772 ---------------------------------------------------------------------------------------------------- 2023-04-06 11:37:23,865 epoch 124 - iter 265/2650 - loss 0.02646129 - time (sec): 19.09 - samples/sec: 7712.25 - lr: 0.025000 2023-04-06 11:37:42,463 epoch 124 - iter 530/2650 - loss 0.02690138 - time (sec): 37.69 - samples/sec: 7821.90 - lr: 0.025000 2023-04-06 11:38:00,326 epoch 124 - iter 795/2650 - loss 0.02709115 - time (sec): 55.55 - samples/sec: 7919.42 - lr: 0.025000 2023-04-06 11:38:18,657 epoch 124 - iter 1060/2650 - loss 0.02694646 - time (sec): 73.89 - samples/sec: 7921.79 - lr: 0.025000 2023-04-06 11:38:36,740 epoch 124 - iter 1325/2650 - loss 0.02670769 - time (sec): 91.97 - samples/sec: 7946.12 - lr: 0.025000 2023-04-06 11:38:54,952 epoch 124 - iter 1590/2650 - loss 0.02669745 - time (sec): 110.18 - samples/sec: 7945.28 - lr: 0.025000 2023-04-06 11:39:13,130 epoch 124 - iter 1855/2650 - loss 0.02681020 - time (sec): 128.36 - samples/sec: 7952.11 - lr: 0.025000 2023-04-06 11:39:31,763 epoch 124 - iter 2120/2650 - loss 0.02677323 - time (sec): 146.99 - samples/sec: 7950.24 - lr: 0.025000 2023-04-06 11:39:50,284 epoch 124 - iter 2385/2650 - loss 0.02690120 - time (sec): 165.51 - samples/sec: 7947.35 - lr: 0.025000 2023-04-06 11:40:08,990 epoch 124 - iter 2650/2650 - loss 0.02699719 - time (sec): 184.22 - samples/sec: 7938.21 - lr: 0.025000 2023-04-06 11:40:08,990 ---------------------------------------------------------------------------------------------------- 2023-04-06 11:40:08,990 EPOCH 124 done: loss 0.0270 - lr 0.025000 2023-04-06 11:40:08,990 BAD EPOCHS (no improvement): 0 2023-04-06 11:40:08,994 ---------------------------------------------------------------------------------------------------- 2023-04-06 11:40:27,302 epoch 125 - iter 265/2650 - loss 0.02612355 - time (sec): 18.31 - samples/sec: 7980.90 - lr: 0.025000 2023-04-06 11:40:45,619 epoch 125 - iter 530/2650 - loss 0.02611982 - time (sec): 36.63 - samples/sec: 8018.43 - lr: 0.025000 2023-04-06 11:41:03,800 epoch 125 - iter 795/2650 - loss 0.02650682 - time (sec): 54.81 - samples/sec: 8000.01 - lr: 0.025000 2023-04-06 11:41:21,887 epoch 125 - iter 1060/2650 - loss 0.02674618 - time (sec): 72.89 - samples/sec: 8013.77 - lr: 0.025000 2023-04-06 11:41:40,560 epoch 125 - iter 1325/2650 - loss 0.02697408 - time (sec): 91.57 - samples/sec: 7988.32 - lr: 0.025000 2023-04-06 11:41:59,118 epoch 125 - iter 1590/2650 - loss 0.02720237 - time (sec): 110.12 - samples/sec: 7977.78 - lr: 0.025000 2023-04-06 11:42:17,490 epoch 125 - iter 1855/2650 - loss 0.02707398 - time (sec): 128.50 - samples/sec: 7971.66 - lr: 0.025000 2023-04-06 11:42:36,082 epoch 125 - iter 2120/2650 - loss 0.02707733 - time (sec): 147.09 - samples/sec: 7952.63 - lr: 0.025000 2023-04-06 11:42:54,892 epoch 125 - iter 2385/2650 - loss 0.02706838 - time (sec): 165.90 - samples/sec: 7939.21 - lr: 0.025000 2023-04-06 11:43:13,634 epoch 125 - iter 2650/2650 - loss 0.02710298 - time (sec): 184.64 - samples/sec: 7920.06 - lr: 0.025000 2023-04-06 11:43:13,634 ---------------------------------------------------------------------------------------------------- 2023-04-06 11:43:13,634 EPOCH 125 done: loss 0.0271 - lr 0.025000 2023-04-06 11:43:13,634 BAD EPOCHS (no improvement): 1 2023-04-06 11:43:13,637 ---------------------------------------------------------------------------------------------------- 2023-04-06 11:43:32,527 epoch 126 - iter 265/2650 - loss 0.02743903 - time (sec): 18.89 - samples/sec: 7734.76 - lr: 0.025000 2023-04-06 11:43:50,738 epoch 126 - iter 530/2650 - loss 0.02685659 - time (sec): 37.10 - samples/sec: 7902.17 - lr: 0.025000 2023-04-06 11:44:08,977 epoch 126 - iter 795/2650 - loss 0.02654346 - time (sec): 55.34 - samples/sec: 7931.14 - lr: 0.025000 2023-04-06 11:44:27,292 epoch 126 - iter 1060/2650 - loss 0.02703749 - time (sec): 73.65 - samples/sec: 7962.44 - lr: 0.025000 2023-04-06 11:44:45,598 epoch 126 - iter 1325/2650 - loss 0.02698742 - time (sec): 91.96 - samples/sec: 7957.09 - lr: 0.025000 2023-04-06 11:45:04,408 epoch 126 - iter 1590/2650 - loss 0.02698185 - time (sec): 110.77 - samples/sec: 7942.67 - lr: 0.025000 2023-04-06 11:45:23,136 epoch 126 - iter 1855/2650 - loss 0.02692939 - time (sec): 129.50 - samples/sec: 7921.85 - lr: 0.025000 2023-04-06 11:45:41,193 epoch 126 - iter 2120/2650 - loss 0.02671943 - time (sec): 147.56 - samples/sec: 7944.55 - lr: 0.025000 2023-04-06 11:45:59,173 epoch 126 - iter 2385/2650 - loss 0.02674861 - time (sec): 165.54 - samples/sec: 7956.15 - lr: 0.025000 2023-04-06 11:46:17,458 epoch 126 - iter 2650/2650 - loss 0.02676811 - time (sec): 183.82 - samples/sec: 7955.37 - lr: 0.025000 2023-04-06 11:46:17,458 ---------------------------------------------------------------------------------------------------- 2023-04-06 11:46:17,458 EPOCH 126 done: loss 0.0268 - lr 0.025000 2023-04-06 11:46:17,458 BAD EPOCHS (no improvement): 0 2023-04-06 11:46:17,461 ---------------------------------------------------------------------------------------------------- 2023-04-06 11:46:35,805 epoch 127 - iter 265/2650 - loss 0.02618426 - time (sec): 18.34 - samples/sec: 8018.78 - lr: 0.025000 2023-04-06 11:46:53,811 epoch 127 - iter 530/2650 - loss 0.02626006 - time (sec): 36.35 - samples/sec: 8081.77 - lr: 0.025000 2023-04-06 11:47:11,865 epoch 127 - iter 795/2650 - loss 0.02621834 - time (sec): 54.40 - samples/sec: 8068.99 - lr: 0.025000 2023-04-06 11:47:30,217 epoch 127 - iter 1060/2650 - loss 0.02653584 - time (sec): 72.76 - samples/sec: 8056.77 - lr: 0.025000 2023-04-06 11:47:48,299 epoch 127 - iter 1325/2650 - loss 0.02668489 - time (sec): 90.84 - samples/sec: 8059.96 - lr: 0.025000 2023-04-06 11:48:06,451 epoch 127 - iter 1590/2650 - loss 0.02684528 - time (sec): 108.99 - samples/sec: 8057.49 - lr: 0.025000 2023-04-06 11:48:24,972 epoch 127 - iter 1855/2650 - loss 0.02680900 - time (sec): 127.51 - samples/sec: 8044.89 - lr: 0.025000 2023-04-06 11:48:43,279 epoch 127 - iter 2120/2650 - loss 0.02690088 - time (sec): 145.82 - samples/sec: 8030.53 - lr: 0.025000 2023-04-06 11:49:02,202 epoch 127 - iter 2385/2650 - loss 0.02698646 - time (sec): 164.74 - samples/sec: 7994.46 - lr: 0.025000 2023-04-06 11:49:20,833 epoch 127 - iter 2650/2650 - loss 0.02693606 - time (sec): 183.37 - samples/sec: 7974.86 - lr: 0.025000 2023-04-06 11:49:20,833 ---------------------------------------------------------------------------------------------------- 2023-04-06 11:49:20,833 EPOCH 127 done: loss 0.0269 - lr 0.025000 2023-04-06 11:49:20,833 BAD EPOCHS (no improvement): 1 2023-04-06 11:49:20,840 ---------------------------------------------------------------------------------------------------- 2023-04-06 11:49:39,368 epoch 128 - iter 265/2650 - loss 0.02682269 - time (sec): 18.53 - samples/sec: 7811.95 - lr: 0.025000 2023-04-06 11:49:58,403 epoch 128 - iter 530/2650 - loss 0.02695510 - time (sec): 37.56 - samples/sec: 7782.27 - lr: 0.025000 2023-04-06 11:50:16,872 epoch 128 - iter 795/2650 - loss 0.02694181 - time (sec): 56.03 - samples/sec: 7813.08 - lr: 0.025000 2023-04-06 11:50:35,601 epoch 128 - iter 1060/2650 - loss 0.02707570 - time (sec): 74.76 - samples/sec: 7832.97 - lr: 0.025000 2023-04-06 11:50:54,128 epoch 128 - iter 1325/2650 - loss 0.02689029 - time (sec): 93.29 - samples/sec: 7862.72 - lr: 0.025000 2023-04-06 11:51:12,514 epoch 128 - iter 1590/2650 - loss 0.02684079 - time (sec): 111.67 - samples/sec: 7864.69 - lr: 0.025000 2023-04-06 11:51:30,989 epoch 128 - iter 1855/2650 - loss 0.02693136 - time (sec): 130.15 - samples/sec: 7873.07 - lr: 0.025000 2023-04-06 11:51:49,240 epoch 128 - iter 2120/2650 - loss 0.02697595 - time (sec): 148.40 - samples/sec: 7891.21 - lr: 0.025000 2023-04-06 11:52:07,661 epoch 128 - iter 2385/2650 - loss 0.02695683 - time (sec): 166.82 - samples/sec: 7898.71 - lr: 0.025000 2023-04-06 11:52:25,603 epoch 128 - iter 2650/2650 - loss 0.02683529 - time (sec): 184.76 - samples/sec: 7914.81 - lr: 0.025000 2023-04-06 11:52:25,603 ---------------------------------------------------------------------------------------------------- 2023-04-06 11:52:25,603 EPOCH 128 done: loss 0.0268 - lr 0.025000 2023-04-06 11:52:25,603 BAD EPOCHS (no improvement): 2 2023-04-06 11:52:25,607 ---------------------------------------------------------------------------------------------------- 2023-04-06 11:52:43,938 epoch 129 - iter 265/2650 - loss 0.02687912 - time (sec): 18.33 - samples/sec: 8022.92 - lr: 0.025000 2023-04-06 11:53:02,223 epoch 129 - iter 530/2650 - loss 0.02628566 - time (sec): 36.62 - samples/sec: 8072.60 - lr: 0.025000 2023-04-06 11:53:20,168 epoch 129 - iter 795/2650 - loss 0.02617875 - time (sec): 54.56 - samples/sec: 8098.83 - lr: 0.025000 2023-04-06 11:53:38,596 epoch 129 - iter 1060/2650 - loss 0.02597046 - time (sec): 72.99 - samples/sec: 8083.04 - lr: 0.025000 2023-04-06 11:53:56,575 epoch 129 - iter 1325/2650 - loss 0.02620927 - time (sec): 90.97 - samples/sec: 8081.21 - lr: 0.025000 2023-04-06 11:54:14,577 epoch 129 - iter 1590/2650 - loss 0.02624410 - time (sec): 108.97 - samples/sec: 8078.12 - lr: 0.025000 2023-04-06 11:54:32,804 epoch 129 - iter 1855/2650 - loss 0.02632753 - time (sec): 127.20 - samples/sec: 8052.80 - lr: 0.025000 2023-04-06 11:54:50,936 epoch 129 - iter 2120/2650 - loss 0.02649055 - time (sec): 145.33 - samples/sec: 8051.04 - lr: 0.025000 2023-04-06 11:55:09,478 epoch 129 - iter 2385/2650 - loss 0.02661060 - time (sec): 163.87 - samples/sec: 8030.80 - lr: 0.025000 2023-04-06 11:55:27,833 epoch 129 - iter 2650/2650 - loss 0.02674456 - time (sec): 182.23 - samples/sec: 8024.97 - lr: 0.025000 2023-04-06 11:55:27,834 ---------------------------------------------------------------------------------------------------- 2023-04-06 11:55:27,834 EPOCH 129 done: loss 0.0267 - lr 0.025000 2023-04-06 11:55:27,834 BAD EPOCHS (no improvement): 0 2023-04-06 11:55:27,837 ---------------------------------------------------------------------------------------------------- 2023-04-06 11:55:46,059 epoch 130 - iter 265/2650 - loss 0.02631793 - time (sec): 18.22 - samples/sec: 8044.34 - lr: 0.025000 2023-04-06 11:56:04,740 epoch 130 - iter 530/2650 - loss 0.02633669 - time (sec): 36.90 - samples/sec: 7963.90 - lr: 0.025000 2023-04-06 11:56:22,908 epoch 130 - iter 795/2650 - loss 0.02639026 - time (sec): 55.07 - samples/sec: 8002.81 - lr: 0.025000 2023-04-06 11:56:40,519 epoch 130 - iter 1060/2650 - loss 0.02648246 - time (sec): 72.68 - samples/sec: 8054.87 - lr: 0.025000 2023-04-06 11:56:58,650 epoch 130 - iter 1325/2650 - loss 0.02658863 - time (sec): 90.81 - samples/sec: 8061.52 - lr: 0.025000 2023-04-06 11:57:16,760 epoch 130 - iter 1590/2650 - loss 0.02656250 - time (sec): 108.92 - samples/sec: 8053.25 - lr: 0.025000 2023-04-06 11:57:35,022 epoch 130 - iter 1855/2650 - loss 0.02664878 - time (sec): 127.18 - samples/sec: 8047.79 - lr: 0.025000 2023-04-06 11:57:53,230 epoch 130 - iter 2120/2650 - loss 0.02665013 - time (sec): 145.39 - samples/sec: 8046.64 - lr: 0.025000 2023-04-06 11:58:11,477 epoch 130 - iter 2385/2650 - loss 0.02658548 - time (sec): 163.64 - samples/sec: 8039.32 - lr: 0.025000 2023-04-06 11:58:29,714 epoch 130 - iter 2650/2650 - loss 0.02661644 - time (sec): 181.88 - samples/sec: 8040.39 - lr: 0.025000 2023-04-06 11:58:29,714 ---------------------------------------------------------------------------------------------------- 2023-04-06 11:58:29,715 EPOCH 130 done: loss 0.0266 - lr 0.025000 2023-04-06 11:58:29,715 BAD EPOCHS (no improvement): 0 2023-04-06 11:58:29,718 ---------------------------------------------------------------------------------------------------- 2023-04-06 11:58:48,477 epoch 131 - iter 265/2650 - loss 0.02614240 - time (sec): 18.76 - samples/sec: 7877.69 - lr: 0.025000 2023-04-06 11:59:06,710 epoch 131 - iter 530/2650 - loss 0.02672166 - time (sec): 36.99 - samples/sec: 7952.13 - lr: 0.025000 2023-04-06 11:59:24,852 epoch 131 - iter 795/2650 - loss 0.02652352 - time (sec): 55.13 - samples/sec: 7996.54 - lr: 0.025000 2023-04-06 11:59:42,948 epoch 131 - iter 1060/2650 - loss 0.02645522 - time (sec): 73.23 - samples/sec: 8009.04 - lr: 0.025000 2023-04-06 12:00:01,139 epoch 131 - iter 1325/2650 - loss 0.02650858 - time (sec): 91.42 - samples/sec: 8012.99 - lr: 0.025000 2023-04-06 12:00:19,180 epoch 131 - iter 1590/2650 - loss 0.02670878 - time (sec): 109.46 - samples/sec: 8034.70 - lr: 0.025000 2023-04-06 12:00:37,162 epoch 131 - iter 1855/2650 - loss 0.02682133 - time (sec): 127.44 - samples/sec: 8040.47 - lr: 0.025000 2023-04-06 12:01:06,658 epoch 131 - iter 2120/2650 - loss 0.02678901 - time (sec): 156.94 - samples/sec: 7457.52 - lr: 0.025000 2023-04-06 12:01:24,861 epoch 131 - iter 2385/2650 - loss 0.02685845 - time (sec): 175.14 - samples/sec: 7522.04 - lr: 0.025000 2023-04-06 12:01:42,960 epoch 131 - iter 2650/2650 - loss 0.02692666 - time (sec): 193.24 - samples/sec: 7567.54 - lr: 0.025000 2023-04-06 12:01:42,960 ---------------------------------------------------------------------------------------------------- 2023-04-06 12:01:42,960 EPOCH 131 done: loss 0.0269 - lr 0.025000 2023-04-06 12:01:42,960 BAD EPOCHS (no improvement): 1 2023-04-06 12:01:42,964 ---------------------------------------------------------------------------------------------------- 2023-04-06 12:02:01,216 epoch 132 - iter 265/2650 - loss 0.02598831 - time (sec): 18.25 - samples/sec: 8013.33 - lr: 0.025000 2023-04-06 12:02:19,382 epoch 132 - iter 530/2650 - loss 0.02658575 - time (sec): 36.42 - samples/sec: 7991.91 - lr: 0.025000 2023-04-06 12:02:37,933 epoch 132 - iter 795/2650 - loss 0.02624271 - time (sec): 54.97 - samples/sec: 7955.80 - lr: 0.025000 2023-04-06 12:02:55,694 epoch 132 - iter 1060/2650 - loss 0.02629874 - time (sec): 72.73 - samples/sec: 8011.12 - lr: 0.025000 2023-04-06 12:03:13,990 epoch 132 - iter 1325/2650 - loss 0.02634924 - time (sec): 91.03 - samples/sec: 8010.20 - lr: 0.025000 2023-04-06 12:03:32,714 epoch 132 - iter 1590/2650 - loss 0.02639424 - time (sec): 109.75 - samples/sec: 7972.37 - lr: 0.025000 2023-04-06 12:03:52,513 epoch 132 - iter 1855/2650 - loss 0.02670428 - time (sec): 129.55 - samples/sec: 7893.16 - lr: 0.025000 2023-04-06 12:04:11,056 epoch 132 - iter 2120/2650 - loss 0.02677724 - time (sec): 148.09 - samples/sec: 7894.44 - lr: 0.025000 2023-04-06 12:04:32,041 epoch 132 - iter 2385/2650 - loss 0.02666199 - time (sec): 169.08 - samples/sec: 7780.83 - lr: 0.025000 2023-04-06 12:04:53,783 epoch 132 - iter 2650/2650 - loss 0.02672938 - time (sec): 190.82 - samples/sec: 7663.58 - lr: 0.025000 2023-04-06 12:04:53,784 ---------------------------------------------------------------------------------------------------- 2023-04-06 12:04:53,784 EPOCH 132 done: loss 0.0267 - lr 0.025000 2023-04-06 12:04:53,784 BAD EPOCHS (no improvement): 2 2023-04-06 12:04:53,788 ---------------------------------------------------------------------------------------------------- 2023-04-06 12:05:14,250 epoch 133 - iter 265/2650 - loss 0.02525971 - time (sec): 20.46 - samples/sec: 7224.75 - lr: 0.025000 2023-04-06 12:05:32,793 epoch 133 - iter 530/2650 - loss 0.02576893 - time (sec): 39.01 - samples/sec: 7530.88 - lr: 0.025000 2023-04-06 12:05:51,531 epoch 133 - iter 795/2650 - loss 0.02598691 - time (sec): 57.74 - samples/sec: 7634.84 - lr: 0.025000 2023-04-06 12:06:10,372 epoch 133 - iter 1060/2650 - loss 0.02597871 - time (sec): 76.58 - samples/sec: 7658.34 - lr: 0.025000 2023-04-06 12:06:33,767 epoch 133 - iter 1325/2650 - loss 0.02588522 - time (sec): 99.98 - samples/sec: 7324.80 - lr: 0.025000 2023-04-06 12:06:54,099 epoch 133 - iter 1590/2650 - loss 0.02609380 - time (sec): 120.31 - samples/sec: 7294.31 - lr: 0.025000 2023-04-06 12:07:15,387 epoch 133 - iter 1855/2650 - loss 0.02619674 - time (sec): 141.60 - samples/sec: 7235.94 - lr: 0.025000 2023-04-06 12:07:34,456 epoch 133 - iter 2120/2650 - loss 0.02613559 - time (sec): 160.67 - samples/sec: 7289.31 - lr: 0.025000 2023-04-06 12:07:54,003 epoch 133 - iter 2385/2650 - loss 0.02619411 - time (sec): 180.22 - samples/sec: 7299.96 - lr: 0.025000 2023-04-06 12:08:15,599 epoch 133 - iter 2650/2650 - loss 0.02634392 - time (sec): 201.81 - samples/sec: 7246.20 - lr: 0.025000 2023-04-06 12:08:15,599 ---------------------------------------------------------------------------------------------------- 2023-04-06 12:08:15,599 EPOCH 133 done: loss 0.0263 - lr 0.025000 2023-04-06 12:08:15,599 BAD EPOCHS (no improvement): 0 2023-04-06 12:08:15,601 ---------------------------------------------------------------------------------------------------- 2023-04-06 12:08:35,053 epoch 134 - iter 265/2650 - loss 0.02713773 - time (sec): 19.45 - samples/sec: 7499.05 - lr: 0.025000 2023-04-06 12:08:56,403 epoch 134 - iter 530/2650 - loss 0.02649932 - time (sec): 40.80 - samples/sec: 7175.59 - lr: 0.025000 2023-04-06 12:09:14,521 epoch 134 - iter 795/2650 - loss 0.02631503 - time (sec): 58.92 - samples/sec: 7456.69 - lr: 0.025000 2023-04-06 12:09:32,613 epoch 134 - iter 1060/2650 - loss 0.02661798 - time (sec): 77.01 - samples/sec: 7590.06 - lr: 0.025000 2023-04-06 12:09:51,053 epoch 134 - iter 1325/2650 - loss 0.02648903 - time (sec): 95.45 - samples/sec: 7674.96 - lr: 0.025000 2023-04-06 12:10:09,228 epoch 134 - iter 1590/2650 - loss 0.02640788 - time (sec): 113.63 - samples/sec: 7732.05 - lr: 0.025000 2023-04-06 12:10:27,332 epoch 134 - iter 1855/2650 - loss 0.02650255 - time (sec): 131.73 - samples/sec: 7770.43 - lr: 0.025000 2023-04-06 12:10:45,533 epoch 134 - iter 2120/2650 - loss 0.02657328 - time (sec): 149.93 - samples/sec: 7804.63 - lr: 0.025000 2023-04-06 12:11:03,789 epoch 134 - iter 2385/2650 - loss 0.02655913 - time (sec): 168.19 - samples/sec: 7832.43 - lr: 0.025000 2023-04-06 12:11:21,731 epoch 134 - iter 2650/2650 - loss 0.02652006 - time (sec): 186.13 - samples/sec: 7856.69 - lr: 0.025000 2023-04-06 12:11:21,731 ---------------------------------------------------------------------------------------------------- 2023-04-06 12:11:21,731 EPOCH 134 done: loss 0.0265 - lr 0.025000 2023-04-06 12:11:21,731 BAD EPOCHS (no improvement): 1 2023-04-06 12:11:21,734 ---------------------------------------------------------------------------------------------------- 2023-04-06 12:11:39,900 epoch 135 - iter 265/2650 - loss 0.02612829 - time (sec): 18.17 - samples/sec: 7992.07 - lr: 0.025000 2023-04-06 12:11:58,989 epoch 135 - iter 530/2650 - loss 0.02582586 - time (sec): 37.25 - samples/sec: 7866.04 - lr: 0.025000 2023-04-06 12:12:17,240 epoch 135 - iter 795/2650 - loss 0.02611304 - time (sec): 55.51 - samples/sec: 7901.18 - lr: 0.025000 2023-04-06 12:12:35,480 epoch 135 - iter 1060/2650 - loss 0.02624228 - time (sec): 73.75 - samples/sec: 7930.16 - lr: 0.025000 2023-04-06 12:12:54,131 epoch 135 - iter 1325/2650 - loss 0.02617346 - time (sec): 92.40 - samples/sec: 7918.51 - lr: 0.025000 2023-04-06 12:13:12,146 epoch 135 - iter 1590/2650 - loss 0.02628733 - time (sec): 110.41 - samples/sec: 7928.31 - lr: 0.025000 2023-04-06 12:13:30,766 epoch 135 - iter 1855/2650 - loss 0.02638516 - time (sec): 129.03 - samples/sec: 7925.20 - lr: 0.025000 2023-04-06 12:13:49,018 epoch 135 - iter 2120/2650 - loss 0.02638967 - time (sec): 147.28 - samples/sec: 7941.33 - lr: 0.025000 2023-04-06 12:14:07,449 epoch 135 - iter 2385/2650 - loss 0.02642862 - time (sec): 165.71 - samples/sec: 7947.36 - lr: 0.025000 2023-04-06 12:14:25,485 epoch 135 - iter 2650/2650 - loss 0.02653396 - time (sec): 183.75 - samples/sec: 7958.38 - lr: 0.025000 2023-04-06 12:14:25,486 ---------------------------------------------------------------------------------------------------- 2023-04-06 12:14:25,486 EPOCH 135 done: loss 0.0265 - lr 0.025000 2023-04-06 12:14:25,486 BAD EPOCHS (no improvement): 2 2023-04-06 12:14:25,488 ---------------------------------------------------------------------------------------------------- 2023-04-06 12:14:43,942 epoch 136 - iter 265/2650 - loss 0.02498349 - time (sec): 18.45 - samples/sec: 7999.41 - lr: 0.025000 2023-04-06 12:15:02,452 epoch 136 - iter 530/2650 - loss 0.02567027 - time (sec): 36.96 - samples/sec: 7938.24 - lr: 0.025000 2023-04-06 12:15:20,402 epoch 136 - iter 795/2650 - loss 0.02587434 - time (sec): 54.91 - samples/sec: 7974.77 - lr: 0.025000 2023-04-06 12:15:38,988 epoch 136 - iter 1060/2650 - loss 0.02648290 - time (sec): 73.50 - samples/sec: 7961.46 - lr: 0.025000 2023-04-06 12:15:57,070 epoch 136 - iter 1325/2650 - loss 0.02657625 - time (sec): 91.58 - samples/sec: 7987.19 - lr: 0.025000 2023-04-06 12:16:15,261 epoch 136 - iter 1590/2650 - loss 0.02674891 - time (sec): 109.77 - samples/sec: 7978.21 - lr: 0.025000 2023-04-06 12:16:33,349 epoch 136 - iter 1855/2650 - loss 0.02666881 - time (sec): 127.86 - samples/sec: 7992.92 - lr: 0.025000 2023-04-06 12:16:51,785 epoch 136 - iter 2120/2650 - loss 0.02668984 - time (sec): 146.30 - samples/sec: 7989.84 - lr: 0.025000 2023-04-06 12:17:10,594 epoch 136 - iter 2385/2650 - loss 0.02662975 - time (sec): 165.11 - samples/sec: 7965.28 - lr: 0.025000 2023-04-06 12:17:29,202 epoch 136 - iter 2650/2650 - loss 0.02660779 - time (sec): 183.71 - samples/sec: 7960.02 - lr: 0.025000 2023-04-06 12:17:29,202 ---------------------------------------------------------------------------------------------------- 2023-04-06 12:17:29,202 EPOCH 136 done: loss 0.0266 - lr 0.025000 2023-04-06 12:17:29,202 BAD EPOCHS (no improvement): 3 2023-04-06 12:17:29,205 ---------------------------------------------------------------------------------------------------- 2023-04-06 12:17:47,391 epoch 137 - iter 265/2650 - loss 0.02745568 - time (sec): 18.19 - samples/sec: 7952.31 - lr: 0.025000 2023-04-06 12:18:05,879 epoch 137 - iter 530/2650 - loss 0.02703099 - time (sec): 36.67 - samples/sec: 7997.30 - lr: 0.025000 2023-04-06 12:18:23,901 epoch 137 - iter 795/2650 - loss 0.02671024 - time (sec): 54.70 - samples/sec: 8002.57 - lr: 0.025000 2023-04-06 12:18:42,278 epoch 137 - iter 1060/2650 - loss 0.02639169 - time (sec): 73.07 - samples/sec: 8010.52 - lr: 0.025000 2023-04-06 12:19:00,774 epoch 137 - iter 1325/2650 - loss 0.02648607 - time (sec): 91.57 - samples/sec: 7992.20 - lr: 0.025000 2023-04-06 12:19:19,519 epoch 137 - iter 1590/2650 - loss 0.02644318 - time (sec): 110.31 - samples/sec: 7954.93 - lr: 0.025000 2023-04-06 12:19:38,376 epoch 137 - iter 1855/2650 - loss 0.02631138 - time (sec): 129.17 - samples/sec: 7932.20 - lr: 0.025000 2023-04-06 12:19:56,502 epoch 137 - iter 2120/2650 - loss 0.02631271 - time (sec): 147.30 - samples/sec: 7944.78 - lr: 0.025000 2023-04-06 12:20:14,502 epoch 137 - iter 2385/2650 - loss 0.02625973 - time (sec): 165.30 - samples/sec: 7960.34 - lr: 0.025000 2023-04-06 12:20:32,659 epoch 137 - iter 2650/2650 - loss 0.02617706 - time (sec): 183.45 - samples/sec: 7971.27 - lr: 0.025000 2023-04-06 12:20:32,659 ---------------------------------------------------------------------------------------------------- 2023-04-06 12:20:32,659 EPOCH 137 done: loss 0.0262 - lr 0.025000 2023-04-06 12:20:32,659 BAD EPOCHS (no improvement): 0 2023-04-06 12:20:32,662 ---------------------------------------------------------------------------------------------------- 2023-04-06 12:20:50,889 epoch 138 - iter 265/2650 - loss 0.02762308 - time (sec): 18.23 - samples/sec: 7993.28 - lr: 0.025000 2023-04-06 12:21:09,099 epoch 138 - iter 530/2650 - loss 0.02669495 - time (sec): 36.44 - samples/sec: 7985.56 - lr: 0.025000 2023-04-06 12:21:27,248 epoch 138 - iter 795/2650 - loss 0.02647993 - time (sec): 54.59 - samples/sec: 8019.26 - lr: 0.025000 2023-04-06 12:21:45,096 epoch 138 - iter 1060/2650 - loss 0.02649977 - time (sec): 72.43 - samples/sec: 8055.88 - lr: 0.025000 2023-04-06 12:22:03,389 epoch 138 - iter 1325/2650 - loss 0.02645581 - time (sec): 90.73 - samples/sec: 8047.39 - lr: 0.025000 2023-04-06 12:22:21,505 epoch 138 - iter 1590/2650 - loss 0.02641312 - time (sec): 108.84 - samples/sec: 8057.70 - lr: 0.025000 2023-04-06 12:22:39,110 epoch 138 - iter 1855/2650 - loss 0.02635407 - time (sec): 126.45 - samples/sec: 8082.90 - lr: 0.025000 2023-04-06 12:22:57,176 epoch 138 - iter 2120/2650 - loss 0.02635989 - time (sec): 144.51 - samples/sec: 8079.50 - lr: 0.025000 2023-04-06 12:23:15,574 epoch 138 - iter 2385/2650 - loss 0.02643411 - time (sec): 162.91 - samples/sec: 8075.08 - lr: 0.025000 2023-04-06 12:23:34,043 epoch 138 - iter 2650/2650 - loss 0.02643150 - time (sec): 181.38 - samples/sec: 8062.40 - lr: 0.025000 2023-04-06 12:23:34,043 ---------------------------------------------------------------------------------------------------- 2023-04-06 12:23:34,043 EPOCH 138 done: loss 0.0264 - lr 0.025000 2023-04-06 12:23:34,043 BAD EPOCHS (no improvement): 1 2023-04-06 12:23:34,047 ---------------------------------------------------------------------------------------------------- 2023-04-06 12:23:52,255 epoch 139 - iter 265/2650 - loss 0.02599257 - time (sec): 18.21 - samples/sec: 8022.87 - lr: 0.025000 2023-04-06 12:24:10,475 epoch 139 - iter 530/2650 - loss 0.02591115 - time (sec): 36.43 - samples/sec: 8012.10 - lr: 0.025000 2023-04-06 12:24:28,660 epoch 139 - iter 795/2650 - loss 0.02598178 - time (sec): 54.61 - samples/sec: 8041.63 - lr: 0.025000 2023-04-06 12:24:46,332 epoch 139 - iter 1060/2650 - loss 0.02599237 - time (sec): 72.29 - samples/sec: 8080.42 - lr: 0.025000 2023-04-06 12:25:04,468 epoch 139 - iter 1325/2650 - loss 0.02584830 - time (sec): 90.42 - samples/sec: 8078.17 - lr: 0.025000 2023-04-06 12:25:22,351 epoch 139 - iter 1590/2650 - loss 0.02601809 - time (sec): 108.30 - samples/sec: 8101.04 - lr: 0.025000 2023-04-06 12:25:40,547 epoch 139 - iter 1855/2650 - loss 0.02599118 - time (sec): 126.50 - samples/sec: 8089.06 - lr: 0.025000 2023-04-06 12:25:58,955 epoch 139 - iter 2120/2650 - loss 0.02596837 - time (sec): 144.91 - samples/sec: 8069.80 - lr: 0.025000 2023-04-06 12:26:17,078 epoch 139 - iter 2385/2650 - loss 0.02581227 - time (sec): 163.03 - samples/sec: 8078.18 - lr: 0.025000 2023-04-06 12:26:35,293 epoch 139 - iter 2650/2650 - loss 0.02590218 - time (sec): 181.25 - samples/sec: 8068.36 - lr: 0.025000 2023-04-06 12:26:35,293 ---------------------------------------------------------------------------------------------------- 2023-04-06 12:26:35,293 EPOCH 139 done: loss 0.0259 - lr 0.025000 2023-04-06 12:26:35,294 BAD EPOCHS (no improvement): 0 2023-04-06 12:26:35,296 ---------------------------------------------------------------------------------------------------- 2023-04-06 12:26:53,046 epoch 140 - iter 265/2650 - loss 0.02557386 - time (sec): 17.75 - samples/sec: 8181.44 - lr: 0.025000 2023-04-06 12:27:11,224 epoch 140 - iter 530/2650 - loss 0.02614134 - time (sec): 35.93 - samples/sec: 8131.87 - lr: 0.025000 2023-04-06 12:27:29,229 epoch 140 - iter 795/2650 - loss 0.02597963 - time (sec): 53.93 - samples/sec: 8133.59 - lr: 0.025000 2023-04-06 12:27:47,165 epoch 140 - iter 1060/2650 - loss 0.02623025 - time (sec): 71.87 - samples/sec: 8118.36 - lr: 0.025000 2023-04-06 12:28:05,336 epoch 140 - iter 1325/2650 - loss 0.02623403 - time (sec): 90.04 - samples/sec: 8110.46 - lr: 0.025000 2023-04-06 12:28:23,455 epoch 140 - iter 1590/2650 - loss 0.02624457 - time (sec): 108.16 - samples/sec: 8115.46 - lr: 0.025000 2023-04-06 12:28:41,255 epoch 140 - iter 1855/2650 - loss 0.02631689 - time (sec): 125.96 - samples/sec: 8118.99 - lr: 0.025000 2023-04-06 12:28:59,625 epoch 140 - iter 2120/2650 - loss 0.02629473 - time (sec): 144.33 - samples/sec: 8101.08 - lr: 0.025000 2023-04-06 12:29:17,828 epoch 140 - iter 2385/2650 - loss 0.02629073 - time (sec): 162.53 - samples/sec: 8091.34 - lr: 0.025000 2023-04-06 12:29:36,439 epoch 140 - iter 2650/2650 - loss 0.02624869 - time (sec): 181.14 - samples/sec: 8072.99 - lr: 0.025000 2023-04-06 12:29:36,439 ---------------------------------------------------------------------------------------------------- 2023-04-06 12:29:36,439 EPOCH 140 done: loss 0.0262 - lr 0.025000 2023-04-06 12:29:36,440 BAD EPOCHS (no improvement): 1 2023-04-06 12:29:36,443 ---------------------------------------------------------------------------------------------------- 2023-04-06 12:29:54,296 epoch 141 - iter 265/2650 - loss 0.02576876 - time (sec): 17.85 - samples/sec: 8172.48 - lr: 0.025000 2023-04-06 12:30:12,433 epoch 141 - iter 530/2650 - loss 0.02641912 - time (sec): 35.99 - samples/sec: 8147.04 - lr: 0.025000 2023-04-06 12:30:30,886 epoch 141 - iter 795/2650 - loss 0.02625009 - time (sec): 54.44 - samples/sec: 8112.08 - lr: 0.025000 2023-04-06 12:30:49,055 epoch 141 - iter 1060/2650 - loss 0.02639039 - time (sec): 72.61 - samples/sec: 8092.87 - lr: 0.025000 2023-04-06 12:31:06,894 epoch 141 - iter 1325/2650 - loss 0.02632833 - time (sec): 90.45 - samples/sec: 8103.07 - lr: 0.025000 2023-04-06 12:31:25,031 epoch 141 - iter 1590/2650 - loss 0.02634991 - time (sec): 108.59 - samples/sec: 8088.87 - lr: 0.025000 2023-04-06 12:31:43,186 epoch 141 - iter 1855/2650 - loss 0.02632427 - time (sec): 126.74 - samples/sec: 8086.85 - lr: 0.025000 2023-04-06 12:32:01,520 epoch 141 - iter 2120/2650 - loss 0.02633151 - time (sec): 145.08 - samples/sec: 8078.31 - lr: 0.025000 2023-04-06 12:32:19,655 epoch 141 - iter 2385/2650 - loss 0.02631504 - time (sec): 163.21 - samples/sec: 8076.70 - lr: 0.025000 2023-04-06 12:32:37,460 epoch 141 - iter 2650/2650 - loss 0.02629694 - time (sec): 181.02 - samples/sec: 8078.62 - lr: 0.025000 2023-04-06 12:32:37,460 ---------------------------------------------------------------------------------------------------- 2023-04-06 12:32:37,460 EPOCH 141 done: loss 0.0263 - lr 0.025000 2023-04-06 12:32:37,460 BAD EPOCHS (no improvement): 2 2023-04-06 12:32:37,463 ---------------------------------------------------------------------------------------------------- 2023-04-06 12:32:55,682 epoch 142 - iter 265/2650 - loss 0.02572878 - time (sec): 18.22 - samples/sec: 8056.34 - lr: 0.025000 2023-04-06 12:33:13,456 epoch 142 - iter 530/2650 - loss 0.02644122 - time (sec): 35.99 - samples/sec: 8087.58 - lr: 0.025000 2023-04-06 12:33:31,292 epoch 142 - iter 795/2650 - loss 0.02633799 - time (sec): 53.83 - samples/sec: 8127.38 - lr: 0.025000 2023-04-06 12:33:49,546 epoch 142 - iter 1060/2650 - loss 0.02609464 - time (sec): 72.08 - samples/sec: 8113.34 - lr: 0.025000 2023-04-06 12:34:08,080 epoch 142 - iter 1325/2650 - loss 0.02609133 - time (sec): 90.62 - samples/sec: 8074.59 - lr: 0.025000 2023-04-06 12:34:26,330 epoch 142 - iter 1590/2650 - loss 0.02596919 - time (sec): 108.87 - samples/sec: 8052.31 - lr: 0.025000 2023-04-06 12:34:44,455 epoch 142 - iter 1855/2650 - loss 0.02594405 - time (sec): 126.99 - samples/sec: 8063.78 - lr: 0.025000 2023-04-06 12:35:02,959 epoch 142 - iter 2120/2650 - loss 0.02592427 - time (sec): 145.50 - samples/sec: 8046.76 - lr: 0.025000 2023-04-06 12:35:20,783 epoch 142 - iter 2385/2650 - loss 0.02596063 - time (sec): 163.32 - samples/sec: 8063.12 - lr: 0.025000 2023-04-06 12:35:39,014 epoch 142 - iter 2650/2650 - loss 0.02593919 - time (sec): 181.55 - samples/sec: 8054.82 - lr: 0.025000 2023-04-06 12:35:39,014 ---------------------------------------------------------------------------------------------------- 2023-04-06 12:35:39,014 EPOCH 142 done: loss 0.0259 - lr 0.025000 2023-04-06 12:35:39,015 BAD EPOCHS (no improvement): 3 2023-04-06 12:35:39,017 ---------------------------------------------------------------------------------------------------- 2023-04-06 12:35:57,595 epoch 143 - iter 265/2650 - loss 0.02539110 - time (sec): 18.58 - samples/sec: 7973.31 - lr: 0.025000 2023-04-06 12:36:16,112 epoch 143 - iter 530/2650 - loss 0.02620968 - time (sec): 37.09 - samples/sec: 7910.08 - lr: 0.025000 2023-04-06 12:36:34,480 epoch 143 - iter 795/2650 - loss 0.02609584 - time (sec): 55.46 - samples/sec: 7929.08 - lr: 0.025000 2023-04-06 12:36:53,036 epoch 143 - iter 1060/2650 - loss 0.02607727 - time (sec): 74.02 - samples/sec: 7928.99 - lr: 0.025000 2023-04-06 12:37:11,762 epoch 143 - iter 1325/2650 - loss 0.02602813 - time (sec): 92.74 - samples/sec: 7924.29 - lr: 0.025000 2023-04-06 12:37:30,152 epoch 143 - iter 1590/2650 - loss 0.02604743 - time (sec): 111.13 - samples/sec: 7917.65 - lr: 0.025000 2023-04-06 12:37:48,108 epoch 143 - iter 1855/2650 - loss 0.02606550 - time (sec): 129.09 - samples/sec: 7926.15 - lr: 0.025000 2023-04-06 12:38:06,874 epoch 143 - iter 2120/2650 - loss 0.02602514 - time (sec): 147.86 - samples/sec: 7922.50 - lr: 0.025000 2023-04-06 12:38:25,059 epoch 143 - iter 2385/2650 - loss 0.02610859 - time (sec): 166.04 - samples/sec: 7933.21 - lr: 0.025000 2023-04-06 12:38:43,665 epoch 143 - iter 2650/2650 - loss 0.02611900 - time (sec): 184.65 - samples/sec: 7919.76 - lr: 0.025000 2023-04-06 12:38:43,665 ---------------------------------------------------------------------------------------------------- 2023-04-06 12:38:43,665 EPOCH 143 done: loss 0.0261 - lr 0.025000 2023-04-06 12:38:43,665 Epoch 143: reducing learning rate of group 0 to 1.2500e-02. 2023-04-06 12:38:43,665 BAD EPOCHS (no improvement): 4 2023-04-06 12:38:43,669 ---------------------------------------------------------------------------------------------------- 2023-04-06 12:39:02,953 epoch 144 - iter 265/2650 - loss 0.02682733 - time (sec): 19.28 - samples/sec: 7649.51 - lr: 0.012500 2023-04-06 12:39:21,174 epoch 144 - iter 530/2650 - loss 0.02614750 - time (sec): 37.50 - samples/sec: 7846.76 - lr: 0.012500 2023-04-06 12:39:39,186 epoch 144 - iter 795/2650 - loss 0.02558707 - time (sec): 55.52 - samples/sec: 7883.54 - lr: 0.012500 2023-04-06 12:39:57,735 epoch 144 - iter 1060/2650 - loss 0.02531073 - time (sec): 74.07 - samples/sec: 7877.27 - lr: 0.012500 2023-04-06 12:40:16,066 epoch 144 - iter 1325/2650 - loss 0.02556399 - time (sec): 92.40 - samples/sec: 7905.99 - lr: 0.012500 2023-04-06 12:40:34,297 epoch 144 - iter 1590/2650 - loss 0.02557806 - time (sec): 110.63 - samples/sec: 7926.18 - lr: 0.012500 2023-04-06 12:40:52,532 epoch 144 - iter 1855/2650 - loss 0.02570859 - time (sec): 128.86 - samples/sec: 7944.19 - lr: 0.012500 2023-04-06 12:41:11,053 epoch 144 - iter 2120/2650 - loss 0.02565045 - time (sec): 147.38 - samples/sec: 7949.47 - lr: 0.012500 2023-04-06 12:41:29,077 epoch 144 - iter 2385/2650 - loss 0.02569423 - time (sec): 165.41 - samples/sec: 7962.14 - lr: 0.012500 2023-04-06 12:41:47,200 epoch 144 - iter 2650/2650 - loss 0.02560617 - time (sec): 183.53 - samples/sec: 7967.93 - lr: 0.012500 2023-04-06 12:41:47,200 ---------------------------------------------------------------------------------------------------- 2023-04-06 12:41:47,200 EPOCH 144 done: loss 0.0256 - lr 0.012500 2023-04-06 12:41:47,200 BAD EPOCHS (no improvement): 0 2023-04-06 12:41:47,203 ---------------------------------------------------------------------------------------------------- 2023-04-06 12:42:05,502 epoch 145 - iter 265/2650 - loss 0.02561083 - time (sec): 18.30 - samples/sec: 8023.43 - lr: 0.012500 2023-04-06 12:42:23,639 epoch 145 - iter 530/2650 - loss 0.02543605 - time (sec): 36.44 - samples/sec: 8017.48 - lr: 0.012500 2023-04-06 12:42:41,746 epoch 145 - iter 795/2650 - loss 0.02578382 - time (sec): 54.54 - samples/sec: 8022.21 - lr: 0.012500 2023-04-06 12:43:00,048 epoch 145 - iter 1060/2650 - loss 0.02564606 - time (sec): 72.84 - samples/sec: 8017.66 - lr: 0.012500 2023-04-06 12:43:18,076 epoch 145 - iter 1325/2650 - loss 0.02556002 - time (sec): 90.87 - samples/sec: 8029.91 - lr: 0.012500 2023-04-06 12:43:36,114 epoch 145 - iter 1590/2650 - loss 0.02563451 - time (sec): 108.91 - samples/sec: 8041.33 - lr: 0.012500 2023-04-06 12:43:54,630 epoch 145 - iter 1855/2650 - loss 0.02554716 - time (sec): 127.43 - samples/sec: 8023.66 - lr: 0.012500 2023-04-06 12:44:12,765 epoch 145 - iter 2120/2650 - loss 0.02549357 - time (sec): 145.56 - samples/sec: 8031.98 - lr: 0.012500 2023-04-06 12:44:31,061 epoch 145 - iter 2385/2650 - loss 0.02548577 - time (sec): 163.86 - samples/sec: 8036.89 - lr: 0.012500 2023-04-06 12:44:48,980 epoch 145 - iter 2650/2650 - loss 0.02552889 - time (sec): 181.78 - samples/sec: 8044.86 - lr: 0.012500 2023-04-06 12:44:48,980 ---------------------------------------------------------------------------------------------------- 2023-04-06 12:44:48,980 EPOCH 145 done: loss 0.0255 - lr 0.012500 2023-04-06 12:44:48,980 BAD EPOCHS (no improvement): 0 2023-04-06 12:44:48,983 ---------------------------------------------------------------------------------------------------- 2023-04-06 12:45:07,260 epoch 146 - iter 265/2650 - loss 0.02478088 - time (sec): 18.28 - samples/sec: 8045.98 - lr: 0.012500 2023-04-06 12:45:26,514 epoch 146 - iter 530/2650 - loss 0.02458594 - time (sec): 37.53 - samples/sec: 7788.86 - lr: 0.012500 2023-04-06 12:45:44,776 epoch 146 - iter 795/2650 - loss 0.02508700 - time (sec): 55.79 - samples/sec: 7844.08 - lr: 0.012500 2023-04-06 12:46:03,074 epoch 146 - iter 1060/2650 - loss 0.02536088 - time (sec): 74.09 - samples/sec: 7892.06 - lr: 0.012500 2023-04-06 12:46:21,346 epoch 146 - iter 1325/2650 - loss 0.02528176 - time (sec): 92.36 - samples/sec: 7918.98 - lr: 0.012500 2023-04-06 12:46:39,628 epoch 146 - iter 1590/2650 - loss 0.02523093 - time (sec): 110.64 - samples/sec: 7937.52 - lr: 0.012500 2023-04-06 12:46:57,630 epoch 146 - iter 1855/2650 - loss 0.02525472 - time (sec): 128.65 - samples/sec: 7970.41 - lr: 0.012500 2023-04-06 12:47:16,206 epoch 146 - iter 2120/2650 - loss 0.02539008 - time (sec): 147.22 - samples/sec: 7963.88 - lr: 0.012500 2023-04-06 12:47:34,420 epoch 146 - iter 2385/2650 - loss 0.02538587 - time (sec): 165.44 - samples/sec: 7966.51 - lr: 0.012500 2023-04-06 12:47:52,485 epoch 146 - iter 2650/2650 - loss 0.02546001 - time (sec): 183.50 - samples/sec: 7969.20 - lr: 0.012500 2023-04-06 12:47:52,485 ---------------------------------------------------------------------------------------------------- 2023-04-06 12:47:52,485 EPOCH 146 done: loss 0.0255 - lr 0.012500 2023-04-06 12:47:52,486 BAD EPOCHS (no improvement): 0 2023-04-06 12:47:52,489 ---------------------------------------------------------------------------------------------------- 2023-04-06 12:48:11,108 epoch 147 - iter 265/2650 - loss 0.02531924 - time (sec): 18.62 - samples/sec: 7747.33 - lr: 0.012500 2023-04-06 12:48:29,650 epoch 147 - iter 530/2650 - loss 0.02503536 - time (sec): 37.16 - samples/sec: 7765.07 - lr: 0.012500 2023-04-06 12:48:48,330 epoch 147 - iter 795/2650 - loss 0.02506386 - time (sec): 55.84 - samples/sec: 7811.07 - lr: 0.012500 2023-04-06 12:49:07,250 epoch 147 - iter 1060/2650 - loss 0.02514493 - time (sec): 74.76 - samples/sec: 7809.45 - lr: 0.012500 2023-04-06 12:49:25,630 epoch 147 - iter 1325/2650 - loss 0.02532059 - time (sec): 93.14 - samples/sec: 7839.76 - lr: 0.012500 2023-04-06 12:49:44,139 epoch 147 - iter 1590/2650 - loss 0.02535167 - time (sec): 111.65 - samples/sec: 7861.54 - lr: 0.012500 2023-04-06 12:50:02,252 epoch 147 - iter 1855/2650 - loss 0.02533830 - time (sec): 129.76 - samples/sec: 7899.52 - lr: 0.012500 2023-04-06 12:50:20,286 epoch 147 - iter 2120/2650 - loss 0.02532892 - time (sec): 147.80 - samples/sec: 7922.95 - lr: 0.012500 2023-04-06 12:50:37,910 epoch 147 - iter 2385/2650 - loss 0.02520430 - time (sec): 165.42 - samples/sec: 7959.04 - lr: 0.012500 2023-04-06 12:50:56,225 epoch 147 - iter 2650/2650 - loss 0.02519070 - time (sec): 183.74 - samples/sec: 7959.05 - lr: 0.012500 2023-04-06 12:50:56,225 ---------------------------------------------------------------------------------------------------- 2023-04-06 12:50:56,225 EPOCH 147 done: loss 0.0252 - lr 0.012500 2023-04-06 12:50:56,225 BAD EPOCHS (no improvement): 0 2023-04-06 12:50:56,231 ---------------------------------------------------------------------------------------------------- 2023-04-06 12:51:14,660 epoch 148 - iter 265/2650 - loss 0.02402321 - time (sec): 18.43 - samples/sec: 7946.09 - lr: 0.012500 2023-04-06 12:51:33,027 epoch 148 - iter 530/2650 - loss 0.02449076 - time (sec): 36.80 - samples/sec: 7966.71 - lr: 0.012500 2023-04-06 12:51:51,179 epoch 148 - iter 795/2650 - loss 0.02445039 - time (sec): 54.95 - samples/sec: 7971.27 - lr: 0.012500 2023-04-06 12:52:09,252 epoch 148 - iter 1060/2650 - loss 0.02480921 - time (sec): 73.02 - samples/sec: 7978.55 - lr: 0.012500 2023-04-06 12:52:27,001 epoch 148 - iter 1325/2650 - loss 0.02501300 - time (sec): 90.77 - samples/sec: 8025.25 - lr: 0.012500 2023-04-06 12:52:45,271 epoch 148 - iter 1590/2650 - loss 0.02498001 - time (sec): 109.04 - samples/sec: 8027.80 - lr: 0.012500 2023-04-06 12:53:03,736 epoch 148 - iter 1855/2650 - loss 0.02502524 - time (sec): 127.50 - samples/sec: 8028.04 - lr: 0.012500 2023-04-06 12:53:21,803 epoch 148 - iter 2120/2650 - loss 0.02504741 - time (sec): 145.57 - samples/sec: 8051.68 - lr: 0.012500 2023-04-06 12:53:39,972 epoch 148 - iter 2385/2650 - loss 0.02501781 - time (sec): 163.74 - samples/sec: 8040.42 - lr: 0.012500 2023-04-06 12:53:58,108 epoch 148 - iter 2650/2650 - loss 0.02502567 - time (sec): 181.88 - samples/sec: 8040.42 - lr: 0.012500 2023-04-06 12:53:58,108 ---------------------------------------------------------------------------------------------------- 2023-04-06 12:53:58,108 EPOCH 148 done: loss 0.0250 - lr 0.012500 2023-04-06 12:53:58,108 BAD EPOCHS (no improvement): 0 2023-04-06 12:53:58,111 ---------------------------------------------------------------------------------------------------- 2023-04-06 12:54:16,743 epoch 149 - iter 265/2650 - loss 0.02467979 - time (sec): 18.63 - samples/sec: 7776.56 - lr: 0.012500 2023-04-06 12:54:35,322 epoch 149 - iter 530/2650 - loss 0.02458319 - time (sec): 37.21 - samples/sec: 7829.95 - lr: 0.012500 2023-04-06 12:54:53,894 epoch 149 - iter 795/2650 - loss 0.02498222 - time (sec): 55.78 - samples/sec: 7845.77 - lr: 0.012500 2023-04-06 12:55:12,637 epoch 149 - iter 1060/2650 - loss 0.02493237 - time (sec): 74.53 - samples/sec: 7846.93 - lr: 0.012500 2023-04-06 12:55:30,985 epoch 149 - iter 1325/2650 - loss 0.02483076 - time (sec): 92.87 - samples/sec: 7879.83 - lr: 0.012500 2023-04-06 12:55:49,182 epoch 149 - iter 1590/2650 - loss 0.02477192 - time (sec): 111.07 - samples/sec: 7898.63 - lr: 0.012500 2023-04-06 12:56:09,371 epoch 149 - iter 1855/2650 - loss 0.02474023 - time (sec): 131.26 - samples/sec: 7798.01 - lr: 0.012500 2023-04-06 12:56:29,260 epoch 149 - iter 2120/2650 - loss 0.02482734 - time (sec): 151.15 - samples/sec: 7737.99 - lr: 0.012500 2023-04-06 12:56:48,870 epoch 149 - iter 2385/2650 - loss 0.02483413 - time (sec): 170.76 - samples/sec: 7700.53 - lr: 0.012500 2023-04-06 12:57:07,776 epoch 149 - iter 2650/2650 - loss 0.02484855 - time (sec): 189.66 - samples/sec: 7710.26 - lr: 0.012500 2023-04-06 12:57:07,776 ---------------------------------------------------------------------------------------------------- 2023-04-06 12:57:07,776 EPOCH 149 done: loss 0.0248 - lr 0.012500 2023-04-06 12:57:07,776 BAD EPOCHS (no improvement): 0 2023-04-06 12:57:07,779 ---------------------------------------------------------------------------------------------------- 2023-04-06 12:57:25,998 epoch 150 - iter 265/2650 - loss 0.02433222 - time (sec): 18.22 - samples/sec: 7980.56 - lr: 0.012500 2023-04-06 12:57:44,435 epoch 150 - iter 530/2650 - loss 0.02383379 - time (sec): 36.66 - samples/sec: 7986.54 - lr: 0.012500 2023-04-06 12:58:02,563 epoch 150 - iter 795/2650 - loss 0.02414831 - time (sec): 54.78 - samples/sec: 8007.08 - lr: 0.012500 2023-04-06 12:58:21,980 epoch 150 - iter 1060/2650 - loss 0.02432075 - time (sec): 74.20 - samples/sec: 7894.22 - lr: 0.012500 2023-04-06 12:58:40,492 epoch 150 - iter 1325/2650 - loss 0.02442200 - time (sec): 92.71 - samples/sec: 7890.80 - lr: 0.012500 2023-04-06 12:59:01,125 epoch 150 - iter 1590/2650 - loss 0.02455813 - time (sec): 113.35 - samples/sec: 7762.82 - lr: 0.012500 2023-04-06 12:59:19,375 epoch 150 - iter 1855/2650 - loss 0.02454424 - time (sec): 131.60 - samples/sec: 7794.77 - lr: 0.012500 2023-04-06 12:59:38,696 epoch 150 - iter 2120/2650 - loss 0.02460402 - time (sec): 150.92 - samples/sec: 7764.32 - lr: 0.012500 2023-04-06 12:59:56,605 epoch 150 - iter 2385/2650 - loss 0.02458019 - time (sec): 168.83 - samples/sec: 7802.38 - lr: 0.012500 2023-04-06 13:00:15,958 epoch 150 - iter 2650/2650 - loss 0.02466815 - time (sec): 188.18 - samples/sec: 7771.15 - lr: 0.012500 2023-04-06 13:00:15,958 ---------------------------------------------------------------------------------------------------- 2023-04-06 13:00:15,958 EPOCH 150 done: loss 0.0247 - lr 0.012500 2023-04-06 13:00:15,958 BAD EPOCHS (no improvement): 0 2023-04-06 13:00:17,284 ---------------------------------------------------------------------------------------------------- 2023-04-06 13:00:17,285 Testing using last state of model ... 2023-04-06 13:00:52,119 Evaluating as a multi-label problem: False 2023-04-06 13:00:53,372 0.9854 0.9854 0.9854 0.9854 2023-04-06 13:00:53,372 Results: - F-score (micro) 0.9854 - F-score (macro) 0.9726 - Accuracy 0.9854 By class: precision recall f1-score support NOUN 0.9856 0.9813 0.9834 29573 VERB 0.9865 0.9918 0.9891 25556 PUNCT 0.9998 1.0000 0.9999 20452 ADP 0.9837 0.9867 0.9852 18929 DET 0.9949 0.9953 0.9951 14951 PROPN 0.9778 0.9830 0.9804 13149 PRON 0.9953 0.9956 0.9954 11968 ADJ 0.9553 0.9505 0.9529 10417 ADV 0.9697 0.9587 0.9642 9065 CCONJ 0.9943 0.9889 0.9916 5126 PART 0.9722 0.9881 0.9801 3357 NUM 0.9834 0.9898 0.9866 3045 AUX 0.9980 0.9975 0.9978 2001 INTJ 0.9794 0.9758 0.9776 1609 SYM 1.0000 0.9712 0.9854 208 X 0.8905 0.7219 0.7974 169 accuracy 0.9854 169575 macro avg 0.9791 0.9673 0.9726 169575 weighted avg 0.9854 0.9854 0.9854 169575 2023-04-06 13:00:53,372 ----------------------------------------------------------------------------------------------------