diff --git "a/training.log" "b/training.log" new file mode 100644--- /dev/null +++ "b/training.log" @@ -0,0 +1,16063 @@ +2021-03-26 04:17:55,632 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:17:55,632 Model: "SequenceTagger( + (embeddings): StackedEmbeddings( + (list_embedding_0): WordEmbeddings('ar') + (list_embedding_1): FlairEmbeddings( + (lm): LanguageModel( + (drop): Dropout(p=0.1, inplace=False) + (encoder): Embedding(7125, 100) + (rnn): LSTM(100, 2048) + (decoder): Linear(in_features=2048, out_features=7125, bias=True) + ) + ) + (list_embedding_2): FlairEmbeddings( + (lm): LanguageModel( + (drop): Dropout(p=0.1, inplace=False) + (encoder): Embedding(7125, 100) + (rnn): LSTM(100, 2048) + (decoder): Linear(in_features=2048, out_features=7125, bias=True) + ) + ) + ) + (word_dropout): WordDropout(p=0.05) + (locked_dropout): LockedDropout(p=0.5) + (embedding2nn): Linear(in_features=4396, out_features=4396, bias=True) + (rnn): LSTM(4396, 256, batch_first=True, bidirectional=True) + (linear): Linear(in_features=512, out_features=206, bias=True) + (beta): 1.0 + (weights): None + (weight_tensor) None +)" +2021-03-26 04:17:55,633 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:17:55,633 Corpus: "MultiCorpus: 1573 train + 176 dev + 195 test sentences + - ColumnCorpus Corpus: 934 train + 104 dev + 115 test sentences + - ColumnCorpus Corpus: 81 train + 9 dev + 10 test sentences + - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences + - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences" +2021-03-26 04:17:55,633 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:17:55,634 Parameters: +2021-03-26 04:17:55,634 - learning_rate: "0.3" +2021-03-26 04:17:55,634 - mini_batch_size: "32" +2021-03-26 04:17:55,634 - patience: "3" +2021-03-26 04:17:55,635 - anneal_factor: "0.5" +2021-03-26 04:17:55,635 - max_epochs: "150" +2021-03-26 04:17:55,635 - shuffle: "True" +2021-03-26 04:17:55,635 - train_with_dev: "False" +2021-03-26 04:17:55,635 - batch_growth_annealing: "False" +2021-03-26 04:17:55,636 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:17:55,636 Model training base path: "/home/tmp/megahedm/models/multipos/multipos_UDMADAR_4Diale-LEV_EGY_GLF_MGR__fasttext_flairbwfw__32__0.3_202103260417" +2021-03-26 04:17:55,636 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:17:55,636 Device: cuda:0 +2021-03-26 04:17:55,636 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:17:55,637 Embeddings storage mode: cpu +2021-03-26 04:17:55,638 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:17:58,321 epoch 1 - iter 5/50 - loss 75.10696945 - samples/sec: 59.66 - lr: 0.300000 +2021-03-26 04:18:00,696 epoch 1 - iter 10/50 - loss 71.54744225 - samples/sec: 67.43 - lr: 0.300000 +2021-03-26 04:18:03,249 epoch 1 - iter 15/50 - loss 65.56100286 - samples/sec: 62.69 - lr: 0.300000 +2021-03-26 04:18:05,739 epoch 1 - iter 20/50 - loss 62.89946499 - samples/sec: 64.29 - lr: 0.300000 +2021-03-26 04:18:08,362 epoch 1 - iter 25/50 - loss 60.70132797 - samples/sec: 61.05 - lr: 0.300000 +2021-03-26 04:18:10,893 epoch 1 - iter 30/50 - loss 57.30805092 - samples/sec: 63.25 - lr: 0.300000 +2021-03-26 04:18:13,216 epoch 1 - iter 35/50 - loss 55.10309862 - samples/sec: 68.92 - lr: 0.300000 +2021-03-26 04:18:15,733 epoch 1 - iter 40/50 - loss 53.12539721 - samples/sec: 63.62 - lr: 0.300000 +2021-03-26 04:18:18,247 epoch 1 - iter 45/50 - loss 51.49834357 - samples/sec: 63.70 - lr: 0.300000 +2021-03-26 04:18:20,689 epoch 1 - iter 50/50 - loss 49.88232040 - samples/sec: 65.55 - lr: 0.300000 +2021-03-26 04:18:20,690 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:18:20,690 EPOCH 1 done: loss 49.8823 - lr 0.3000000 +2021-03-26 04:18:22,048 DEV : loss 30.73543357849121 - score 0.507 +2021-03-26 04:18:22,069 BAD EPOCHS (no improvement): 0 +2021-03-26 04:18:31,763 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:18:33,775 epoch 2 - iter 5/50 - loss 32.49798546 - samples/sec: 79.62 - lr: 0.300000 +2021-03-26 04:18:35,664 epoch 2 - iter 10/50 - loss 31.19827633 - samples/sec: 84.79 - lr: 0.300000 +2021-03-26 04:18:37,726 epoch 2 - iter 15/50 - loss 30.67719905 - samples/sec: 77.69 - lr: 0.300000 +2021-03-26 04:18:39,723 epoch 2 - iter 20/50 - loss 29.86658115 - samples/sec: 80.18 - lr: 0.300000 +2021-03-26 04:18:41,766 epoch 2 - iter 25/50 - loss 29.56375023 - samples/sec: 78.41 - lr: 0.300000 +2021-03-26 04:18:43,775 epoch 2 - iter 30/50 - loss 28.81621901 - samples/sec: 79.72 - lr: 0.300000 +2021-03-26 04:18:45,721 epoch 2 - iter 35/50 - loss 28.27086552 - samples/sec: 82.30 - lr: 0.300000 +2021-03-26 04:18:47,632 epoch 2 - iter 40/50 - loss 27.63589411 - samples/sec: 83.85 - lr: 0.300000 +2021-03-26 04:18:49,710 epoch 2 - iter 45/50 - loss 27.50761443 - samples/sec: 77.06 - lr: 0.300000 +2021-03-26 04:18:51,660 epoch 2 - iter 50/50 - loss 27.20592896 - samples/sec: 82.13 - lr: 0.300000 +2021-03-26 04:18:51,661 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:18:51,661 EPOCH 2 done: loss 27.2059 - lr 0.3000000 +2021-03-26 04:18:52,475 DEV : loss 17.981204986572266 - score 0.6999 +2021-03-26 04:18:52,501 BAD EPOCHS (no improvement): 0 +2021-03-26 04:19:02,566 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:19:04,559 epoch 3 - iter 5/50 - loss 19.67391281 - samples/sec: 80.41 - lr: 0.300000 +2021-03-26 04:19:06,670 epoch 3 - iter 10/50 - loss 20.82917442 - samples/sec: 75.83 - lr: 0.300000 +2021-03-26 04:19:08,786 epoch 3 - iter 15/50 - loss 21.23497098 - samples/sec: 75.69 - lr: 0.300000 +2021-03-26 04:19:10,635 epoch 3 - iter 20/50 - loss 20.67392168 - samples/sec: 86.64 - lr: 0.300000 +2021-03-26 04:19:12,548 epoch 3 - iter 25/50 - loss 19.83488708 - samples/sec: 83.76 - lr: 0.300000 +2021-03-26 04:19:14,539 epoch 3 - iter 30/50 - loss 19.89728591 - samples/sec: 80.40 - lr: 0.300000 +2021-03-26 04:19:16,432 epoch 3 - iter 35/50 - loss 19.76958389 - samples/sec: 84.61 - lr: 0.300000 +2021-03-26 04:19:18,402 epoch 3 - iter 40/50 - loss 19.47361774 - samples/sec: 81.30 - lr: 0.300000 +2021-03-26 04:19:20,338 epoch 3 - iter 45/50 - loss 19.23065883 - samples/sec: 82.73 - lr: 0.300000 +2021-03-26 04:19:22,103 epoch 3 - iter 50/50 - loss 19.27288631 - samples/sec: 90.77 - lr: 0.300000 +2021-03-26 04:19:22,103 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:19:22,104 EPOCH 3 done: loss 19.2729 - lr 0.3000000 +2021-03-26 04:19:22,902 DEV : loss 13.695486068725586 - score 0.7567 +2021-03-26 04:19:22,928 BAD EPOCHS (no improvement): 0 +2021-03-26 04:19:32,480 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:19:34,387 epoch 4 - iter 5/50 - loss 14.91749001 - samples/sec: 84.04 - lr: 0.300000 +2021-03-26 04:19:36,328 epoch 4 - iter 10/50 - loss 16.39088926 - samples/sec: 82.48 - lr: 0.300000 +2021-03-26 04:19:38,223 epoch 4 - iter 15/50 - loss 16.06209895 - samples/sec: 84.54 - lr: 0.300000 +2021-03-26 04:19:40,308 epoch 4 - iter 20/50 - loss 16.54561882 - samples/sec: 76.79 - lr: 0.300000 +2021-03-26 04:19:42,262 epoch 4 - iter 25/50 - loss 16.21666988 - samples/sec: 81.94 - lr: 0.300000 +2021-03-26 04:19:44,231 epoch 4 - iter 30/50 - loss 16.23734223 - samples/sec: 81.36 - lr: 0.300000 +2021-03-26 04:19:46,120 epoch 4 - iter 35/50 - loss 16.14153720 - samples/sec: 84.81 - lr: 0.300000 +2021-03-26 04:19:48,060 epoch 4 - iter 40/50 - loss 15.98076320 - samples/sec: 82.52 - lr: 0.300000 +2021-03-26 04:19:50,062 epoch 4 - iter 45/50 - loss 15.74216646 - samples/sec: 79.97 - lr: 0.300000 +2021-03-26 04:19:51,884 epoch 4 - iter 50/50 - loss 15.68846325 - samples/sec: 87.92 - lr: 0.300000 +2021-03-26 04:19:51,885 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:19:51,885 EPOCH 4 done: loss 15.6885 - lr 0.3000000 +2021-03-26 04:19:52,680 DEV : loss 12.137931823730469 - score 0.7942 +2021-03-26 04:19:52,706 BAD EPOCHS (no improvement): 0 +2021-03-26 04:20:02,622 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:20:04,608 epoch 5 - iter 5/50 - loss 15.45747547 - samples/sec: 80.65 - lr: 0.300000 +2021-03-26 04:20:06,776 epoch 5 - iter 10/50 - loss 15.33912888 - samples/sec: 73.90 - lr: 0.300000 +2021-03-26 04:20:08,865 epoch 5 - iter 15/50 - loss 15.48315074 - samples/sec: 76.70 - lr: 0.300000 +2021-03-26 04:20:10,899 epoch 5 - iter 20/50 - loss 14.89399667 - samples/sec: 78.77 - lr: 0.300000 +2021-03-26 04:20:12,904 epoch 5 - iter 25/50 - loss 14.98029118 - samples/sec: 79.87 - lr: 0.300000 +2021-03-26 04:20:14,756 epoch 5 - iter 30/50 - loss 14.63713500 - samples/sec: 86.50 - lr: 0.300000 +2021-03-26 04:20:16,879 epoch 5 - iter 35/50 - loss 14.07365886 - samples/sec: 75.45 - lr: 0.300000 +2021-03-26 04:20:18,870 epoch 5 - iter 40/50 - loss 14.03398926 - samples/sec: 80.41 - lr: 0.300000 +2021-03-26 04:20:20,746 epoch 5 - iter 45/50 - loss 13.89231139 - samples/sec: 85.41 - lr: 0.300000 +2021-03-26 04:20:22,557 epoch 5 - iter 50/50 - loss 13.74129833 - samples/sec: 88.42 - lr: 0.300000 +2021-03-26 04:20:22,558 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:20:22,559 EPOCH 5 done: loss 13.7413 - lr 0.3000000 +2021-03-26 04:20:23,442 DEV : loss 10.568546295166016 - score 0.8127 +2021-03-26 04:20:23,460 BAD EPOCHS (no improvement): 0 +2021-03-26 04:20:33,013 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:20:34,946 epoch 6 - iter 5/50 - loss 12.02973881 - samples/sec: 82.90 - lr: 0.300000 +2021-03-26 04:20:37,336 epoch 6 - iter 10/50 - loss 12.09904261 - samples/sec: 67.01 - lr: 0.300000 +2021-03-26 04:20:39,441 epoch 6 - iter 15/50 - loss 11.87269878 - samples/sec: 76.07 - lr: 0.300000 +2021-03-26 04:20:41,745 epoch 6 - iter 20/50 - loss 11.90691776 - samples/sec: 69.51 - lr: 0.300000 +2021-03-26 04:20:43,681 epoch 6 - iter 25/50 - loss 11.86476704 - samples/sec: 82.70 - lr: 0.300000 +2021-03-26 04:20:45,619 epoch 6 - iter 30/50 - loss 11.94253292 - samples/sec: 82.64 - lr: 0.300000 +2021-03-26 04:20:47,465 epoch 6 - iter 35/50 - loss 12.14770309 - samples/sec: 86.76 - lr: 0.300000 +2021-03-26 04:20:49,570 epoch 6 - iter 40/50 - loss 12.19662697 - samples/sec: 76.07 - lr: 0.300000 +2021-03-26 04:20:51,449 epoch 6 - iter 45/50 - loss 12.11139940 - samples/sec: 85.25 - lr: 0.300000 +2021-03-26 04:20:53,099 epoch 6 - iter 50/50 - loss 11.94814116 - samples/sec: 97.09 - lr: 0.300000 +2021-03-26 04:20:53,100 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:20:53,100 EPOCH 6 done: loss 11.9481 - lr 0.3000000 +2021-03-26 04:20:53,894 DEV : loss 9.031622886657715 - score 0.8425 +2021-03-26 04:20:53,920 BAD EPOCHS (no improvement): 0 +2021-03-26 04:21:03,930 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:21:06,014 epoch 7 - iter 5/50 - loss 12.33336906 - samples/sec: 76.88 - lr: 0.300000 +2021-03-26 04:21:08,062 epoch 7 - iter 10/50 - loss 10.94311547 - samples/sec: 78.20 - lr: 0.300000 +2021-03-26 04:21:09,922 epoch 7 - iter 15/50 - loss 11.11774225 - samples/sec: 86.08 - lr: 0.300000 +2021-03-26 04:21:11,821 epoch 7 - iter 20/50 - loss 11.46735899 - samples/sec: 84.37 - lr: 0.300000 +2021-03-26 04:21:13,675 epoch 7 - iter 25/50 - loss 11.08219643 - samples/sec: 86.40 - lr: 0.300000 +2021-03-26 04:21:15,552 epoch 7 - iter 30/50 - loss 11.15123660 - samples/sec: 85.30 - lr: 0.300000 +2021-03-26 04:21:17,521 epoch 7 - iter 35/50 - loss 11.02137399 - samples/sec: 81.31 - lr: 0.300000 +2021-03-26 04:21:19,751 epoch 7 - iter 40/50 - loss 11.08207619 - samples/sec: 71.83 - lr: 0.300000 +2021-03-26 04:21:21,811 epoch 7 - iter 45/50 - loss 11.05912957 - samples/sec: 77.73 - lr: 0.300000 +2021-03-26 04:21:23,589 epoch 7 - iter 50/50 - loss 10.84630838 - samples/sec: 90.08 - lr: 0.300000 +2021-03-26 04:21:23,590 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:21:23,591 EPOCH 7 done: loss 10.8463 - lr 0.3000000 +2021-03-26 04:21:24,378 DEV : loss 9.201812744140625 - score 0.8337 +2021-03-26 04:21:24,403 BAD EPOCHS (no improvement): 1 +2021-03-26 04:21:24,404 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:21:26,358 epoch 8 - iter 5/50 - loss 9.95667782 - samples/sec: 81.97 - lr: 0.300000 +2021-03-26 04:21:28,455 epoch 8 - iter 10/50 - loss 10.97621489 - samples/sec: 76.36 - lr: 0.300000 +2021-03-26 04:21:30,506 epoch 8 - iter 15/50 - loss 10.84686445 - samples/sec: 78.10 - lr: 0.300000 +2021-03-26 04:21:32,501 epoch 8 - iter 20/50 - loss 10.96737380 - samples/sec: 80.26 - lr: 0.300000 +2021-03-26 04:21:34,422 epoch 8 - iter 25/50 - loss 10.52111063 - samples/sec: 83.41 - lr: 0.300000 +2021-03-26 04:21:36,394 epoch 8 - iter 30/50 - loss 10.18619401 - samples/sec: 81.20 - lr: 0.300000 +2021-03-26 04:21:38,854 epoch 8 - iter 35/50 - loss 10.46665453 - samples/sec: 65.07 - lr: 0.300000 +2021-03-26 04:21:41,130 epoch 8 - iter 40/50 - loss 10.44879591 - samples/sec: 70.37 - lr: 0.300000 +2021-03-26 04:21:43,139 epoch 8 - iter 45/50 - loss 10.45465293 - samples/sec: 79.70 - lr: 0.300000 +2021-03-26 04:21:45,011 epoch 8 - iter 50/50 - loss 10.33866776 - samples/sec: 85.56 - lr: 0.300000 +2021-03-26 04:21:45,012 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:21:45,012 EPOCH 8 done: loss 10.3387 - lr 0.3000000 +2021-03-26 04:21:45,817 DEV : loss 7.98112678527832 - score 0.8624 +2021-03-26 04:21:45,842 BAD EPOCHS (no improvement): 0 +2021-03-26 04:21:55,706 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:21:57,603 epoch 9 - iter 5/50 - loss 9.39634228 - samples/sec: 84.46 - lr: 0.300000 +2021-03-26 04:21:59,638 epoch 9 - iter 10/50 - loss 9.66039915 - samples/sec: 78.68 - lr: 0.300000 +2021-03-26 04:22:01,612 epoch 9 - iter 15/50 - loss 9.76121337 - samples/sec: 81.13 - lr: 0.300000 +2021-03-26 04:22:03,619 epoch 9 - iter 20/50 - loss 9.80530524 - samples/sec: 79.83 - lr: 0.300000 +2021-03-26 04:22:05,705 epoch 9 - iter 25/50 - loss 9.52289055 - samples/sec: 76.78 - lr: 0.300000 +2021-03-26 04:22:07,848 epoch 9 - iter 30/50 - loss 9.68419746 - samples/sec: 74.75 - lr: 0.300000 +2021-03-26 04:22:09,921 epoch 9 - iter 35/50 - loss 9.60363786 - samples/sec: 77.29 - lr: 0.300000 +2021-03-26 04:22:11,889 epoch 9 - iter 40/50 - loss 9.56846452 - samples/sec: 81.38 - lr: 0.300000 +2021-03-26 04:22:13,679 epoch 9 - iter 45/50 - loss 9.52679759 - samples/sec: 89.44 - lr: 0.300000 +2021-03-26 04:22:15,610 epoch 9 - iter 50/50 - loss 9.51809608 - samples/sec: 82.98 - lr: 0.300000 +2021-03-26 04:22:15,611 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:22:15,611 EPOCH 9 done: loss 9.5181 - lr 0.3000000 +2021-03-26 04:22:16,446 DEV : loss 7.542094707489014 - score 0.8666 +2021-03-26 04:22:16,467 BAD EPOCHS (no improvement): 0 +2021-03-26 04:22:26,106 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:22:28,089 epoch 10 - iter 5/50 - loss 8.68427620 - samples/sec: 80.79 - lr: 0.300000 +2021-03-26 04:22:30,020 epoch 10 - iter 10/50 - loss 8.22454109 - samples/sec: 83.05 - lr: 0.300000 +2021-03-26 04:22:32,134 epoch 10 - iter 15/50 - loss 8.22477700 - samples/sec: 75.75 - lr: 0.300000 +2021-03-26 04:22:34,141 epoch 10 - iter 20/50 - loss 8.36938996 - samples/sec: 79.82 - lr: 0.300000 +2021-03-26 04:22:35,979 epoch 10 - iter 25/50 - loss 8.27269264 - samples/sec: 87.13 - lr: 0.300000 +2021-03-26 04:22:37,934 epoch 10 - iter 30/50 - loss 8.49550018 - samples/sec: 81.94 - lr: 0.300000 +2021-03-26 04:22:40,038 epoch 10 - iter 35/50 - loss 8.60381297 - samples/sec: 76.17 - lr: 0.300000 +2021-03-26 04:22:41,867 epoch 10 - iter 40/50 - loss 8.52518930 - samples/sec: 87.61 - lr: 0.300000 +2021-03-26 04:22:43,828 epoch 10 - iter 45/50 - loss 8.66577231 - samples/sec: 81.65 - lr: 0.300000 +2021-03-26 04:22:45,532 epoch 10 - iter 50/50 - loss 8.64119944 - samples/sec: 94.00 - lr: 0.300000 +2021-03-26 04:22:45,532 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:22:45,533 EPOCH 10 done: loss 8.6412 - lr 0.3000000 +2021-03-26 04:22:46,324 DEV : loss 6.95101261138916 - score 0.874 +2021-03-26 04:22:46,348 BAD EPOCHS (no improvement): 0 +2021-03-26 04:22:56,214 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:22:58,403 epoch 11 - iter 5/50 - loss 8.42196026 - samples/sec: 73.20 - lr: 0.300000 +2021-03-26 04:23:00,322 epoch 11 - iter 10/50 - loss 8.40741711 - samples/sec: 83.50 - lr: 0.300000 +2021-03-26 04:23:02,263 epoch 11 - iter 15/50 - loss 8.24041354 - samples/sec: 82.53 - lr: 0.300000 +2021-03-26 04:23:04,403 epoch 11 - iter 20/50 - loss 8.41640503 - samples/sec: 74.84 - lr: 0.300000 +2021-03-26 04:23:06,478 epoch 11 - iter 25/50 - loss 8.47183853 - samples/sec: 77.21 - lr: 0.300000 +2021-03-26 04:23:08,442 epoch 11 - iter 30/50 - loss 8.52703635 - samples/sec: 81.53 - lr: 0.300000 +2021-03-26 04:23:10,264 epoch 11 - iter 35/50 - loss 8.45981086 - samples/sec: 87.98 - lr: 0.300000 +2021-03-26 04:23:12,391 epoch 11 - iter 40/50 - loss 8.43065636 - samples/sec: 75.27 - lr: 0.300000 +2021-03-26 04:23:14,376 epoch 11 - iter 45/50 - loss 8.42949425 - samples/sec: 80.67 - lr: 0.300000 +2021-03-26 04:23:16,426 epoch 11 - iter 50/50 - loss 8.36444000 - samples/sec: 78.16 - lr: 0.300000 +2021-03-26 04:23:16,427 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:23:16,427 EPOCH 11 done: loss 8.3644 - lr 0.3000000 +2021-03-26 04:23:17,267 DEV : loss 6.946200847625732 - score 0.8772 +2021-03-26 04:23:17,293 BAD EPOCHS (no improvement): 0 +2021-03-26 04:23:26,956 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:23:29,007 epoch 12 - iter 5/50 - loss 6.99961243 - samples/sec: 78.12 - lr: 0.300000 +2021-03-26 04:23:30,967 epoch 12 - iter 10/50 - loss 7.69392977 - samples/sec: 81.69 - lr: 0.300000 +2021-03-26 04:23:33,135 epoch 12 - iter 15/50 - loss 8.04296926 - samples/sec: 73.88 - lr: 0.300000 +2021-03-26 04:23:35,160 epoch 12 - iter 20/50 - loss 8.05934105 - samples/sec: 79.08 - lr: 0.300000 +2021-03-26 04:23:37,187 epoch 12 - iter 25/50 - loss 7.93668758 - samples/sec: 79.03 - lr: 0.300000 +2021-03-26 04:23:39,866 epoch 12 - iter 30/50 - loss 8.06570826 - samples/sec: 59.75 - lr: 0.300000 +2021-03-26 04:23:41,897 epoch 12 - iter 35/50 - loss 7.95962033 - samples/sec: 78.92 - lr: 0.300000 +2021-03-26 04:23:44,121 epoch 12 - iter 40/50 - loss 7.95873179 - samples/sec: 71.98 - lr: 0.300000 +2021-03-26 04:23:46,042 epoch 12 - iter 45/50 - loss 7.85535208 - samples/sec: 83.38 - lr: 0.300000 +2021-03-26 04:23:47,838 epoch 12 - iter 50/50 - loss 7.84040368 - samples/sec: 89.18 - lr: 0.300000 +2021-03-26 04:23:47,839 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:23:47,839 EPOCH 12 done: loss 7.8404 - lr 0.3000000 +2021-03-26 04:23:48,675 DEV : loss 6.661767482757568 - score 0.8829 +2021-03-26 04:23:48,703 BAD EPOCHS (no improvement): 0 +2021-03-26 04:23:58,778 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:24:00,931 epoch 13 - iter 5/50 - loss 7.16372118 - samples/sec: 74.43 - lr: 0.300000 +2021-03-26 04:24:02,729 epoch 13 - iter 10/50 - loss 7.43499422 - samples/sec: 89.11 - lr: 0.300000 +2021-03-26 04:24:04,635 epoch 13 - iter 15/50 - loss 7.52820756 - samples/sec: 84.04 - lr: 0.300000 +2021-03-26 04:24:06,834 epoch 13 - iter 20/50 - loss 7.58331962 - samples/sec: 72.83 - lr: 0.300000 +2021-03-26 04:24:08,895 epoch 13 - iter 25/50 - loss 7.71773420 - samples/sec: 77.70 - lr: 0.300000 +2021-03-26 04:24:11,069 epoch 13 - iter 30/50 - loss 7.67236060 - samples/sec: 73.67 - lr: 0.300000 +2021-03-26 04:24:13,130 epoch 13 - iter 35/50 - loss 7.62056946 - samples/sec: 77.68 - lr: 0.300000 +2021-03-26 04:24:15,003 epoch 13 - iter 40/50 - loss 7.53186643 - samples/sec: 85.50 - lr: 0.300000 +2021-03-26 04:24:16,956 epoch 13 - iter 45/50 - loss 7.55881039 - samples/sec: 82.04 - lr: 0.300000 +2021-03-26 04:24:18,742 epoch 13 - iter 50/50 - loss 7.46832686 - samples/sec: 89.62 - lr: 0.300000 +2021-03-26 04:24:18,743 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:24:18,743 EPOCH 13 done: loss 7.4683 - lr 0.3000000 +2021-03-26 04:24:19,542 DEV : loss 6.98322057723999 - score 0.8823 +2021-03-26 04:24:19,560 BAD EPOCHS (no improvement): 1 +2021-03-26 04:24:19,561 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:24:21,655 epoch 14 - iter 5/50 - loss 6.10111704 - samples/sec: 76.47 - lr: 0.300000 +2021-03-26 04:24:23,668 epoch 14 - iter 10/50 - loss 6.29064140 - samples/sec: 79.60 - lr: 0.300000 +2021-03-26 04:24:25,618 epoch 14 - iter 15/50 - loss 6.46209361 - samples/sec: 82.10 - lr: 0.300000 +2021-03-26 04:24:27,656 epoch 14 - iter 20/50 - loss 6.76686928 - samples/sec: 78.57 - lr: 0.300000 +2021-03-26 04:24:29,664 epoch 14 - iter 25/50 - loss 6.86153515 - samples/sec: 79.73 - lr: 0.300000 +2021-03-26 04:24:31,549 epoch 14 - iter 30/50 - loss 6.75523327 - samples/sec: 84.97 - lr: 0.300000 +2021-03-26 04:24:33,582 epoch 14 - iter 35/50 - loss 6.96098984 - samples/sec: 78.75 - lr: 0.300000 +2021-03-26 04:24:35,489 epoch 14 - iter 40/50 - loss 6.94293357 - samples/sec: 84.01 - lr: 0.300000 +2021-03-26 04:24:37,446 epoch 14 - iter 45/50 - loss 6.96709191 - samples/sec: 81.85 - lr: 0.300000 +2021-03-26 04:24:39,275 epoch 14 - iter 50/50 - loss 7.01545926 - samples/sec: 87.59 - lr: 0.300000 +2021-03-26 04:24:39,275 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:24:39,276 EPOCH 14 done: loss 7.0155 - lr 0.3000000 +2021-03-26 04:24:40,079 DEV : loss 6.541140556335449 - score 0.8916 +2021-03-26 04:24:40,105 BAD EPOCHS (no improvement): 0 +2021-03-26 04:24:49,775 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:24:51,891 epoch 15 - iter 5/50 - loss 7.28769703 - samples/sec: 75.72 - lr: 0.300000 +2021-03-26 04:24:53,966 epoch 15 - iter 10/50 - loss 7.33594060 - samples/sec: 77.18 - lr: 0.300000 +2021-03-26 04:24:55,862 epoch 15 - iter 15/50 - loss 6.93184964 - samples/sec: 84.43 - lr: 0.300000 +2021-03-26 04:24:57,869 epoch 15 - iter 20/50 - loss 6.87248054 - samples/sec: 79.79 - lr: 0.300000 +2021-03-26 04:24:59,767 epoch 15 - iter 25/50 - loss 6.95717945 - samples/sec: 84.43 - lr: 0.300000 +2021-03-26 04:25:01,869 epoch 15 - iter 30/50 - loss 7.01571414 - samples/sec: 76.18 - lr: 0.300000 +2021-03-26 04:25:03,809 epoch 15 - iter 35/50 - loss 7.14505937 - samples/sec: 82.55 - lr: 0.300000 +2021-03-26 04:25:05,793 epoch 15 - iter 40/50 - loss 7.03667165 - samples/sec: 80.75 - lr: 0.300000 +2021-03-26 04:25:07,699 epoch 15 - iter 45/50 - loss 6.90432454 - samples/sec: 84.02 - lr: 0.300000 +2021-03-26 04:25:09,309 epoch 15 - iter 50/50 - loss 6.88903606 - samples/sec: 99.54 - lr: 0.300000 +2021-03-26 04:25:09,310 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:25:09,310 EPOCH 15 done: loss 6.8890 - lr 0.3000000 +2021-03-26 04:25:10,121 DEV : loss 6.795278072357178 - score 0.8813 +2021-03-26 04:25:10,146 BAD EPOCHS (no improvement): 1 +2021-03-26 04:25:10,147 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:25:12,174 epoch 16 - iter 5/50 - loss 6.26615057 - samples/sec: 78.98 - lr: 0.300000 +2021-03-26 04:25:14,157 epoch 16 - iter 10/50 - loss 6.86684566 - samples/sec: 80.79 - lr: 0.300000 +2021-03-26 04:25:16,097 epoch 16 - iter 15/50 - loss 6.87310921 - samples/sec: 82.56 - lr: 0.300000 +2021-03-26 04:25:17,936 epoch 16 - iter 20/50 - loss 6.56489854 - samples/sec: 87.12 - lr: 0.300000 +2021-03-26 04:25:19,906 epoch 16 - iter 25/50 - loss 6.47284828 - samples/sec: 81.28 - lr: 0.300000 +2021-03-26 04:25:21,812 epoch 16 - iter 30/50 - loss 6.31857347 - samples/sec: 84.05 - lr: 0.300000 +2021-03-26 04:25:23,865 epoch 16 - iter 35/50 - loss 6.49668576 - samples/sec: 77.97 - lr: 0.300000 +2021-03-26 04:25:25,833 epoch 16 - iter 40/50 - loss 6.44836339 - samples/sec: 81.39 - lr: 0.300000 +2021-03-26 04:25:27,753 epoch 16 - iter 45/50 - loss 6.49273729 - samples/sec: 83.43 - lr: 0.300000 +2021-03-26 04:25:29,478 epoch 16 - iter 50/50 - loss 6.52631361 - samples/sec: 92.83 - lr: 0.300000 +2021-03-26 04:25:29,478 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:25:29,479 EPOCH 16 done: loss 6.5263 - lr 0.3000000 +2021-03-26 04:25:30,312 DEV : loss 6.253621578216553 - score 0.8935 +2021-03-26 04:25:30,338 BAD EPOCHS (no improvement): 0 +2021-03-26 04:25:40,033 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:25:42,008 epoch 17 - iter 5/50 - loss 6.54145203 - samples/sec: 81.17 - lr: 0.300000 +2021-03-26 04:25:44,025 epoch 17 - iter 10/50 - loss 6.48270683 - samples/sec: 79.37 - lr: 0.300000 +2021-03-26 04:25:46,119 epoch 17 - iter 15/50 - loss 6.22515707 - samples/sec: 76.49 - lr: 0.300000 +2021-03-26 04:25:47,974 epoch 17 - iter 20/50 - loss 6.20492010 - samples/sec: 86.36 - lr: 0.300000 +2021-03-26 04:25:49,913 epoch 17 - iter 25/50 - loss 6.13051668 - samples/sec: 82.62 - lr: 0.300000 +2021-03-26 04:25:51,864 epoch 17 - iter 30/50 - loss 6.13832904 - samples/sec: 82.09 - lr: 0.300000 +2021-03-26 04:25:53,769 epoch 17 - iter 35/50 - loss 6.12548726 - samples/sec: 84.10 - lr: 0.300000 +2021-03-26 04:25:55,680 epoch 17 - iter 40/50 - loss 6.18831244 - samples/sec: 83.84 - lr: 0.300000 +2021-03-26 04:25:57,818 epoch 17 - iter 45/50 - loss 6.28022619 - samples/sec: 74.89 - lr: 0.300000 +2021-03-26 04:25:59,757 epoch 17 - iter 50/50 - loss 6.22215544 - samples/sec: 82.58 - lr: 0.300000 +2021-03-26 04:25:59,757 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:25:59,758 EPOCH 17 done: loss 6.2222 - lr 0.3000000 +2021-03-26 04:26:00,587 DEV : loss 5.916778564453125 - score 0.8984 +2021-03-26 04:26:00,612 BAD EPOCHS (no improvement): 0 +2021-03-26 04:26:10,517 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:26:12,469 epoch 18 - iter 5/50 - loss 5.82463799 - samples/sec: 82.16 - lr: 0.300000 +2021-03-26 04:26:14,563 epoch 18 - iter 10/50 - loss 5.79875684 - samples/sec: 76.45 - lr: 0.300000 +2021-03-26 04:26:16,654 epoch 18 - iter 15/50 - loss 5.68639402 - samples/sec: 76.57 - lr: 0.300000 +2021-03-26 04:26:18,633 epoch 18 - iter 20/50 - loss 5.87758648 - samples/sec: 80.94 - lr: 0.300000 +2021-03-26 04:26:20,589 epoch 18 - iter 25/50 - loss 5.76586943 - samples/sec: 81.89 - lr: 0.300000 +2021-03-26 04:26:22,563 epoch 18 - iter 30/50 - loss 5.90244009 - samples/sec: 81.19 - lr: 0.300000 +2021-03-26 04:26:24,687 epoch 18 - iter 35/50 - loss 5.86991540 - samples/sec: 75.39 - lr: 0.300000 +2021-03-26 04:26:26,702 epoch 18 - iter 40/50 - loss 5.96687905 - samples/sec: 79.48 - lr: 0.300000 +2021-03-26 04:26:28,538 epoch 18 - iter 45/50 - loss 5.96893187 - samples/sec: 87.22 - lr: 0.300000 +2021-03-26 04:26:30,673 epoch 18 - iter 50/50 - loss 6.12081981 - samples/sec: 74.98 - lr: 0.300000 +2021-03-26 04:26:30,674 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:26:30,675 EPOCH 18 done: loss 6.1208 - lr 0.3000000 +2021-03-26 04:26:31,500 DEV : loss 6.017233848571777 - score 0.899 +2021-03-26 04:26:31,525 BAD EPOCHS (no improvement): 0 +2021-03-26 04:26:41,353 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:26:43,368 epoch 19 - iter 5/50 - loss 5.00032930 - samples/sec: 79.47 - lr: 0.300000 +2021-03-26 04:26:45,421 epoch 19 - iter 10/50 - loss 5.22974877 - samples/sec: 78.03 - lr: 0.300000 +2021-03-26 04:26:47,424 epoch 19 - iter 15/50 - loss 5.28217349 - samples/sec: 79.97 - lr: 0.300000 +2021-03-26 04:26:49,427 epoch 19 - iter 20/50 - loss 5.33064126 - samples/sec: 79.96 - lr: 0.300000 +2021-03-26 04:26:51,311 epoch 19 - iter 25/50 - loss 5.43961421 - samples/sec: 85.00 - lr: 0.300000 +2021-03-26 04:26:53,174 epoch 19 - iter 30/50 - loss 5.54057804 - samples/sec: 85.97 - lr: 0.300000 +2021-03-26 04:26:55,322 epoch 19 - iter 35/50 - loss 5.59828159 - samples/sec: 74.54 - lr: 0.300000 +2021-03-26 04:26:57,570 epoch 19 - iter 40/50 - loss 5.63219038 - samples/sec: 71.24 - lr: 0.300000 +2021-03-26 04:26:59,866 epoch 19 - iter 45/50 - loss 5.62833966 - samples/sec: 69.77 - lr: 0.300000 +2021-03-26 04:27:02,021 epoch 19 - iter 50/50 - loss 5.73004192 - samples/sec: 74.29 - lr: 0.300000 +2021-03-26 04:27:02,022 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:27:02,022 EPOCH 19 done: loss 5.7300 - lr 0.3000000 +2021-03-26 04:27:02,832 DEV : loss 5.986006736755371 - score 0.9011 +2021-03-26 04:27:02,856 BAD EPOCHS (no improvement): 0 +2021-03-26 04:27:12,395 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:27:14,577 epoch 20 - iter 5/50 - loss 4.56922116 - samples/sec: 73.43 - lr: 0.300000 +2021-03-26 04:27:16,568 epoch 20 - iter 10/50 - loss 5.21878107 - samples/sec: 80.42 - lr: 0.300000 +2021-03-26 04:27:18,629 epoch 20 - iter 15/50 - loss 5.54959607 - samples/sec: 77.68 - lr: 0.300000 +2021-03-26 04:27:20,511 epoch 20 - iter 20/50 - loss 5.40102017 - samples/sec: 85.12 - lr: 0.300000 +2021-03-26 04:27:22,577 epoch 20 - iter 25/50 - loss 5.53657951 - samples/sec: 77.50 - lr: 0.300000 +2021-03-26 04:27:24,522 epoch 20 - iter 30/50 - loss 5.57236757 - samples/sec: 82.34 - lr: 0.300000 +2021-03-26 04:27:26,745 epoch 20 - iter 35/50 - loss 5.51763896 - samples/sec: 72.02 - lr: 0.300000 +2021-03-26 04:27:28,880 epoch 20 - iter 40/50 - loss 5.48591815 - samples/sec: 75.02 - lr: 0.300000 +2021-03-26 04:27:30,937 epoch 20 - iter 45/50 - loss 5.49633989 - samples/sec: 77.91 - lr: 0.300000 +2021-03-26 04:27:32,852 epoch 20 - iter 50/50 - loss 5.57298695 - samples/sec: 83.60 - lr: 0.300000 +2021-03-26 04:27:32,853 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:27:32,853 EPOCH 20 done: loss 5.5730 - lr 0.3000000 +2021-03-26 04:27:33,652 DEV : loss 5.930326461791992 - score 0.8968 +2021-03-26 04:27:33,676 BAD EPOCHS (no improvement): 1 +2021-03-26 04:27:33,677 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:27:35,714 epoch 21 - iter 5/50 - loss 6.82391586 - samples/sec: 78.62 - lr: 0.300000 +2021-03-26 04:27:37,885 epoch 21 - iter 10/50 - loss 5.70353832 - samples/sec: 73.74 - lr: 0.300000 +2021-03-26 04:27:39,715 epoch 21 - iter 15/50 - loss 5.08343248 - samples/sec: 87.50 - lr: 0.300000 +2021-03-26 04:27:41,571 epoch 21 - iter 20/50 - loss 5.16639459 - samples/sec: 86.31 - lr: 0.300000 +2021-03-26 04:27:43,425 epoch 21 - iter 25/50 - loss 5.16516088 - samples/sec: 86.37 - lr: 0.300000 +2021-03-26 04:27:45,541 epoch 21 - iter 30/50 - loss 5.25475041 - samples/sec: 75.71 - lr: 0.300000 +2021-03-26 04:27:47,672 epoch 21 - iter 35/50 - loss 5.19062170 - samples/sec: 75.17 - lr: 0.300000 +2021-03-26 04:27:49,833 epoch 21 - iter 40/50 - loss 5.14780552 - samples/sec: 74.10 - lr: 0.300000 +2021-03-26 04:27:51,757 epoch 21 - iter 45/50 - loss 5.16311884 - samples/sec: 83.27 - lr: 0.300000 +2021-03-26 04:27:53,831 epoch 21 - iter 50/50 - loss 5.24303764 - samples/sec: 77.21 - lr: 0.300000 +2021-03-26 04:27:53,831 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:27:53,832 EPOCH 21 done: loss 5.2430 - lr 0.3000000 +2021-03-26 04:27:54,631 DEV : loss 5.876016139984131 - score 0.9041 +2021-03-26 04:27:54,653 BAD EPOCHS (no improvement): 0 +2021-03-26 04:28:04,386 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:28:06,584 epoch 22 - iter 5/50 - loss 5.56373177 - samples/sec: 72.87 - lr: 0.300000 +2021-03-26 04:28:08,505 epoch 22 - iter 10/50 - loss 5.34489045 - samples/sec: 83.37 - lr: 0.300000 +2021-03-26 04:28:10,423 epoch 22 - iter 15/50 - loss 5.28978802 - samples/sec: 83.54 - lr: 0.300000 +2021-03-26 04:28:12,292 epoch 22 - iter 20/50 - loss 5.25375621 - samples/sec: 85.72 - lr: 0.300000 +2021-03-26 04:28:14,329 epoch 22 - iter 25/50 - loss 5.11982265 - samples/sec: 78.62 - lr: 0.300000 +2021-03-26 04:28:16,538 epoch 22 - iter 30/50 - loss 5.22046140 - samples/sec: 72.49 - lr: 0.300000 +2021-03-26 04:28:19,075 epoch 22 - iter 35/50 - loss 5.05933951 - samples/sec: 63.11 - lr: 0.300000 +2021-03-26 04:28:21,098 epoch 22 - iter 40/50 - loss 5.06113884 - samples/sec: 79.16 - lr: 0.300000 +2021-03-26 04:28:23,017 epoch 22 - iter 45/50 - loss 5.06400488 - samples/sec: 83.44 - lr: 0.300000 +2021-03-26 04:28:25,049 epoch 22 - iter 50/50 - loss 5.00299379 - samples/sec: 78.79 - lr: 0.300000 +2021-03-26 04:28:25,050 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:28:25,050 EPOCH 22 done: loss 5.0030 - lr 0.3000000 +2021-03-26 04:28:25,858 DEV : loss 6.265571594238281 - score 0.9001 +2021-03-26 04:28:25,882 BAD EPOCHS (no improvement): 1 +2021-03-26 04:28:25,883 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:28:27,807 epoch 23 - iter 5/50 - loss 4.88015375 - samples/sec: 83.26 - lr: 0.300000 +2021-03-26 04:28:29,610 epoch 23 - iter 10/50 - loss 4.93044343 - samples/sec: 88.80 - lr: 0.300000 +2021-03-26 04:28:31,525 epoch 23 - iter 15/50 - loss 4.89805943 - samples/sec: 83.63 - lr: 0.300000 +2021-03-26 04:28:33,514 epoch 23 - iter 20/50 - loss 4.81811663 - samples/sec: 80.51 - lr: 0.300000 +2021-03-26 04:28:35,611 epoch 23 - iter 25/50 - loss 4.94307913 - samples/sec: 76.41 - lr: 0.300000 +2021-03-26 04:28:37,546 epoch 23 - iter 30/50 - loss 4.94647453 - samples/sec: 82.73 - lr: 0.300000 +2021-03-26 04:28:39,574 epoch 23 - iter 35/50 - loss 5.03459977 - samples/sec: 79.00 - lr: 0.300000 +2021-03-26 04:28:41,466 epoch 23 - iter 40/50 - loss 4.94717068 - samples/sec: 84.63 - lr: 0.300000 +2021-03-26 04:28:43,578 epoch 23 - iter 45/50 - loss 5.00173565 - samples/sec: 75.83 - lr: 0.300000 +2021-03-26 04:28:45,557 epoch 23 - iter 50/50 - loss 5.05484855 - samples/sec: 80.93 - lr: 0.300000 +2021-03-26 04:28:45,557 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:28:45,557 EPOCH 23 done: loss 5.0548 - lr 0.3000000 +2021-03-26 04:28:46,352 DEV : loss 6.006520748138428 - score 0.9057 +2021-03-26 04:28:46,373 BAD EPOCHS (no improvement): 0 +2021-03-26 04:28:55,851 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:28:57,870 epoch 24 - iter 5/50 - loss 3.85893168 - samples/sec: 79.32 - lr: 0.300000 +2021-03-26 04:28:59,880 epoch 24 - iter 10/50 - loss 4.09285135 - samples/sec: 79.73 - lr: 0.300000 +2021-03-26 04:29:01,886 epoch 24 - iter 15/50 - loss 4.19558508 - samples/sec: 79.80 - lr: 0.300000 +2021-03-26 04:29:03,783 epoch 24 - iter 20/50 - loss 4.47492408 - samples/sec: 84.48 - lr: 0.300000 +2021-03-26 04:29:05,795 epoch 24 - iter 25/50 - loss 4.56721498 - samples/sec: 79.56 - lr: 0.300000 +2021-03-26 04:29:07,745 epoch 24 - iter 30/50 - loss 4.72045338 - samples/sec: 82.12 - lr: 0.300000 +2021-03-26 04:29:09,649 epoch 24 - iter 35/50 - loss 4.74745126 - samples/sec: 84.18 - lr: 0.300000 +2021-03-26 04:29:11,558 epoch 24 - iter 40/50 - loss 4.66962092 - samples/sec: 83.88 - lr: 0.300000 +2021-03-26 04:29:13,521 epoch 24 - iter 45/50 - loss 4.73731511 - samples/sec: 81.60 - lr: 0.300000 +2021-03-26 04:29:15,381 epoch 24 - iter 50/50 - loss 4.78475182 - samples/sec: 86.14 - lr: 0.300000 +2021-03-26 04:29:15,382 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:29:15,382 EPOCH 24 done: loss 4.7848 - lr 0.3000000 +2021-03-26 04:29:16,207 DEV : loss 6.140750885009766 - score 0.9007 +2021-03-26 04:29:16,232 BAD EPOCHS (no improvement): 1 +2021-03-26 04:29:16,233 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:29:18,379 epoch 25 - iter 5/50 - loss 4.88933020 - samples/sec: 74.64 - lr: 0.300000 +2021-03-26 04:29:20,394 epoch 25 - iter 10/50 - loss 4.66825428 - samples/sec: 79.44 - lr: 0.300000 +2021-03-26 04:29:22,242 epoch 25 - iter 15/50 - loss 4.67216385 - samples/sec: 86.68 - lr: 0.300000 +2021-03-26 04:29:24,195 epoch 25 - iter 20/50 - loss 4.49212998 - samples/sec: 82.01 - lr: 0.300000 +2021-03-26 04:29:26,091 epoch 25 - iter 25/50 - loss 4.58727453 - samples/sec: 84.47 - lr: 0.300000 +2021-03-26 04:29:28,017 epoch 25 - iter 30/50 - loss 4.55948855 - samples/sec: 83.18 - lr: 0.300000 +2021-03-26 04:29:29,965 epoch 25 - iter 35/50 - loss 4.46674612 - samples/sec: 82.24 - lr: 0.300000 +2021-03-26 04:29:31,937 epoch 25 - iter 40/50 - loss 4.52352757 - samples/sec: 81.23 - lr: 0.300000 +2021-03-26 04:29:33,897 epoch 25 - iter 45/50 - loss 4.58539889 - samples/sec: 81.71 - lr: 0.300000 +2021-03-26 04:29:35,781 epoch 25 - iter 50/50 - loss 4.57801357 - samples/sec: 84.99 - lr: 0.300000 +2021-03-26 04:29:35,782 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:29:35,782 EPOCH 25 done: loss 4.5780 - lr 0.3000000 +2021-03-26 04:29:36,609 DEV : loss 6.105770111083984 - score 0.9059 +2021-03-26 04:29:36,635 BAD EPOCHS (no improvement): 0 +2021-03-26 04:29:46,517 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:29:48,522 epoch 26 - iter 5/50 - loss 4.06674557 - samples/sec: 79.91 - lr: 0.300000 +2021-03-26 04:29:50,450 epoch 26 - iter 10/50 - loss 4.42879331 - samples/sec: 83.06 - lr: 0.300000 +2021-03-26 04:29:52,488 epoch 26 - iter 15/50 - loss 4.60063888 - samples/sec: 78.55 - lr: 0.300000 +2021-03-26 04:29:54,433 epoch 26 - iter 20/50 - loss 4.58111991 - samples/sec: 82.38 - lr: 0.300000 +2021-03-26 04:29:56,492 epoch 26 - iter 25/50 - loss 4.44824369 - samples/sec: 77.74 - lr: 0.300000 +2021-03-26 04:29:58,473 epoch 26 - iter 30/50 - loss 4.39979011 - samples/sec: 80.85 - lr: 0.300000 +2021-03-26 04:30:00,547 epoch 26 - iter 35/50 - loss 4.48905651 - samples/sec: 77.23 - lr: 0.300000 +2021-03-26 04:30:02,655 epoch 26 - iter 40/50 - loss 4.54615889 - samples/sec: 75.93 - lr: 0.300000 +2021-03-26 04:30:04,549 epoch 26 - iter 45/50 - loss 4.57882829 - samples/sec: 84.55 - lr: 0.300000 +2021-03-26 04:30:06,321 epoch 26 - iter 50/50 - loss 4.49696365 - samples/sec: 90.39 - lr: 0.300000 +2021-03-26 04:30:06,322 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:30:06,322 EPOCH 26 done: loss 4.4970 - lr 0.3000000 +2021-03-26 04:30:07,134 DEV : loss 6.628708839416504 - score 0.8974 +2021-03-26 04:30:07,156 BAD EPOCHS (no improvement): 1 +2021-03-26 04:30:07,156 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:30:08,969 epoch 27 - iter 5/50 - loss 4.78592529 - samples/sec: 88.35 - lr: 0.300000 +2021-03-26 04:30:11,153 epoch 27 - iter 10/50 - loss 4.92443051 - samples/sec: 73.30 - lr: 0.300000 +2021-03-26 04:30:13,078 epoch 27 - iter 15/50 - loss 4.73327821 - samples/sec: 83.24 - lr: 0.300000 +2021-03-26 04:30:15,045 epoch 27 - iter 20/50 - loss 4.75421574 - samples/sec: 81.39 - lr: 0.300000 +2021-03-26 04:30:17,141 epoch 27 - iter 25/50 - loss 4.66145059 - samples/sec: 76.43 - lr: 0.300000 +2021-03-26 04:30:19,233 epoch 27 - iter 30/50 - loss 4.61727355 - samples/sec: 76.54 - lr: 0.300000 +2021-03-26 04:30:21,261 epoch 27 - iter 35/50 - loss 4.65609898 - samples/sec: 78.95 - lr: 0.300000 +2021-03-26 04:30:23,220 epoch 27 - iter 40/50 - loss 4.61374962 - samples/sec: 81.81 - lr: 0.300000 +2021-03-26 04:30:25,081 epoch 27 - iter 45/50 - loss 4.52198956 - samples/sec: 86.08 - lr: 0.300000 +2021-03-26 04:30:26,948 epoch 27 - iter 50/50 - loss 4.54031735 - samples/sec: 85.77 - lr: 0.300000 +2021-03-26 04:30:26,949 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:30:26,949 EPOCH 27 done: loss 4.5403 - lr 0.3000000 +2021-03-26 04:30:27,744 DEV : loss 6.179070949554443 - score 0.9033 +2021-03-26 04:30:27,762 BAD EPOCHS (no improvement): 2 +2021-03-26 04:30:27,762 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:30:29,742 epoch 28 - iter 5/50 - loss 4.99302526 - samples/sec: 80.93 - lr: 0.300000 +2021-03-26 04:30:31,815 epoch 28 - iter 10/50 - loss 4.83438566 - samples/sec: 77.24 - lr: 0.300000 +2021-03-26 04:30:33,693 epoch 28 - iter 15/50 - loss 4.58077038 - samples/sec: 85.31 - lr: 0.300000 +2021-03-26 04:30:35,654 epoch 28 - iter 20/50 - loss 4.53201609 - samples/sec: 81.64 - lr: 0.300000 +2021-03-26 04:30:37,621 epoch 28 - iter 25/50 - loss 4.37551620 - samples/sec: 81.42 - lr: 0.300000 +2021-03-26 04:30:39,540 epoch 28 - iter 30/50 - loss 4.24277393 - samples/sec: 83.44 - lr: 0.300000 +2021-03-26 04:30:41,456 epoch 28 - iter 35/50 - loss 4.30316538 - samples/sec: 83.61 - lr: 0.300000 +2021-03-26 04:30:43,577 epoch 28 - iter 40/50 - loss 4.33001579 - samples/sec: 75.47 - lr: 0.300000 +2021-03-26 04:30:45,534 epoch 28 - iter 45/50 - loss 4.35705420 - samples/sec: 81.82 - lr: 0.300000 +2021-03-26 04:30:47,427 epoch 28 - iter 50/50 - loss 4.31629688 - samples/sec: 84.59 - lr: 0.300000 +2021-03-26 04:30:47,428 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:30:47,428 EPOCH 28 done: loss 4.3163 - lr 0.3000000 +2021-03-26 04:30:48,250 DEV : loss 5.845078945159912 - score 0.9038 +2021-03-26 04:30:48,270 BAD EPOCHS (no improvement): 3 +2021-03-26 04:30:48,271 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:30:50,432 epoch 29 - iter 5/50 - loss 4.47561340 - samples/sec: 74.12 - lr: 0.300000 +2021-03-26 04:30:52,438 epoch 29 - iter 10/50 - loss 4.05306280 - samples/sec: 79.88 - lr: 0.300000 +2021-03-26 04:30:54,531 epoch 29 - iter 15/50 - loss 4.26914605 - samples/sec: 76.51 - lr: 0.300000 +2021-03-26 04:30:56,485 epoch 29 - iter 20/50 - loss 4.22125438 - samples/sec: 81.99 - lr: 0.300000 +2021-03-26 04:30:58,434 epoch 29 - iter 25/50 - loss 4.19066468 - samples/sec: 82.18 - lr: 0.300000 +2021-03-26 04:31:00,471 epoch 29 - iter 30/50 - loss 4.11440060 - samples/sec: 78.61 - lr: 0.300000 +2021-03-26 04:31:02,363 epoch 29 - iter 35/50 - loss 4.13777413 - samples/sec: 84.71 - lr: 0.300000 +2021-03-26 04:31:04,222 epoch 29 - iter 40/50 - loss 4.09913780 - samples/sec: 86.15 - lr: 0.300000 +2021-03-26 04:31:06,304 epoch 29 - iter 45/50 - loss 4.15816302 - samples/sec: 76.93 - lr: 0.300000 +2021-03-26 04:31:08,093 epoch 29 - iter 50/50 - loss 4.18301320 - samples/sec: 89.62 - lr: 0.300000 +2021-03-26 04:31:08,094 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:31:08,094 EPOCH 29 done: loss 4.1830 - lr 0.3000000 +2021-03-26 04:31:08,926 DEV : loss 6.092258453369141 - score 0.9051 +2021-03-26 04:31:08,950 BAD EPOCHS (no improvement): 4 +2021-03-26 04:31:08,951 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:31:11,072 epoch 30 - iter 5/50 - loss 4.20534439 - samples/sec: 75.49 - lr: 0.150000 +2021-03-26 04:31:13,164 epoch 30 - iter 10/50 - loss 4.13369124 - samples/sec: 76.52 - lr: 0.150000 +2021-03-26 04:31:15,215 epoch 30 - iter 15/50 - loss 3.90392586 - samples/sec: 78.13 - lr: 0.150000 +2021-03-26 04:31:17,081 epoch 30 - iter 20/50 - loss 3.84214165 - samples/sec: 85.89 - lr: 0.150000 +2021-03-26 04:31:19,071 epoch 30 - iter 25/50 - loss 3.86618098 - samples/sec: 80.45 - lr: 0.150000 +2021-03-26 04:31:21,025 epoch 30 - iter 30/50 - loss 3.75677759 - samples/sec: 82.00 - lr: 0.150000 +2021-03-26 04:31:23,051 epoch 30 - iter 35/50 - loss 3.81993992 - samples/sec: 79.05 - lr: 0.150000 +2021-03-26 04:31:25,031 epoch 30 - iter 40/50 - loss 3.82808222 - samples/sec: 80.90 - lr: 0.150000 +2021-03-26 04:31:26,997 epoch 30 - iter 45/50 - loss 3.81757127 - samples/sec: 81.46 - lr: 0.150000 +2021-03-26 04:31:28,853 epoch 30 - iter 50/50 - loss 3.96143827 - samples/sec: 86.29 - lr: 0.150000 +2021-03-26 04:31:28,854 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:31:28,854 EPOCH 30 done: loss 3.9614 - lr 0.1500000 +2021-03-26 04:31:29,685 DEV : loss 5.713150978088379 - score 0.9096 +2021-03-26 04:31:29,711 BAD EPOCHS (no improvement): 0 +2021-03-26 04:31:39,433 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:31:41,513 epoch 31 - iter 5/50 - loss 3.97427645 - samples/sec: 77.01 - lr: 0.150000 +2021-03-26 04:31:43,538 epoch 31 - iter 10/50 - loss 4.07154245 - samples/sec: 79.13 - lr: 0.150000 +2021-03-26 04:31:45,428 epoch 31 - iter 15/50 - loss 3.84109275 - samples/sec: 84.73 - lr: 0.150000 +2021-03-26 04:31:47,265 epoch 31 - iter 20/50 - loss 3.63473686 - samples/sec: 87.20 - lr: 0.150000 +2021-03-26 04:31:49,372 epoch 31 - iter 25/50 - loss 3.69866761 - samples/sec: 75.98 - lr: 0.150000 +2021-03-26 04:31:51,259 epoch 31 - iter 30/50 - loss 3.57191054 - samples/sec: 84.97 - lr: 0.150000 +2021-03-26 04:31:53,281 epoch 31 - iter 35/50 - loss 3.60921316 - samples/sec: 79.19 - lr: 0.150000 +2021-03-26 04:31:55,098 epoch 31 - iter 40/50 - loss 3.56092208 - samples/sec: 88.21 - lr: 0.150000 +2021-03-26 04:31:57,014 epoch 31 - iter 45/50 - loss 3.53439759 - samples/sec: 83.57 - lr: 0.150000 +2021-03-26 04:31:58,797 epoch 31 - iter 50/50 - loss 3.53695005 - samples/sec: 89.86 - lr: 0.150000 +2021-03-26 04:31:58,798 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:31:58,798 EPOCH 31 done: loss 3.5370 - lr 0.1500000 +2021-03-26 04:31:59,609 DEV : loss 5.8165435791015625 - score 0.9118 +2021-03-26 04:31:59,635 BAD EPOCHS (no improvement): 0 +2021-03-26 04:32:09,492 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:32:11,392 epoch 32 - iter 5/50 - loss 4.09934840 - samples/sec: 84.37 - lr: 0.150000 +2021-03-26 04:32:13,356 epoch 32 - iter 10/50 - loss 3.60038693 - samples/sec: 81.54 - lr: 0.150000 +2021-03-26 04:32:15,493 epoch 32 - iter 15/50 - loss 3.67805559 - samples/sec: 74.95 - lr: 0.150000 +2021-03-26 04:32:17,365 epoch 32 - iter 20/50 - loss 3.61150162 - samples/sec: 85.60 - lr: 0.150000 +2021-03-26 04:32:19,404 epoch 32 - iter 25/50 - loss 3.41172076 - samples/sec: 78.51 - lr: 0.150000 +2021-03-26 04:32:21,448 epoch 32 - iter 30/50 - loss 3.48814637 - samples/sec: 78.38 - lr: 0.150000 +2021-03-26 04:32:23,451 epoch 32 - iter 35/50 - loss 3.48316173 - samples/sec: 79.92 - lr: 0.150000 +2021-03-26 04:32:25,542 epoch 32 - iter 40/50 - loss 3.47152191 - samples/sec: 76.57 - lr: 0.150000 +2021-03-26 04:32:27,716 epoch 32 - iter 45/50 - loss 3.48673698 - samples/sec: 73.68 - lr: 0.150000 +2021-03-26 04:32:29,340 epoch 32 - iter 50/50 - loss 3.40475936 - samples/sec: 98.62 - lr: 0.150000 +2021-03-26 04:32:29,341 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:32:29,341 EPOCH 32 done: loss 3.4048 - lr 0.1500000 +2021-03-26 04:32:31,473 DEV : loss 5.617457389831543 - score 0.9136 +2021-03-26 04:32:31,510 BAD EPOCHS (no improvement): 0 +2021-03-26 04:32:41,354 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:32:43,608 epoch 33 - iter 5/50 - loss 2.96983914 - samples/sec: 71.05 - lr: 0.150000 +2021-03-26 04:32:45,636 epoch 33 - iter 10/50 - loss 3.19039111 - samples/sec: 79.00 - lr: 0.150000 +2021-03-26 04:32:47,564 epoch 33 - iter 15/50 - loss 3.03779140 - samples/sec: 83.09 - lr: 0.150000 +2021-03-26 04:32:49,550 epoch 33 - iter 20/50 - loss 3.12249979 - samples/sec: 80.64 - lr: 0.150000 +2021-03-26 04:32:51,600 epoch 33 - iter 25/50 - loss 3.13500700 - samples/sec: 78.09 - lr: 0.150000 +2021-03-26 04:32:53,471 epoch 33 - iter 30/50 - loss 3.13828891 - samples/sec: 85.65 - lr: 0.150000 +2021-03-26 04:32:55,516 epoch 33 - iter 35/50 - loss 3.20323824 - samples/sec: 78.33 - lr: 0.150000 +2021-03-26 04:32:57,567 epoch 33 - iter 40/50 - loss 3.16869446 - samples/sec: 78.06 - lr: 0.150000 +2021-03-26 04:32:59,656 epoch 33 - iter 45/50 - loss 3.27099452 - samples/sec: 76.67 - lr: 0.150000 +2021-03-26 04:33:01,478 epoch 33 - iter 50/50 - loss 3.27532232 - samples/sec: 87.91 - lr: 0.150000 +2021-03-26 04:33:01,479 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:33:01,479 EPOCH 33 done: loss 3.2753 - lr 0.1500000 +2021-03-26 04:33:02,316 DEV : loss 5.714725494384766 - score 0.9124 +2021-03-26 04:33:02,342 BAD EPOCHS (no improvement): 1 +2021-03-26 04:33:02,342 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:33:04,309 epoch 34 - iter 5/50 - loss 2.89319005 - samples/sec: 81.46 - lr: 0.150000 +2021-03-26 04:33:06,232 epoch 34 - iter 10/50 - loss 2.75303411 - samples/sec: 83.27 - lr: 0.150000 +2021-03-26 04:33:08,338 epoch 34 - iter 15/50 - loss 3.11904596 - samples/sec: 76.02 - lr: 0.150000 +2021-03-26 04:33:10,602 epoch 34 - iter 20/50 - loss 3.22779783 - samples/sec: 70.74 - lr: 0.150000 +2021-03-26 04:33:12,540 epoch 34 - iter 25/50 - loss 3.27530612 - samples/sec: 82.64 - lr: 0.150000 +2021-03-26 04:33:14,553 epoch 34 - iter 30/50 - loss 3.29493998 - samples/sec: 79.52 - lr: 0.150000 +2021-03-26 04:33:16,467 epoch 34 - iter 35/50 - loss 3.31849298 - samples/sec: 83.68 - lr: 0.150000 +2021-03-26 04:33:18,275 epoch 34 - iter 40/50 - loss 3.23385530 - samples/sec: 88.59 - lr: 0.150000 +2021-03-26 04:33:20,195 epoch 34 - iter 45/50 - loss 3.17170356 - samples/sec: 83.49 - lr: 0.150000 +2021-03-26 04:33:22,037 epoch 34 - iter 50/50 - loss 3.18908337 - samples/sec: 86.92 - lr: 0.150000 +2021-03-26 04:33:22,038 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:33:22,038 EPOCH 34 done: loss 3.1891 - lr 0.1500000 +2021-03-26 04:33:22,852 DEV : loss 5.6389312744140625 - score 0.9136 +2021-03-26 04:33:22,873 BAD EPOCHS (no improvement): 2 +2021-03-26 04:33:22,874 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:33:24,832 epoch 35 - iter 5/50 - loss 2.92917271 - samples/sec: 81.80 - lr: 0.150000 +2021-03-26 04:33:26,841 epoch 35 - iter 10/50 - loss 3.03267279 - samples/sec: 79.73 - lr: 0.150000 +2021-03-26 04:33:28,869 epoch 35 - iter 15/50 - loss 3.13962334 - samples/sec: 78.96 - lr: 0.150000 +2021-03-26 04:33:30,717 epoch 35 - iter 20/50 - loss 3.20024943 - samples/sec: 86.65 - lr: 0.150000 +2021-03-26 04:33:32,584 epoch 35 - iter 25/50 - loss 3.23139182 - samples/sec: 85.76 - lr: 0.150000 +2021-03-26 04:33:34,464 epoch 35 - iter 30/50 - loss 3.11910969 - samples/sec: 85.18 - lr: 0.150000 +2021-03-26 04:33:36,597 epoch 35 - iter 35/50 - loss 3.06835806 - samples/sec: 75.12 - lr: 0.150000 +2021-03-26 04:33:38,676 epoch 35 - iter 40/50 - loss 3.11270922 - samples/sec: 77.00 - lr: 0.150000 +2021-03-26 04:33:40,518 epoch 35 - iter 45/50 - loss 3.10661338 - samples/sec: 86.92 - lr: 0.150000 +2021-03-26 04:33:42,433 epoch 35 - iter 50/50 - loss 3.14953357 - samples/sec: 83.64 - lr: 0.150000 +2021-03-26 04:33:42,434 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:33:42,434 EPOCH 35 done: loss 3.1495 - lr 0.1500000 +2021-03-26 04:33:43,271 DEV : loss 5.767107963562012 - score 0.9122 +2021-03-26 04:33:43,298 BAD EPOCHS (no improvement): 3 +2021-03-26 04:33:43,298 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:33:45,465 epoch 36 - iter 5/50 - loss 3.18284845 - samples/sec: 73.89 - lr: 0.150000 +2021-03-26 04:33:47,373 epoch 36 - iter 10/50 - loss 3.61262190 - samples/sec: 83.94 - lr: 0.150000 +2021-03-26 04:33:49,321 epoch 36 - iter 15/50 - loss 3.56332256 - samples/sec: 82.22 - lr: 0.150000 +2021-03-26 04:33:51,328 epoch 36 - iter 20/50 - loss 3.38617566 - samples/sec: 79.77 - lr: 0.150000 +2021-03-26 04:33:53,248 epoch 36 - iter 25/50 - loss 3.27218135 - samples/sec: 83.45 - lr: 0.150000 +2021-03-26 04:33:55,266 epoch 36 - iter 30/50 - loss 3.26657916 - samples/sec: 79.35 - lr: 0.150000 +2021-03-26 04:33:57,401 epoch 36 - iter 35/50 - loss 3.25774821 - samples/sec: 74.98 - lr: 0.150000 +2021-03-26 04:33:59,259 epoch 36 - iter 40/50 - loss 3.29469748 - samples/sec: 86.29 - lr: 0.150000 +2021-03-26 04:34:01,156 epoch 36 - iter 45/50 - loss 3.27597940 - samples/sec: 84.39 - lr: 0.150000 +2021-03-26 04:34:02,977 epoch 36 - iter 50/50 - loss 3.32588531 - samples/sec: 87.99 - lr: 0.150000 +2021-03-26 04:34:02,977 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:34:02,978 EPOCH 36 done: loss 3.3259 - lr 0.1500000 +2021-03-26 04:34:03,795 DEV : loss 5.621600151062012 - score 0.9114 +2021-03-26 04:34:03,821 BAD EPOCHS (no improvement): 4 +2021-03-26 04:34:03,822 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:34:05,909 epoch 37 - iter 5/50 - loss 3.24834247 - samples/sec: 76.73 - lr: 0.075000 +2021-03-26 04:34:07,829 epoch 37 - iter 10/50 - loss 3.15661476 - samples/sec: 83.48 - lr: 0.075000 +2021-03-26 04:34:09,776 epoch 37 - iter 15/50 - loss 3.05580727 - samples/sec: 82.25 - lr: 0.075000 +2021-03-26 04:34:11,596 epoch 37 - iter 20/50 - loss 2.99344176 - samples/sec: 87.97 - lr: 0.075000 +2021-03-26 04:34:13,697 epoch 37 - iter 25/50 - loss 2.99454953 - samples/sec: 76.23 - lr: 0.075000 +2021-03-26 04:34:15,759 epoch 37 - iter 30/50 - loss 3.03316812 - samples/sec: 77.64 - lr: 0.075000 +2021-03-26 04:34:17,722 epoch 37 - iter 35/50 - loss 3.03424073 - samples/sec: 81.60 - lr: 0.075000 +2021-03-26 04:34:19,646 epoch 37 - iter 40/50 - loss 3.06192328 - samples/sec: 83.23 - lr: 0.075000 +2021-03-26 04:34:22,044 epoch 37 - iter 45/50 - loss 3.01700522 - samples/sec: 66.75 - lr: 0.075000 +2021-03-26 04:34:24,021 epoch 37 - iter 50/50 - loss 3.06639231 - samples/sec: 81.03 - lr: 0.075000 +2021-03-26 04:34:24,021 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:34:24,022 EPOCH 37 done: loss 3.0664 - lr 0.0750000 +2021-03-26 04:34:24,840 DEV : loss 5.518418788909912 - score 0.9183 +2021-03-26 04:34:24,859 BAD EPOCHS (no improvement): 0 +2021-03-26 04:34:34,545 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:34:36,543 epoch 38 - iter 5/50 - loss 3.03597603 - samples/sec: 80.20 - lr: 0.075000 +2021-03-26 04:34:38,828 epoch 38 - iter 10/50 - loss 2.76591871 - samples/sec: 70.06 - lr: 0.075000 +2021-03-26 04:34:40,861 epoch 38 - iter 15/50 - loss 3.07190213 - samples/sec: 78.78 - lr: 0.075000 +2021-03-26 04:34:42,892 epoch 38 - iter 20/50 - loss 3.16678113 - samples/sec: 78.86 - lr: 0.075000 +2021-03-26 04:34:44,907 epoch 38 - iter 25/50 - loss 3.06128056 - samples/sec: 79.52 - lr: 0.075000 +2021-03-26 04:34:46,942 epoch 38 - iter 30/50 - loss 3.09672271 - samples/sec: 78.69 - lr: 0.075000 +2021-03-26 04:34:48,862 epoch 38 - iter 35/50 - loss 3.00959673 - samples/sec: 83.42 - lr: 0.075000 +2021-03-26 04:34:50,782 epoch 38 - iter 40/50 - loss 3.02298063 - samples/sec: 83.42 - lr: 0.075000 +2021-03-26 04:34:52,693 epoch 38 - iter 45/50 - loss 3.00045899 - samples/sec: 83.81 - lr: 0.075000 +2021-03-26 04:34:54,592 epoch 38 - iter 50/50 - loss 2.97319459 - samples/sec: 84.31 - lr: 0.075000 +2021-03-26 04:34:54,593 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:34:54,593 EPOCH 38 done: loss 2.9732 - lr 0.0750000 +2021-03-26 04:34:55,436 DEV : loss 5.583366394042969 - score 0.9142 +2021-03-26 04:34:55,455 BAD EPOCHS (no improvement): 1 +2021-03-26 04:34:55,456 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:34:57,401 epoch 39 - iter 5/50 - loss 2.80546980 - samples/sec: 82.30 - lr: 0.075000 +2021-03-26 04:34:59,405 epoch 39 - iter 10/50 - loss 3.02134721 - samples/sec: 79.92 - lr: 0.075000 +2021-03-26 04:35:01,331 epoch 39 - iter 15/50 - loss 2.87578603 - samples/sec: 83.13 - lr: 0.075000 +2021-03-26 04:35:03,179 epoch 39 - iter 20/50 - loss 2.81496171 - samples/sec: 86.66 - lr: 0.075000 +2021-03-26 04:35:05,238 epoch 39 - iter 25/50 - loss 2.82369418 - samples/sec: 77.80 - lr: 0.075000 +2021-03-26 04:35:07,215 epoch 39 - iter 30/50 - loss 2.83144122 - samples/sec: 81.04 - lr: 0.075000 +2021-03-26 04:35:09,314 epoch 39 - iter 35/50 - loss 2.85793325 - samples/sec: 76.26 - lr: 0.075000 +2021-03-26 04:35:11,180 epoch 39 - iter 40/50 - loss 2.88679821 - samples/sec: 85.82 - lr: 0.075000 +2021-03-26 04:35:13,101 epoch 39 - iter 45/50 - loss 2.85130506 - samples/sec: 83.38 - lr: 0.075000 +2021-03-26 04:35:14,930 epoch 39 - iter 50/50 - loss 2.84462428 - samples/sec: 87.53 - lr: 0.075000 +2021-03-26 04:35:14,931 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:35:14,931 EPOCH 39 done: loss 2.8446 - lr 0.0750000 +2021-03-26 04:35:15,708 DEV : loss 5.578202724456787 - score 0.9146 +2021-03-26 04:35:15,733 BAD EPOCHS (no improvement): 2 +2021-03-26 04:35:15,734 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:35:17,691 epoch 40 - iter 5/50 - loss 3.21980987 - samples/sec: 81.84 - lr: 0.075000 +2021-03-26 04:35:19,685 epoch 40 - iter 10/50 - loss 2.90155966 - samples/sec: 80.31 - lr: 0.075000 +2021-03-26 04:35:21,654 epoch 40 - iter 15/50 - loss 2.88553217 - samples/sec: 81.31 - lr: 0.075000 +2021-03-26 04:35:23,646 epoch 40 - iter 20/50 - loss 2.84431648 - samples/sec: 80.43 - lr: 0.075000 +2021-03-26 04:35:25,461 epoch 40 - iter 25/50 - loss 2.83064776 - samples/sec: 88.21 - lr: 0.075000 +2021-03-26 04:35:27,363 epoch 40 - iter 30/50 - loss 2.81809179 - samples/sec: 84.26 - lr: 0.075000 +2021-03-26 04:35:29,390 epoch 40 - iter 35/50 - loss 2.87427166 - samples/sec: 78.99 - lr: 0.075000 +2021-03-26 04:35:31,415 epoch 40 - iter 40/50 - loss 2.90430885 - samples/sec: 79.08 - lr: 0.075000 +2021-03-26 04:35:33,373 epoch 40 - iter 45/50 - loss 2.86053363 - samples/sec: 81.78 - lr: 0.075000 +2021-03-26 04:35:35,158 epoch 40 - iter 50/50 - loss 2.90439391 - samples/sec: 89.76 - lr: 0.075000 +2021-03-26 04:35:35,159 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:35:35,159 EPOCH 40 done: loss 2.9044 - lr 0.0750000 +2021-03-26 04:35:35,978 DEV : loss 5.7049994468688965 - score 0.9118 +2021-03-26 04:35:36,004 BAD EPOCHS (no improvement): 3 +2021-03-26 04:35:36,005 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:35:38,161 epoch 41 - iter 5/50 - loss 2.84771671 - samples/sec: 74.30 - lr: 0.075000 +2021-03-26 04:35:40,251 epoch 41 - iter 10/50 - loss 2.94583633 - samples/sec: 76.62 - lr: 0.075000 +2021-03-26 04:35:42,371 epoch 41 - iter 15/50 - loss 2.72260372 - samples/sec: 75.54 - lr: 0.075000 +2021-03-26 04:35:44,450 epoch 41 - iter 20/50 - loss 2.76606383 - samples/sec: 77.02 - lr: 0.075000 +2021-03-26 04:35:46,377 epoch 41 - iter 25/50 - loss 2.76165267 - samples/sec: 83.13 - lr: 0.075000 +2021-03-26 04:35:48,253 epoch 41 - iter 30/50 - loss 2.85141704 - samples/sec: 85.36 - lr: 0.075000 +2021-03-26 04:35:50,314 epoch 41 - iter 35/50 - loss 2.80431631 - samples/sec: 77.72 - lr: 0.075000 +2021-03-26 04:35:52,563 epoch 41 - iter 40/50 - loss 2.77395735 - samples/sec: 71.21 - lr: 0.075000 +2021-03-26 04:35:54,603 epoch 41 - iter 45/50 - loss 2.82585680 - samples/sec: 78.54 - lr: 0.075000 +2021-03-26 04:35:56,527 epoch 41 - iter 50/50 - loss 2.76732529 - samples/sec: 83.24 - lr: 0.075000 +2021-03-26 04:35:56,528 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:35:56,528 EPOCH 41 done: loss 2.7673 - lr 0.0750000 +2021-03-26 04:35:57,451 DEV : loss 5.642393112182617 - score 0.9177 +2021-03-26 04:35:57,477 BAD EPOCHS (no improvement): 4 +2021-03-26 04:35:57,478 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:35:59,690 epoch 42 - iter 5/50 - loss 3.21034460 - samples/sec: 72.39 - lr: 0.037500 +2021-03-26 04:36:02,342 epoch 42 - iter 10/50 - loss 3.35436149 - samples/sec: 60.40 - lr: 0.037500 +2021-03-26 04:36:04,451 epoch 42 - iter 15/50 - loss 3.15462260 - samples/sec: 75.98 - lr: 0.037500 +2021-03-26 04:36:06,524 epoch 42 - iter 20/50 - loss 3.04606973 - samples/sec: 77.25 - lr: 0.037500 +2021-03-26 04:36:08,527 epoch 42 - iter 25/50 - loss 2.99213443 - samples/sec: 79.98 - lr: 0.037500 +2021-03-26 04:36:10,483 epoch 42 - iter 30/50 - loss 2.90308900 - samples/sec: 81.90 - lr: 0.037500 +2021-03-26 04:36:12,373 epoch 42 - iter 35/50 - loss 2.88003882 - samples/sec: 84.74 - lr: 0.037500 +2021-03-26 04:36:14,356 epoch 42 - iter 40/50 - loss 2.84434015 - samples/sec: 80.74 - lr: 0.037500 +2021-03-26 04:36:16,462 epoch 42 - iter 45/50 - loss 2.76004986 - samples/sec: 76.07 - lr: 0.037500 +2021-03-26 04:36:18,278 epoch 42 - iter 50/50 - loss 2.71170438 - samples/sec: 88.20 - lr: 0.037500 +2021-03-26 04:36:18,279 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:36:18,279 EPOCH 42 done: loss 2.7117 - lr 0.0375000 +2021-03-26 04:36:19,157 DEV : loss 5.596333980560303 - score 0.9191 +2021-03-26 04:36:19,181 BAD EPOCHS (no improvement): 0 +2021-03-26 04:36:28,982 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:36:31,053 epoch 43 - iter 5/50 - loss 2.53796349 - samples/sec: 77.34 - lr: 0.037500 +2021-03-26 04:36:33,117 epoch 43 - iter 10/50 - loss 2.67341330 - samples/sec: 77.59 - lr: 0.037500 +2021-03-26 04:36:35,206 epoch 43 - iter 15/50 - loss 2.67378864 - samples/sec: 76.66 - lr: 0.037500 +2021-03-26 04:36:37,234 epoch 43 - iter 20/50 - loss 2.65061052 - samples/sec: 78.95 - lr: 0.037500 +2021-03-26 04:36:39,252 epoch 43 - iter 25/50 - loss 2.57634023 - samples/sec: 79.35 - lr: 0.037500 +2021-03-26 04:36:41,419 epoch 43 - iter 30/50 - loss 2.63065434 - samples/sec: 73.90 - lr: 0.037500 +2021-03-26 04:36:43,523 epoch 43 - iter 35/50 - loss 2.63950038 - samples/sec: 76.12 - lr: 0.037500 +2021-03-26 04:36:45,589 epoch 43 - iter 40/50 - loss 2.64751761 - samples/sec: 77.55 - lr: 0.037500 +2021-03-26 04:36:47,700 epoch 43 - iter 45/50 - loss 2.74424743 - samples/sec: 75.88 - lr: 0.037500 +2021-03-26 04:36:49,751 epoch 43 - iter 50/50 - loss 2.72231129 - samples/sec: 78.06 - lr: 0.037500 +2021-03-26 04:36:49,752 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:36:49,752 EPOCH 43 done: loss 2.7223 - lr 0.0375000 +2021-03-26 04:36:50,556 DEV : loss 5.48341703414917 - score 0.9203 +2021-03-26 04:36:50,582 BAD EPOCHS (no improvement): 0 +2021-03-26 04:37:00,171 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:37:02,366 epoch 44 - iter 5/50 - loss 2.98065414 - samples/sec: 73.00 - lr: 0.037500 +2021-03-26 04:37:04,384 epoch 44 - iter 10/50 - loss 3.02249229 - samples/sec: 79.34 - lr: 0.037500 +2021-03-26 04:37:06,401 epoch 44 - iter 15/50 - loss 2.85490796 - samples/sec: 79.42 - lr: 0.037500 +2021-03-26 04:37:08,446 epoch 44 - iter 20/50 - loss 2.86533296 - samples/sec: 78.29 - lr: 0.037500 +2021-03-26 04:37:10,490 epoch 44 - iter 25/50 - loss 2.80952433 - samples/sec: 78.33 - lr: 0.037500 +2021-03-26 04:37:12,411 epoch 44 - iter 30/50 - loss 2.80893228 - samples/sec: 83.40 - lr: 0.037500 +2021-03-26 04:37:14,429 epoch 44 - iter 35/50 - loss 2.75288236 - samples/sec: 79.36 - lr: 0.037500 +2021-03-26 04:37:16,351 epoch 44 - iter 40/50 - loss 2.72165121 - samples/sec: 83.34 - lr: 0.037500 +2021-03-26 04:37:18,338 epoch 44 - iter 45/50 - loss 2.73430733 - samples/sec: 80.57 - lr: 0.037500 +2021-03-26 04:37:20,362 epoch 44 - iter 50/50 - loss 2.72831919 - samples/sec: 79.12 - lr: 0.037500 +2021-03-26 04:37:20,363 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:37:20,363 EPOCH 44 done: loss 2.7283 - lr 0.0375000 +2021-03-26 04:37:21,167 DEV : loss 5.58450984954834 - score 0.9177 +2021-03-26 04:37:21,191 BAD EPOCHS (no improvement): 1 +2021-03-26 04:37:21,192 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:37:23,156 epoch 45 - iter 5/50 - loss 2.30449636 - samples/sec: 81.54 - lr: 0.037500 +2021-03-26 04:37:25,015 epoch 45 - iter 10/50 - loss 2.32092698 - samples/sec: 86.15 - lr: 0.037500 +2021-03-26 04:37:27,011 epoch 45 - iter 15/50 - loss 2.29700800 - samples/sec: 80.25 - lr: 0.037500 +2021-03-26 04:37:29,031 epoch 45 - iter 20/50 - loss 2.43191308 - samples/sec: 79.28 - lr: 0.037500 +2021-03-26 04:37:30,997 epoch 45 - iter 25/50 - loss 2.51080059 - samples/sec: 81.45 - lr: 0.037500 +2021-03-26 04:37:32,982 epoch 45 - iter 30/50 - loss 2.52594601 - samples/sec: 80.73 - lr: 0.037500 +2021-03-26 04:37:34,963 epoch 45 - iter 35/50 - loss 2.51447598 - samples/sec: 80.89 - lr: 0.037500 +2021-03-26 04:37:37,083 epoch 45 - iter 40/50 - loss 2.48263493 - samples/sec: 75.55 - lr: 0.037500 +2021-03-26 04:37:39,153 epoch 45 - iter 45/50 - loss 2.48890561 - samples/sec: 77.40 - lr: 0.037500 +2021-03-26 04:37:41,196 epoch 45 - iter 50/50 - loss 2.48906700 - samples/sec: 78.40 - lr: 0.037500 +2021-03-26 04:37:41,197 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:37:41,197 EPOCH 45 done: loss 2.4891 - lr 0.0375000 +2021-03-26 04:37:42,141 DEV : loss 5.558457374572754 - score 0.9191 +2021-03-26 04:37:42,165 BAD EPOCHS (no improvement): 2 +2021-03-26 04:37:42,166 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:37:44,169 epoch 46 - iter 5/50 - loss 2.66223438 - samples/sec: 79.97 - lr: 0.037500 +2021-03-26 04:37:46,352 epoch 46 - iter 10/50 - loss 2.64043747 - samples/sec: 73.35 - lr: 0.037500 +2021-03-26 04:37:48,443 epoch 46 - iter 15/50 - loss 2.68533693 - samples/sec: 76.58 - lr: 0.037500 +2021-03-26 04:37:50,538 epoch 46 - iter 20/50 - loss 2.90953861 - samples/sec: 76.46 - lr: 0.037500 +2021-03-26 04:37:52,680 epoch 46 - iter 25/50 - loss 2.93996088 - samples/sec: 74.79 - lr: 0.037500 +2021-03-26 04:37:54,552 epoch 46 - iter 30/50 - loss 2.83967495 - samples/sec: 85.57 - lr: 0.037500 +2021-03-26 04:37:56,390 epoch 46 - iter 35/50 - loss 2.74778470 - samples/sec: 87.13 - lr: 0.037500 +2021-03-26 04:37:58,518 epoch 46 - iter 40/50 - loss 2.73911650 - samples/sec: 75.25 - lr: 0.037500 +2021-03-26 04:38:00,941 epoch 46 - iter 45/50 - loss 2.68517716 - samples/sec: 66.09 - lr: 0.037500 +2021-03-26 04:38:02,810 epoch 46 - iter 50/50 - loss 2.67263688 - samples/sec: 85.68 - lr: 0.037500 +2021-03-26 04:38:02,811 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:38:02,811 EPOCH 46 done: loss 2.6726 - lr 0.0375000 +2021-03-26 04:38:03,601 DEV : loss 5.60237979888916 - score 0.9211 +2021-03-26 04:38:03,625 BAD EPOCHS (no improvement): 0 +2021-03-26 04:38:13,494 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:38:15,481 epoch 47 - iter 5/50 - loss 2.47122407 - samples/sec: 80.64 - lr: 0.037500 +2021-03-26 04:38:17,378 epoch 47 - iter 10/50 - loss 2.47364318 - samples/sec: 84.41 - lr: 0.037500 +2021-03-26 04:38:19,518 epoch 47 - iter 15/50 - loss 2.48115468 - samples/sec: 74.84 - lr: 0.037500 +2021-03-26 04:38:21,375 epoch 47 - iter 20/50 - loss 2.65376525 - samples/sec: 86.29 - lr: 0.037500 +2021-03-26 04:38:23,460 epoch 47 - iter 25/50 - loss 2.67308200 - samples/sec: 76.81 - lr: 0.037500 +2021-03-26 04:38:25,329 epoch 47 - iter 30/50 - loss 2.60481541 - samples/sec: 85.70 - lr: 0.037500 +2021-03-26 04:38:27,280 epoch 47 - iter 35/50 - loss 2.56278869 - samples/sec: 82.09 - lr: 0.037500 +2021-03-26 04:38:29,182 epoch 47 - iter 40/50 - loss 2.61160978 - samples/sec: 84.22 - lr: 0.037500 +2021-03-26 04:38:31,447 epoch 47 - iter 45/50 - loss 2.58385894 - samples/sec: 70.68 - lr: 0.037500 +2021-03-26 04:38:33,414 epoch 47 - iter 50/50 - loss 2.51415426 - samples/sec: 81.44 - lr: 0.037500 +2021-03-26 04:38:33,415 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:38:33,415 EPOCH 47 done: loss 2.5142 - lr 0.0375000 +2021-03-26 04:38:34,253 DEV : loss 5.621846675872803 - score 0.9215 +2021-03-26 04:38:34,274 BAD EPOCHS (no improvement): 0 +2021-03-26 04:38:43,880 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:38:46,076 epoch 48 - iter 5/50 - loss 2.57413118 - samples/sec: 72.95 - lr: 0.037500 +2021-03-26 04:38:48,107 epoch 48 - iter 10/50 - loss 2.61529049 - samples/sec: 78.86 - lr: 0.037500 +2021-03-26 04:38:50,015 epoch 48 - iter 15/50 - loss 2.66174406 - samples/sec: 83.90 - lr: 0.037500 +2021-03-26 04:38:52,139 epoch 48 - iter 20/50 - loss 2.75963891 - samples/sec: 75.39 - lr: 0.037500 +2021-03-26 04:38:54,253 epoch 48 - iter 25/50 - loss 2.66337224 - samples/sec: 75.75 - lr: 0.037500 +2021-03-26 04:38:56,278 epoch 48 - iter 30/50 - loss 2.67778315 - samples/sec: 79.08 - lr: 0.037500 +2021-03-26 04:38:58,402 epoch 48 - iter 35/50 - loss 2.64026489 - samples/sec: 75.41 - lr: 0.037500 +2021-03-26 04:39:00,478 epoch 48 - iter 40/50 - loss 2.59820676 - samples/sec: 77.12 - lr: 0.037500 +2021-03-26 04:39:02,539 epoch 48 - iter 45/50 - loss 2.64941340 - samples/sec: 77.68 - lr: 0.037500 +2021-03-26 04:39:04,142 epoch 48 - iter 50/50 - loss 2.60686312 - samples/sec: 100.01 - lr: 0.037500 +2021-03-26 04:39:04,143 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:39:04,143 EPOCH 48 done: loss 2.6069 - lr 0.0375000 +2021-03-26 04:39:04,964 DEV : loss 5.574971675872803 - score 0.9211 +2021-03-26 04:39:04,990 BAD EPOCHS (no improvement): 1 +2021-03-26 04:39:04,990 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:39:06,953 epoch 49 - iter 5/50 - loss 2.26608279 - samples/sec: 81.62 - lr: 0.037500 +2021-03-26 04:39:08,952 epoch 49 - iter 10/50 - loss 2.48984057 - samples/sec: 80.06 - lr: 0.037500 +2021-03-26 04:39:10,891 epoch 49 - iter 15/50 - loss 2.41273263 - samples/sec: 82.59 - lr: 0.037500 +2021-03-26 04:39:13,164 epoch 49 - iter 20/50 - loss 2.46317679 - samples/sec: 70.44 - lr: 0.037500 +2021-03-26 04:39:15,268 epoch 49 - iter 25/50 - loss 2.48580365 - samples/sec: 76.10 - lr: 0.037500 +2021-03-26 04:39:17,266 epoch 49 - iter 30/50 - loss 2.52267183 - samples/sec: 80.20 - lr: 0.037500 +2021-03-26 04:39:19,066 epoch 49 - iter 35/50 - loss 2.55116108 - samples/sec: 88.96 - lr: 0.037500 +2021-03-26 04:39:21,139 epoch 49 - iter 40/50 - loss 2.50691254 - samples/sec: 77.22 - lr: 0.037500 +2021-03-26 04:39:23,089 epoch 49 - iter 45/50 - loss 2.54569444 - samples/sec: 82.13 - lr: 0.037500 +2021-03-26 04:39:24,887 epoch 49 - iter 50/50 - loss 2.50360020 - samples/sec: 89.10 - lr: 0.037500 +2021-03-26 04:39:24,887 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:39:24,888 EPOCH 49 done: loss 2.5036 - lr 0.0375000 +2021-03-26 04:39:25,683 DEV : loss 5.674612998962402 - score 0.9167 +2021-03-26 04:39:25,707 BAD EPOCHS (no improvement): 2 +2021-03-26 04:39:25,708 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:39:27,531 epoch 50 - iter 5/50 - loss 2.49787436 - samples/sec: 87.87 - lr: 0.037500 +2021-03-26 04:39:29,470 epoch 50 - iter 10/50 - loss 2.54350984 - samples/sec: 82.65 - lr: 0.037500 +2021-03-26 04:39:31,617 epoch 50 - iter 15/50 - loss 2.55895030 - samples/sec: 74.60 - lr: 0.037500 +2021-03-26 04:39:33,746 epoch 50 - iter 20/50 - loss 2.53379130 - samples/sec: 75.19 - lr: 0.037500 +2021-03-26 04:39:35,777 epoch 50 - iter 25/50 - loss 2.46748724 - samples/sec: 78.90 - lr: 0.037500 +2021-03-26 04:39:37,667 epoch 50 - iter 30/50 - loss 2.53201131 - samples/sec: 84.77 - lr: 0.037500 +2021-03-26 04:39:39,587 epoch 50 - iter 35/50 - loss 2.58285206 - samples/sec: 83.45 - lr: 0.037500 +2021-03-26 04:39:41,597 epoch 50 - iter 40/50 - loss 2.54398156 - samples/sec: 79.71 - lr: 0.037500 +2021-03-26 04:39:43,507 epoch 50 - iter 45/50 - loss 2.51924728 - samples/sec: 83.86 - lr: 0.037500 +2021-03-26 04:39:45,276 epoch 50 - iter 50/50 - loss 2.56936092 - samples/sec: 90.53 - lr: 0.037500 +2021-03-26 04:39:45,277 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:39:45,277 EPOCH 50 done: loss 2.5694 - lr 0.0375000 +2021-03-26 04:39:46,087 DEV : loss 5.591398239135742 - score 0.9179 +2021-03-26 04:39:46,113 BAD EPOCHS (no improvement): 3 +2021-03-26 04:39:46,114 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:39:48,144 epoch 51 - iter 5/50 - loss 2.60005298 - samples/sec: 78.89 - lr: 0.037500 +2021-03-26 04:39:50,046 epoch 51 - iter 10/50 - loss 2.50737890 - samples/sec: 84.21 - lr: 0.037500 +2021-03-26 04:39:51,950 epoch 51 - iter 15/50 - loss 2.58061322 - samples/sec: 84.13 - lr: 0.037500 +2021-03-26 04:39:53,823 epoch 51 - iter 20/50 - loss 2.49857840 - samples/sec: 85.48 - lr: 0.037500 +2021-03-26 04:39:56,201 epoch 51 - iter 25/50 - loss 2.54270968 - samples/sec: 67.33 - lr: 0.037500 +2021-03-26 04:39:58,075 epoch 51 - iter 30/50 - loss 2.56095430 - samples/sec: 85.51 - lr: 0.037500 +2021-03-26 04:40:00,130 epoch 51 - iter 35/50 - loss 2.56534418 - samples/sec: 77.90 - lr: 0.037500 +2021-03-26 04:40:02,152 epoch 51 - iter 40/50 - loss 2.54329309 - samples/sec: 79.28 - lr: 0.037500 +2021-03-26 04:40:04,114 epoch 51 - iter 45/50 - loss 2.54833703 - samples/sec: 81.60 - lr: 0.037500 +2021-03-26 04:40:05,955 epoch 51 - iter 50/50 - loss 2.54080164 - samples/sec: 87.01 - lr: 0.037500 +2021-03-26 04:40:05,955 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:40:05,955 EPOCH 51 done: loss 2.5408 - lr 0.0375000 +2021-03-26 04:40:06,771 DEV : loss 5.714648246765137 - score 0.916 +2021-03-26 04:40:06,801 BAD EPOCHS (no improvement): 4 +2021-03-26 04:40:06,802 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:40:08,792 epoch 52 - iter 5/50 - loss 2.18448515 - samples/sec: 80.49 - lr: 0.018750 +2021-03-26 04:40:10,804 epoch 52 - iter 10/50 - loss 2.40422935 - samples/sec: 79.58 - lr: 0.018750 +2021-03-26 04:40:12,773 epoch 52 - iter 15/50 - loss 2.46985007 - samples/sec: 81.38 - lr: 0.018750 +2021-03-26 04:40:14,770 epoch 52 - iter 20/50 - loss 2.40598304 - samples/sec: 80.20 - lr: 0.018750 +2021-03-26 04:40:16,798 epoch 52 - iter 25/50 - loss 2.37731241 - samples/sec: 78.96 - lr: 0.018750 +2021-03-26 04:40:18,969 epoch 52 - iter 30/50 - loss 2.41620742 - samples/sec: 73.77 - lr: 0.018750 +2021-03-26 04:40:21,096 epoch 52 - iter 35/50 - loss 2.39733038 - samples/sec: 75.34 - lr: 0.018750 +2021-03-26 04:40:23,180 epoch 52 - iter 40/50 - loss 2.37697385 - samples/sec: 76.83 - lr: 0.018750 +2021-03-26 04:40:25,171 epoch 52 - iter 45/50 - loss 2.41595177 - samples/sec: 80.45 - lr: 0.018750 +2021-03-26 04:40:27,080 epoch 52 - iter 50/50 - loss 2.47739849 - samples/sec: 83.89 - lr: 0.018750 +2021-03-26 04:40:27,081 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:40:27,082 EPOCH 52 done: loss 2.4774 - lr 0.0187500 +2021-03-26 04:40:27,914 DEV : loss 5.6428985595703125 - score 0.9215 +2021-03-26 04:40:27,940 BAD EPOCHS (no improvement): 1 +2021-03-26 04:40:27,941 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:40:29,869 epoch 53 - iter 5/50 - loss 2.80800118 - samples/sec: 83.07 - lr: 0.018750 +2021-03-26 04:40:31,897 epoch 53 - iter 10/50 - loss 2.71635209 - samples/sec: 79.00 - lr: 0.018750 +2021-03-26 04:40:33,762 epoch 53 - iter 15/50 - loss 2.40103800 - samples/sec: 85.88 - lr: 0.018750 +2021-03-26 04:40:35,795 epoch 53 - iter 20/50 - loss 2.41370821 - samples/sec: 78.77 - lr: 0.018750 +2021-03-26 04:40:37,757 epoch 53 - iter 25/50 - loss 2.45257851 - samples/sec: 81.64 - lr: 0.018750 +2021-03-26 04:40:39,823 epoch 53 - iter 30/50 - loss 2.40813193 - samples/sec: 77.52 - lr: 0.018750 +2021-03-26 04:40:41,944 epoch 53 - iter 35/50 - loss 2.44468188 - samples/sec: 75.50 - lr: 0.018750 +2021-03-26 04:40:43,768 epoch 53 - iter 40/50 - loss 2.44019739 - samples/sec: 87.82 - lr: 0.018750 +2021-03-26 04:40:45,736 epoch 53 - iter 45/50 - loss 2.48135226 - samples/sec: 81.40 - lr: 0.018750 +2021-03-26 04:40:47,540 epoch 53 - iter 50/50 - loss 2.48608873 - samples/sec: 88.79 - lr: 0.018750 +2021-03-26 04:40:47,541 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:40:47,541 EPOCH 53 done: loss 2.4861 - lr 0.0187500 +2021-03-26 04:40:48,354 DEV : loss 5.656498908996582 - score 0.9187 +2021-03-26 04:40:48,376 BAD EPOCHS (no improvement): 2 +2021-03-26 04:40:48,376 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:40:50,448 epoch 54 - iter 5/50 - loss 2.60976076 - samples/sec: 77.31 - lr: 0.018750 +2021-03-26 04:40:52,395 epoch 54 - iter 10/50 - loss 2.35027825 - samples/sec: 82.28 - lr: 0.018750 +2021-03-26 04:40:54,351 epoch 54 - iter 15/50 - loss 2.36107490 - samples/sec: 81.90 - lr: 0.018750 +2021-03-26 04:40:56,955 epoch 54 - iter 20/50 - loss 2.33634679 - samples/sec: 61.48 - lr: 0.018750 +2021-03-26 04:40:58,967 epoch 54 - iter 25/50 - loss 2.40109615 - samples/sec: 79.57 - lr: 0.018750 +2021-03-26 04:41:00,997 epoch 54 - iter 30/50 - loss 2.39482352 - samples/sec: 78.91 - lr: 0.018750 +2021-03-26 04:41:03,058 epoch 54 - iter 35/50 - loss 2.35816078 - samples/sec: 77.67 - lr: 0.018750 +2021-03-26 04:41:05,099 epoch 54 - iter 40/50 - loss 2.32958505 - samples/sec: 78.46 - lr: 0.018750 +2021-03-26 04:41:06,923 epoch 54 - iter 45/50 - loss 2.29746844 - samples/sec: 87.84 - lr: 0.018750 +2021-03-26 04:41:08,791 epoch 54 - iter 50/50 - loss 2.40223788 - samples/sec: 85.72 - lr: 0.018750 +2021-03-26 04:41:08,791 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:41:08,792 EPOCH 54 done: loss 2.4022 - lr 0.0187500 +2021-03-26 04:41:09,622 DEV : loss 5.651618957519531 - score 0.9195 +2021-03-26 04:41:09,648 BAD EPOCHS (no improvement): 3 +2021-03-26 04:41:09,649 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:41:11,510 epoch 55 - iter 5/50 - loss 2.56955271 - samples/sec: 86.06 - lr: 0.018750 +2021-03-26 04:41:13,555 epoch 55 - iter 10/50 - loss 2.34034249 - samples/sec: 78.29 - lr: 0.018750 +2021-03-26 04:41:15,529 epoch 55 - iter 15/50 - loss 2.39607155 - samples/sec: 81.13 - lr: 0.018750 +2021-03-26 04:41:17,510 epoch 55 - iter 20/50 - loss 2.40379005 - samples/sec: 80.84 - lr: 0.018750 +2021-03-26 04:41:19,606 epoch 55 - iter 25/50 - loss 2.42211420 - samples/sec: 76.42 - lr: 0.018750 +2021-03-26 04:41:21,654 epoch 55 - iter 30/50 - loss 2.49672683 - samples/sec: 78.15 - lr: 0.018750 +2021-03-26 04:41:23,777 epoch 55 - iter 35/50 - loss 2.52185025 - samples/sec: 75.41 - lr: 0.018750 +2021-03-26 04:41:25,738 epoch 55 - iter 40/50 - loss 2.51877766 - samples/sec: 81.67 - lr: 0.018750 +2021-03-26 04:41:27,698 epoch 55 - iter 45/50 - loss 2.46066902 - samples/sec: 81.76 - lr: 0.018750 +2021-03-26 04:41:29,483 epoch 55 - iter 50/50 - loss 2.42432467 - samples/sec: 89.70 - lr: 0.018750 +2021-03-26 04:41:29,484 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:41:29,484 EPOCH 55 done: loss 2.4243 - lr 0.0187500 +2021-03-26 04:41:30,304 DEV : loss 5.660974025726318 - score 0.9203 +2021-03-26 04:41:30,330 BAD EPOCHS (no improvement): 4 +2021-03-26 04:41:30,331 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:41:32,470 epoch 56 - iter 5/50 - loss 2.67091203 - samples/sec: 74.90 - lr: 0.009375 +2021-03-26 04:41:34,414 epoch 56 - iter 10/50 - loss 2.46447575 - samples/sec: 82.34 - lr: 0.009375 +2021-03-26 04:41:36,460 epoch 56 - iter 15/50 - loss 2.54704790 - samples/sec: 78.26 - lr: 0.009375 +2021-03-26 04:41:38,353 epoch 56 - iter 20/50 - loss 2.60251430 - samples/sec: 84.60 - lr: 0.009375 +2021-03-26 04:41:40,128 epoch 56 - iter 25/50 - loss 2.57725821 - samples/sec: 90.22 - lr: 0.009375 +2021-03-26 04:41:42,056 epoch 56 - iter 30/50 - loss 2.54962186 - samples/sec: 83.10 - lr: 0.009375 +2021-03-26 04:41:44,079 epoch 56 - iter 35/50 - loss 2.56064803 - samples/sec: 79.12 - lr: 0.009375 +2021-03-26 04:41:46,153 epoch 56 - iter 40/50 - loss 2.56441651 - samples/sec: 77.23 - lr: 0.009375 +2021-03-26 04:41:48,222 epoch 56 - iter 45/50 - loss 2.51202908 - samples/sec: 77.36 - lr: 0.009375 +2021-03-26 04:41:50,244 epoch 56 - iter 50/50 - loss 2.49388498 - samples/sec: 79.22 - lr: 0.009375 +2021-03-26 04:41:50,244 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:41:50,245 EPOCH 56 done: loss 2.4939 - lr 0.0093750 +2021-03-26 04:41:51,083 DEV : loss 5.629773139953613 - score 0.9203 +2021-03-26 04:41:51,109 BAD EPOCHS (no improvement): 1 +2021-03-26 04:41:51,109 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:41:53,098 epoch 57 - iter 5/50 - loss 2.37442803 - samples/sec: 80.55 - lr: 0.009375 +2021-03-26 04:41:55,213 epoch 57 - iter 10/50 - loss 2.48838272 - samples/sec: 75.69 - lr: 0.009375 +2021-03-26 04:41:57,093 epoch 57 - iter 15/50 - loss 2.43360175 - samples/sec: 85.25 - lr: 0.009375 +2021-03-26 04:41:58,931 epoch 57 - iter 20/50 - loss 2.46973049 - samples/sec: 87.16 - lr: 0.009375 +2021-03-26 04:42:01,042 epoch 57 - iter 25/50 - loss 2.48110202 - samples/sec: 75.83 - lr: 0.009375 +2021-03-26 04:42:03,177 epoch 57 - iter 30/50 - loss 2.52635504 - samples/sec: 75.02 - lr: 0.009375 +2021-03-26 04:42:05,553 epoch 57 - iter 35/50 - loss 2.52847188 - samples/sec: 67.40 - lr: 0.009375 +2021-03-26 04:42:07,300 epoch 57 - iter 40/50 - loss 2.49505937 - samples/sec: 91.65 - lr: 0.009375 +2021-03-26 04:42:09,169 epoch 57 - iter 45/50 - loss 2.49363049 - samples/sec: 85.67 - lr: 0.009375 +2021-03-26 04:42:11,128 epoch 57 - iter 50/50 - loss 2.52488782 - samples/sec: 81.79 - lr: 0.009375 +2021-03-26 04:42:11,129 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:42:11,130 EPOCH 57 done: loss 2.5249 - lr 0.0093750 +2021-03-26 04:42:12,062 DEV : loss 5.650582313537598 - score 0.9211 +2021-03-26 04:42:12,098 BAD EPOCHS (no improvement): 2 +2021-03-26 04:42:12,099 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:42:14,152 epoch 58 - iter 5/50 - loss 2.54769721 - samples/sec: 77.99 - lr: 0.009375 +2021-03-26 04:42:16,131 epoch 58 - iter 10/50 - loss 2.53803582 - samples/sec: 80.96 - lr: 0.009375 +2021-03-26 04:42:18,098 epoch 58 - iter 15/50 - loss 2.54988208 - samples/sec: 81.40 - lr: 0.009375 +2021-03-26 04:42:19,767 epoch 58 - iter 20/50 - loss 2.41201873 - samples/sec: 95.94 - lr: 0.009375 +2021-03-26 04:42:21,656 epoch 58 - iter 25/50 - loss 2.37423041 - samples/sec: 84.80 - lr: 0.009375 +2021-03-26 04:42:23,469 epoch 58 - iter 30/50 - loss 2.37531073 - samples/sec: 88.32 - lr: 0.009375 +2021-03-26 04:42:25,436 epoch 58 - iter 35/50 - loss 2.32408940 - samples/sec: 81.38 - lr: 0.009375 +2021-03-26 04:42:27,178 epoch 58 - iter 40/50 - loss 2.32021767 - samples/sec: 91.94 - lr: 0.009375 +2021-03-26 04:42:29,348 epoch 58 - iter 45/50 - loss 2.34878608 - samples/sec: 73.79 - lr: 0.009375 +2021-03-26 04:42:31,262 epoch 58 - iter 50/50 - loss 2.37261997 - samples/sec: 83.68 - lr: 0.009375 +2021-03-26 04:42:31,262 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:42:31,263 EPOCH 58 done: loss 2.3726 - lr 0.0093750 +2021-03-26 04:42:32,056 DEV : loss 5.631865501403809 - score 0.9199 +2021-03-26 04:42:32,079 BAD EPOCHS (no improvement): 3 +2021-03-26 04:42:32,080 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:42:34,065 epoch 59 - iter 5/50 - loss 2.36048121 - samples/sec: 80.69 - lr: 0.009375 +2021-03-26 04:42:36,004 epoch 59 - iter 10/50 - loss 2.70518186 - samples/sec: 82.74 - lr: 0.009375 +2021-03-26 04:42:37,866 epoch 59 - iter 15/50 - loss 2.60348249 - samples/sec: 86.04 - lr: 0.009375 +2021-03-26 04:42:39,778 epoch 59 - iter 20/50 - loss 2.60562146 - samples/sec: 83.77 - lr: 0.009375 +2021-03-26 04:42:41,668 epoch 59 - iter 25/50 - loss 2.54358743 - samples/sec: 84.77 - lr: 0.009375 +2021-03-26 04:42:43,594 epoch 59 - iter 30/50 - loss 2.49220409 - samples/sec: 83.17 - lr: 0.009375 +2021-03-26 04:42:45,377 epoch 59 - iter 35/50 - loss 2.49576769 - samples/sec: 89.86 - lr: 0.009375 +2021-03-26 04:42:47,357 epoch 59 - iter 40/50 - loss 2.47423758 - samples/sec: 80.89 - lr: 0.009375 +2021-03-26 04:42:49,368 epoch 59 - iter 45/50 - loss 2.44007614 - samples/sec: 79.62 - lr: 0.009375 +2021-03-26 04:42:51,451 epoch 59 - iter 50/50 - loss 2.43062379 - samples/sec: 76.90 - lr: 0.009375 +2021-03-26 04:42:51,451 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:42:51,452 EPOCH 59 done: loss 2.4306 - lr 0.0093750 +2021-03-26 04:42:52,262 DEV : loss 5.658109188079834 - score 0.9207 +2021-03-26 04:42:52,287 BAD EPOCHS (no improvement): 4 +2021-03-26 04:42:52,288 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:42:54,144 epoch 60 - iter 5/50 - loss 2.35025871 - samples/sec: 86.30 - lr: 0.004687 +2021-03-26 04:42:56,046 epoch 60 - iter 10/50 - loss 2.37897037 - samples/sec: 84.19 - lr: 0.004687 +2021-03-26 04:42:58,067 epoch 60 - iter 15/50 - loss 2.31608614 - samples/sec: 79.26 - lr: 0.004687 +2021-03-26 04:42:59,960 epoch 60 - iter 20/50 - loss 2.26537485 - samples/sec: 84.60 - lr: 0.004687 +2021-03-26 04:43:02,052 epoch 60 - iter 25/50 - loss 2.26644080 - samples/sec: 76.55 - lr: 0.004687 +2021-03-26 04:43:04,049 epoch 60 - iter 30/50 - loss 2.31688050 - samples/sec: 80.19 - lr: 0.004687 +2021-03-26 04:43:05,871 epoch 60 - iter 35/50 - loss 2.28219617 - samples/sec: 87.90 - lr: 0.004687 +2021-03-26 04:43:07,753 epoch 60 - iter 40/50 - loss 2.31520236 - samples/sec: 85.11 - lr: 0.004687 +2021-03-26 04:43:09,875 epoch 60 - iter 45/50 - loss 2.37652651 - samples/sec: 75.44 - lr: 0.004687 +2021-03-26 04:43:11,657 epoch 60 - iter 50/50 - loss 2.42843126 - samples/sec: 89.89 - lr: 0.004687 +2021-03-26 04:43:11,657 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:43:11,657 EPOCH 60 done: loss 2.4284 - lr 0.0046875 +2021-03-26 04:43:12,453 DEV : loss 5.651933193206787 - score 0.9215 +2021-03-26 04:43:12,474 BAD EPOCHS (no improvement): 1 +2021-03-26 04:43:12,475 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:43:14,399 epoch 61 - iter 5/50 - loss 2.08282530 - samples/sec: 83.25 - lr: 0.004687 +2021-03-26 04:43:16,399 epoch 61 - iter 10/50 - loss 2.24820324 - samples/sec: 80.05 - lr: 0.004687 +2021-03-26 04:43:18,466 epoch 61 - iter 15/50 - loss 2.41197851 - samples/sec: 77.50 - lr: 0.004687 +2021-03-26 04:43:20,499 epoch 61 - iter 20/50 - loss 2.40561969 - samples/sec: 78.75 - lr: 0.004687 +2021-03-26 04:43:22,264 epoch 61 - iter 25/50 - loss 2.32103359 - samples/sec: 90.74 - lr: 0.004687 +2021-03-26 04:43:24,306 epoch 61 - iter 30/50 - loss 2.34994412 - samples/sec: 78.46 - lr: 0.004687 +2021-03-26 04:43:26,241 epoch 61 - iter 35/50 - loss 2.32774382 - samples/sec: 82.75 - lr: 0.004687 +2021-03-26 04:43:28,374 epoch 61 - iter 40/50 - loss 2.37477439 - samples/sec: 75.08 - lr: 0.004687 +2021-03-26 04:43:30,341 epoch 61 - iter 45/50 - loss 2.39251536 - samples/sec: 81.44 - lr: 0.004687 +2021-03-26 04:43:32,049 epoch 61 - iter 50/50 - loss 2.39368558 - samples/sec: 93.81 - lr: 0.004687 +2021-03-26 04:43:32,050 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:43:32,050 EPOCH 61 done: loss 2.3937 - lr 0.0046875 +2021-03-26 04:43:32,862 DEV : loss 5.637948513031006 - score 0.9219 +2021-03-26 04:43:32,887 BAD EPOCHS (no improvement): 0 +2021-03-26 04:43:42,750 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:43:44,828 epoch 62 - iter 5/50 - loss 2.27241840 - samples/sec: 77.08 - lr: 0.004687 +2021-03-26 04:43:46,844 epoch 62 - iter 10/50 - loss 2.29186729 - samples/sec: 79.50 - lr: 0.004687 +2021-03-26 04:43:48,784 epoch 62 - iter 15/50 - loss 2.39521653 - samples/sec: 82.55 - lr: 0.004687 +2021-03-26 04:43:50,634 epoch 62 - iter 20/50 - loss 2.37930581 - samples/sec: 86.59 - lr: 0.004687 +2021-03-26 04:43:52,580 epoch 62 - iter 25/50 - loss 2.40283502 - samples/sec: 82.29 - lr: 0.004687 +2021-03-26 04:43:54,497 epoch 62 - iter 30/50 - loss 2.36301094 - samples/sec: 83.55 - lr: 0.004687 +2021-03-26 04:43:56,460 epoch 62 - iter 35/50 - loss 2.35278542 - samples/sec: 81.57 - lr: 0.004687 +2021-03-26 04:43:58,571 epoch 62 - iter 40/50 - loss 2.35601684 - samples/sec: 75.85 - lr: 0.004687 +2021-03-26 04:44:00,361 epoch 62 - iter 45/50 - loss 2.27771665 - samples/sec: 89.47 - lr: 0.004687 +2021-03-26 04:44:02,302 epoch 62 - iter 50/50 - loss 2.30656911 - samples/sec: 82.48 - lr: 0.004687 +2021-03-26 04:44:02,303 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:44:02,303 EPOCH 62 done: loss 2.3066 - lr 0.0046875 +2021-03-26 04:44:03,125 DEV : loss 5.627078056335449 - score 0.9227 +2021-03-26 04:44:03,150 BAD EPOCHS (no improvement): 0 +2021-03-26 04:44:12,835 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:44:14,853 epoch 63 - iter 5/50 - loss 2.72502465 - samples/sec: 79.39 - lr: 0.004687 +2021-03-26 04:44:16,844 epoch 63 - iter 10/50 - loss 2.34835476 - samples/sec: 80.43 - lr: 0.004687 +2021-03-26 04:44:18,808 epoch 63 - iter 15/50 - loss 2.35418735 - samples/sec: 81.55 - lr: 0.004687 +2021-03-26 04:44:20,744 epoch 63 - iter 20/50 - loss 2.42223947 - samples/sec: 82.74 - lr: 0.004687 +2021-03-26 04:44:22,567 epoch 63 - iter 25/50 - loss 2.34308921 - samples/sec: 87.86 - lr: 0.004687 +2021-03-26 04:44:24,548 epoch 63 - iter 30/50 - loss 2.36702642 - samples/sec: 80.82 - lr: 0.004687 +2021-03-26 04:44:26,579 epoch 63 - iter 35/50 - loss 2.38747502 - samples/sec: 78.86 - lr: 0.004687 +2021-03-26 04:44:28,479 epoch 63 - iter 40/50 - loss 2.41883632 - samples/sec: 84.27 - lr: 0.004687 +2021-03-26 04:44:30,417 epoch 63 - iter 45/50 - loss 2.35734841 - samples/sec: 82.64 - lr: 0.004687 +2021-03-26 04:44:32,130 epoch 63 - iter 50/50 - loss 2.43212230 - samples/sec: 93.49 - lr: 0.004687 +2021-03-26 04:44:32,131 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:44:32,131 EPOCH 63 done: loss 2.4321 - lr 0.0046875 +2021-03-26 04:44:32,941 DEV : loss 5.634641170501709 - score 0.9219 +2021-03-26 04:44:32,959 BAD EPOCHS (no improvement): 1 +2021-03-26 04:44:32,959 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:44:34,944 epoch 64 - iter 5/50 - loss 2.49137278 - samples/sec: 80.71 - lr: 0.004687 +2021-03-26 04:44:36,845 epoch 64 - iter 10/50 - loss 2.36707425 - samples/sec: 84.24 - lr: 0.004687 +2021-03-26 04:44:38,788 epoch 64 - iter 15/50 - loss 2.28359599 - samples/sec: 82.43 - lr: 0.004687 +2021-03-26 04:44:40,820 epoch 64 - iter 20/50 - loss 2.38758727 - samples/sec: 78.82 - lr: 0.004687 +2021-03-26 04:44:42,799 epoch 64 - iter 25/50 - loss 2.38408761 - samples/sec: 80.93 - lr: 0.004687 +2021-03-26 04:44:44,723 epoch 64 - iter 30/50 - loss 2.34029110 - samples/sec: 83.23 - lr: 0.004687 +2021-03-26 04:44:46,698 epoch 64 - iter 35/50 - loss 2.27839816 - samples/sec: 81.06 - lr: 0.004687 +2021-03-26 04:44:48,669 epoch 64 - iter 40/50 - loss 2.30497065 - samples/sec: 81.25 - lr: 0.004687 +2021-03-26 04:44:50,880 epoch 64 - iter 45/50 - loss 2.31095112 - samples/sec: 72.44 - lr: 0.004687 +2021-03-26 04:44:52,645 epoch 64 - iter 50/50 - loss 2.27450335 - samples/sec: 90.75 - lr: 0.004687 +2021-03-26 04:44:52,646 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:44:52,646 EPOCH 64 done: loss 2.2745 - lr 0.0046875 +2021-03-26 04:44:53,447 DEV : loss 5.627133846282959 - score 0.9219 +2021-03-26 04:44:53,465 BAD EPOCHS (no improvement): 2 +2021-03-26 04:44:53,466 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:44:55,469 epoch 65 - iter 5/50 - loss 2.32387009 - samples/sec: 79.94 - lr: 0.004687 +2021-03-26 04:44:57,327 epoch 65 - iter 10/50 - loss 2.41782620 - samples/sec: 86.23 - lr: 0.004687 +2021-03-26 04:44:59,270 epoch 65 - iter 15/50 - loss 2.43123651 - samples/sec: 82.47 - lr: 0.004687 +2021-03-26 04:45:01,246 epoch 65 - iter 20/50 - loss 2.31962968 - samples/sec: 81.04 - lr: 0.004687 +2021-03-26 04:45:03,205 epoch 65 - iter 25/50 - loss 2.49893980 - samples/sec: 81.73 - lr: 0.004687 +2021-03-26 04:45:05,153 epoch 65 - iter 30/50 - loss 2.51514649 - samples/sec: 82.21 - lr: 0.004687 +2021-03-26 04:45:07,174 epoch 65 - iter 35/50 - loss 2.49580066 - samples/sec: 79.23 - lr: 0.004687 +2021-03-26 04:45:09,077 epoch 65 - iter 40/50 - loss 2.48866832 - samples/sec: 84.16 - lr: 0.004687 +2021-03-26 04:45:11,216 epoch 65 - iter 45/50 - loss 2.44491422 - samples/sec: 74.89 - lr: 0.004687 +2021-03-26 04:45:13,019 epoch 65 - iter 50/50 - loss 2.40319876 - samples/sec: 88.88 - lr: 0.004687 +2021-03-26 04:45:13,019 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:45:13,020 EPOCH 65 done: loss 2.4032 - lr 0.0046875 +2021-03-26 04:45:13,845 DEV : loss 5.6327924728393555 - score 0.9211 +2021-03-26 04:45:13,862 BAD EPOCHS (no improvement): 3 +2021-03-26 04:45:13,863 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:45:15,852 epoch 66 - iter 5/50 - loss 2.18675711 - samples/sec: 80.51 - lr: 0.004687 +2021-03-26 04:45:17,915 epoch 66 - iter 10/50 - loss 2.41399018 - samples/sec: 77.62 - lr: 0.004687 +2021-03-26 04:45:19,903 epoch 66 - iter 15/50 - loss 2.34998701 - samples/sec: 80.58 - lr: 0.004687 +2021-03-26 04:45:21,768 epoch 66 - iter 20/50 - loss 2.41503422 - samples/sec: 85.86 - lr: 0.004687 +2021-03-26 04:45:23,785 epoch 66 - iter 25/50 - loss 2.48407470 - samples/sec: 79.36 - lr: 0.004687 +2021-03-26 04:45:25,725 epoch 66 - iter 30/50 - loss 2.56181589 - samples/sec: 82.56 - lr: 0.004687 +2021-03-26 04:45:27,754 epoch 66 - iter 35/50 - loss 2.55085475 - samples/sec: 78.96 - lr: 0.004687 +2021-03-26 04:45:29,807 epoch 66 - iter 40/50 - loss 2.59409171 - samples/sec: 78.00 - lr: 0.004687 +2021-03-26 04:45:31,803 epoch 66 - iter 45/50 - loss 2.54425696 - samples/sec: 80.21 - lr: 0.004687 +2021-03-26 04:45:33,703 epoch 66 - iter 50/50 - loss 2.56617797 - samples/sec: 84.30 - lr: 0.004687 +2021-03-26 04:45:33,704 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:45:33,704 EPOCH 66 done: loss 2.5662 - lr 0.0046875 +2021-03-26 04:45:34,510 DEV : loss 5.638400077819824 - score 0.9219 +2021-03-26 04:45:34,536 BAD EPOCHS (no improvement): 4 +2021-03-26 04:45:34,536 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:45:36,572 epoch 67 - iter 5/50 - loss 2.58468990 - samples/sec: 78.70 - lr: 0.002344 +2021-03-26 04:45:38,449 epoch 67 - iter 10/50 - loss 2.50182832 - samples/sec: 85.32 - lr: 0.002344 +2021-03-26 04:45:40,440 epoch 67 - iter 15/50 - loss 2.59449295 - samples/sec: 80.43 - lr: 0.002344 +2021-03-26 04:45:42,515 epoch 67 - iter 20/50 - loss 2.48832155 - samples/sec: 77.18 - lr: 0.002344 +2021-03-26 04:45:44,387 epoch 67 - iter 25/50 - loss 2.36310293 - samples/sec: 85.54 - lr: 0.002344 +2021-03-26 04:45:46,359 epoch 67 - iter 30/50 - loss 2.38655980 - samples/sec: 81.23 - lr: 0.002344 +2021-03-26 04:45:48,268 epoch 67 - iter 35/50 - loss 2.30594007 - samples/sec: 83.87 - lr: 0.002344 +2021-03-26 04:45:50,207 epoch 67 - iter 40/50 - loss 2.31153803 - samples/sec: 82.57 - lr: 0.002344 +2021-03-26 04:45:52,310 epoch 67 - iter 45/50 - loss 2.30876457 - samples/sec: 76.18 - lr: 0.002344 +2021-03-26 04:45:54,232 epoch 67 - iter 50/50 - loss 2.38392017 - samples/sec: 83.37 - lr: 0.002344 +2021-03-26 04:45:54,233 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:45:54,234 EPOCH 67 done: loss 2.3839 - lr 0.0023437 +2021-03-26 04:45:55,041 DEV : loss 5.640754699707031 - score 0.9223 +2021-03-26 04:45:55,059 BAD EPOCHS (no improvement): 1 +2021-03-26 04:45:55,060 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:45:57,046 epoch 68 - iter 5/50 - loss 2.16239529 - samples/sec: 80.64 - lr: 0.002344 +2021-03-26 04:45:58,987 epoch 68 - iter 10/50 - loss 2.30322955 - samples/sec: 82.52 - lr: 0.002344 +2021-03-26 04:46:00,860 epoch 68 - iter 15/50 - loss 2.18448377 - samples/sec: 85.58 - lr: 0.002344 +2021-03-26 04:46:02,924 epoch 68 - iter 20/50 - loss 2.20179555 - samples/sec: 77.58 - lr: 0.002344 +2021-03-26 04:46:05,128 epoch 68 - iter 25/50 - loss 2.22976829 - samples/sec: 72.65 - lr: 0.002344 +2021-03-26 04:46:07,365 epoch 68 - iter 30/50 - loss 2.25753242 - samples/sec: 71.55 - lr: 0.002344 +2021-03-26 04:46:09,277 epoch 68 - iter 35/50 - loss 2.28681386 - samples/sec: 83.78 - lr: 0.002344 +2021-03-26 04:46:11,277 epoch 68 - iter 40/50 - loss 2.32609110 - samples/sec: 80.06 - lr: 0.002344 +2021-03-26 04:46:13,328 epoch 68 - iter 45/50 - loss 2.32432895 - samples/sec: 78.07 - lr: 0.002344 +2021-03-26 04:46:15,228 epoch 68 - iter 50/50 - loss 2.35555371 - samples/sec: 84.31 - lr: 0.002344 +2021-03-26 04:46:15,228 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:46:15,228 EPOCH 68 done: loss 2.3556 - lr 0.0023437 +2021-03-26 04:46:16,039 DEV : loss 5.634337425231934 - score 0.9219 +2021-03-26 04:46:16,058 BAD EPOCHS (no improvement): 2 +2021-03-26 04:46:16,058 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:46:18,204 epoch 69 - iter 5/50 - loss 2.39963605 - samples/sec: 74.63 - lr: 0.002344 +2021-03-26 04:46:20,128 epoch 69 - iter 10/50 - loss 2.52453003 - samples/sec: 83.23 - lr: 0.002344 +2021-03-26 04:46:22,015 epoch 69 - iter 15/50 - loss 2.39732469 - samples/sec: 84.88 - lr: 0.002344 +2021-03-26 04:46:24,063 epoch 69 - iter 20/50 - loss 2.45164004 - samples/sec: 78.19 - lr: 0.002344 +2021-03-26 04:46:26,350 epoch 69 - iter 25/50 - loss 2.34485009 - samples/sec: 70.01 - lr: 0.002344 +2021-03-26 04:46:28,440 epoch 69 - iter 30/50 - loss 2.39694957 - samples/sec: 76.65 - lr: 0.002344 +2021-03-26 04:46:30,466 epoch 69 - iter 35/50 - loss 2.38782328 - samples/sec: 79.05 - lr: 0.002344 +2021-03-26 04:46:32,426 epoch 69 - iter 40/50 - loss 2.42859703 - samples/sec: 81.72 - lr: 0.002344 +2021-03-26 04:46:34,551 epoch 69 - iter 45/50 - loss 2.44324458 - samples/sec: 75.37 - lr: 0.002344 +2021-03-26 04:46:36,287 epoch 69 - iter 50/50 - loss 2.40381415 - samples/sec: 92.26 - lr: 0.002344 +2021-03-26 04:46:36,287 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:46:36,288 EPOCH 69 done: loss 2.4038 - lr 0.0023437 +2021-03-26 04:46:37,101 DEV : loss 5.629244804382324 - score 0.9227 +2021-03-26 04:46:37,128 BAD EPOCHS (no improvement): 3 +2021-03-26 04:46:37,128 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:46:39,318 epoch 70 - iter 5/50 - loss 2.64318256 - samples/sec: 73.14 - lr: 0.002344 +2021-03-26 04:46:41,224 epoch 70 - iter 10/50 - loss 2.71038888 - samples/sec: 84.09 - lr: 0.002344 +2021-03-26 04:46:43,243 epoch 70 - iter 15/50 - loss 2.52488495 - samples/sec: 79.31 - lr: 0.002344 +2021-03-26 04:46:45,187 epoch 70 - iter 20/50 - loss 2.47664029 - samples/sec: 82.37 - lr: 0.002344 +2021-03-26 04:46:47,639 epoch 70 - iter 25/50 - loss 2.42149183 - samples/sec: 65.34 - lr: 0.002344 +2021-03-26 04:46:49,665 epoch 70 - iter 30/50 - loss 2.40650520 - samples/sec: 79.08 - lr: 0.002344 +2021-03-26 04:46:51,770 epoch 70 - iter 35/50 - loss 2.41927083 - samples/sec: 76.09 - lr: 0.002344 +2021-03-26 04:46:53,764 epoch 70 - iter 40/50 - loss 2.44539106 - samples/sec: 80.32 - lr: 0.002344 +2021-03-26 04:46:56,201 epoch 70 - iter 45/50 - loss 2.41105876 - samples/sec: 65.72 - lr: 0.002344 +2021-03-26 04:46:58,028 epoch 70 - iter 50/50 - loss 2.42444482 - samples/sec: 87.70 - lr: 0.002344 +2021-03-26 04:46:58,028 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:46:58,029 EPOCH 70 done: loss 2.4244 - lr 0.0023437 +2021-03-26 04:46:58,886 DEV : loss 5.630977630615234 - score 0.9227 +2021-03-26 04:46:58,911 BAD EPOCHS (no improvement): 4 +2021-03-26 04:46:58,912 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:47:00,991 epoch 71 - iter 5/50 - loss 2.40335622 - samples/sec: 77.04 - lr: 0.001172 +2021-03-26 04:47:02,794 epoch 71 - iter 10/50 - loss 2.38454808 - samples/sec: 88.90 - lr: 0.001172 +2021-03-26 04:47:04,708 epoch 71 - iter 15/50 - loss 2.48061834 - samples/sec: 83.68 - lr: 0.001172 +2021-03-26 04:47:06,716 epoch 71 - iter 20/50 - loss 2.37713302 - samples/sec: 79.75 - lr: 0.001172 +2021-03-26 04:47:08,771 epoch 71 - iter 25/50 - loss 2.34294545 - samples/sec: 77.92 - lr: 0.001172 +2021-03-26 04:47:10,700 epoch 71 - iter 30/50 - loss 2.42969246 - samples/sec: 83.04 - lr: 0.001172 +2021-03-26 04:47:12,515 epoch 71 - iter 35/50 - loss 2.39980226 - samples/sec: 88.26 - lr: 0.001172 +2021-03-26 04:47:14,409 epoch 71 - iter 40/50 - loss 2.35062600 - samples/sec: 84.58 - lr: 0.001172 +2021-03-26 04:47:16,304 epoch 71 - iter 45/50 - loss 2.38621088 - samples/sec: 84.48 - lr: 0.001172 +2021-03-26 04:47:18,211 epoch 71 - iter 50/50 - loss 2.40086455 - samples/sec: 83.99 - lr: 0.001172 +2021-03-26 04:47:18,212 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:47:18,212 EPOCH 71 done: loss 2.4009 - lr 0.0011719 +2021-03-26 04:47:18,999 DEV : loss 5.630173683166504 - score 0.9231 +2021-03-26 04:47:19,025 BAD EPOCHS (no improvement): 0 +2021-03-26 04:47:28,586 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:47:30,506 epoch 72 - iter 5/50 - loss 2.07270017 - samples/sec: 83.44 - lr: 0.001172 +2021-03-26 04:47:32,500 epoch 72 - iter 10/50 - loss 2.45771600 - samples/sec: 80.36 - lr: 0.001172 +2021-03-26 04:47:34,636 epoch 72 - iter 15/50 - loss 2.47021635 - samples/sec: 74.99 - lr: 0.001172 +2021-03-26 04:47:36,580 epoch 72 - iter 20/50 - loss 2.45928237 - samples/sec: 82.37 - lr: 0.001172 +2021-03-26 04:47:38,553 epoch 72 - iter 25/50 - loss 2.39187438 - samples/sec: 81.18 - lr: 0.001172 +2021-03-26 04:47:40,357 epoch 72 - iter 30/50 - loss 2.37344064 - samples/sec: 88.81 - lr: 0.001172 +2021-03-26 04:47:42,555 epoch 72 - iter 35/50 - loss 2.39183445 - samples/sec: 72.86 - lr: 0.001172 +2021-03-26 04:47:44,500 epoch 72 - iter 40/50 - loss 2.40346884 - samples/sec: 82.29 - lr: 0.001172 +2021-03-26 04:47:46,573 epoch 72 - iter 45/50 - loss 2.39540144 - samples/sec: 77.26 - lr: 0.001172 +2021-03-26 04:47:48,432 epoch 72 - iter 50/50 - loss 2.39621652 - samples/sec: 86.20 - lr: 0.001172 +2021-03-26 04:47:48,433 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:47:48,433 EPOCH 72 done: loss 2.3962 - lr 0.0011719 +2021-03-26 04:47:49,272 DEV : loss 5.62816047668457 - score 0.9223 +2021-03-26 04:47:49,301 BAD EPOCHS (no improvement): 1 +2021-03-26 04:47:49,302 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:47:51,341 epoch 73 - iter 5/50 - loss 2.44974518 - samples/sec: 78.55 - lr: 0.001172 +2021-03-26 04:47:53,368 epoch 73 - iter 10/50 - loss 2.35539858 - samples/sec: 79.03 - lr: 0.001172 +2021-03-26 04:47:55,307 epoch 73 - iter 15/50 - loss 2.29412109 - samples/sec: 82.56 - lr: 0.001172 +2021-03-26 04:47:57,467 epoch 73 - iter 20/50 - loss 2.45073715 - samples/sec: 74.15 - lr: 0.001172 +2021-03-26 04:47:59,439 epoch 73 - iter 25/50 - loss 2.43627182 - samples/sec: 81.18 - lr: 0.001172 +2021-03-26 04:48:01,643 epoch 73 - iter 30/50 - loss 2.40200396 - samples/sec: 72.64 - lr: 0.001172 +2021-03-26 04:48:03,974 epoch 73 - iter 35/50 - loss 2.32076040 - samples/sec: 68.70 - lr: 0.001172 +2021-03-26 04:48:05,935 epoch 73 - iter 40/50 - loss 2.28614058 - samples/sec: 81.67 - lr: 0.001172 +2021-03-26 04:48:08,167 epoch 73 - iter 45/50 - loss 2.32075121 - samples/sec: 71.74 - lr: 0.001172 +2021-03-26 04:48:10,007 epoch 73 - iter 50/50 - loss 2.29757083 - samples/sec: 87.00 - lr: 0.001172 +2021-03-26 04:48:10,008 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:48:10,008 EPOCH 73 done: loss 2.2976 - lr 0.0011719 +2021-03-26 04:48:10,837 DEV : loss 5.6278395652771 - score 0.9223 +2021-03-26 04:48:10,857 BAD EPOCHS (no improvement): 2 +2021-03-26 04:48:10,857 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:48:12,812 epoch 74 - iter 5/50 - loss 1.83286681 - samples/sec: 81.90 - lr: 0.001172 +2021-03-26 04:48:14,874 epoch 74 - iter 10/50 - loss 2.12402097 - samples/sec: 77.68 - lr: 0.001172 +2021-03-26 04:48:16,757 epoch 74 - iter 15/50 - loss 2.21711052 - samples/sec: 85.10 - lr: 0.001172 +2021-03-26 04:48:18,867 epoch 74 - iter 20/50 - loss 2.23210856 - samples/sec: 75.88 - lr: 0.001172 +2021-03-26 04:48:20,803 epoch 74 - iter 25/50 - loss 2.21322096 - samples/sec: 82.73 - lr: 0.001172 +2021-03-26 04:48:22,924 epoch 74 - iter 30/50 - loss 2.25693878 - samples/sec: 75.48 - lr: 0.001172 +2021-03-26 04:48:24,900 epoch 74 - iter 35/50 - loss 2.25561285 - samples/sec: 81.07 - lr: 0.001172 +2021-03-26 04:48:26,926 epoch 74 - iter 40/50 - loss 2.30016709 - samples/sec: 79.03 - lr: 0.001172 +2021-03-26 04:48:28,873 epoch 74 - iter 45/50 - loss 2.32310001 - samples/sec: 82.27 - lr: 0.001172 +2021-03-26 04:48:30,848 epoch 74 - iter 50/50 - loss 2.28137210 - samples/sec: 81.07 - lr: 0.001172 +2021-03-26 04:48:30,849 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:48:30,849 EPOCH 74 done: loss 2.2814 - lr 0.0011719 +2021-03-26 04:48:31,665 DEV : loss 5.626428127288818 - score 0.9227 +2021-03-26 04:48:31,691 BAD EPOCHS (no improvement): 3 +2021-03-26 04:48:31,692 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:48:33,724 epoch 75 - iter 5/50 - loss 2.25622938 - samples/sec: 78.81 - lr: 0.001172 +2021-03-26 04:48:35,719 epoch 75 - iter 10/50 - loss 2.31577545 - samples/sec: 80.29 - lr: 0.001172 +2021-03-26 04:48:37,656 epoch 75 - iter 15/50 - loss 2.28079580 - samples/sec: 82.73 - lr: 0.001172 +2021-03-26 04:48:39,560 epoch 75 - iter 20/50 - loss 2.29470981 - samples/sec: 84.09 - lr: 0.001172 +2021-03-26 04:48:41,617 epoch 75 - iter 25/50 - loss 2.29392951 - samples/sec: 77.84 - lr: 0.001172 +2021-03-26 04:48:43,732 epoch 75 - iter 30/50 - loss 2.40328041 - samples/sec: 75.71 - lr: 0.001172 +2021-03-26 04:48:45,581 epoch 75 - iter 35/50 - loss 2.35129993 - samples/sec: 86.61 - lr: 0.001172 +2021-03-26 04:48:47,623 epoch 75 - iter 40/50 - loss 2.34900579 - samples/sec: 78.42 - lr: 0.001172 +2021-03-26 04:48:49,596 epoch 75 - iter 45/50 - loss 2.37384658 - samples/sec: 81.22 - lr: 0.001172 +2021-03-26 04:48:51,544 epoch 75 - iter 50/50 - loss 2.37896151 - samples/sec: 82.21 - lr: 0.001172 +2021-03-26 04:48:51,545 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:48:51,545 EPOCH 75 done: loss 2.3790 - lr 0.0011719 +2021-03-26 04:48:52,366 DEV : loss 5.630959987640381 - score 0.9223 +2021-03-26 04:48:52,392 BAD EPOCHS (no improvement): 4 +2021-03-26 04:48:52,393 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:48:54,461 epoch 76 - iter 5/50 - loss 2.13097723 - samples/sec: 77.45 - lr: 0.000586 +2021-03-26 04:48:56,410 epoch 76 - iter 10/50 - loss 2.27908762 - samples/sec: 82.15 - lr: 0.000586 +2021-03-26 04:48:58,379 epoch 76 - iter 15/50 - loss 2.36238278 - samples/sec: 81.35 - lr: 0.000586 +2021-03-26 04:49:00,472 epoch 76 - iter 20/50 - loss 2.38692014 - samples/sec: 76.53 - lr: 0.000586 +2021-03-26 04:49:02,415 epoch 76 - iter 25/50 - loss 2.39408958 - samples/sec: 82.47 - lr: 0.000586 +2021-03-26 04:49:04,457 epoch 76 - iter 30/50 - loss 2.45271692 - samples/sec: 78.41 - lr: 0.000586 +2021-03-26 04:49:06,291 epoch 76 - iter 35/50 - loss 2.41970376 - samples/sec: 87.34 - lr: 0.000586 +2021-03-26 04:49:08,060 epoch 76 - iter 40/50 - loss 2.39700398 - samples/sec: 90.57 - lr: 0.000586 +2021-03-26 04:49:10,153 epoch 76 - iter 45/50 - loss 2.42808259 - samples/sec: 76.57 - lr: 0.000586 +2021-03-26 04:49:12,061 epoch 76 - iter 50/50 - loss 2.42875048 - samples/sec: 83.94 - lr: 0.000586 +2021-03-26 04:49:12,062 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:49:12,062 EPOCH 76 done: loss 2.4288 - lr 0.0005859 +2021-03-26 04:49:12,880 DEV : loss 5.630477428436279 - score 0.9219 +2021-03-26 04:49:12,906 BAD EPOCHS (no improvement): 1 +2021-03-26 04:49:12,907 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:49:14,791 epoch 77 - iter 5/50 - loss 1.92447205 - samples/sec: 85.01 - lr: 0.000586 +2021-03-26 04:49:16,874 epoch 77 - iter 10/50 - loss 2.23509965 - samples/sec: 76.87 - lr: 0.000586 +2021-03-26 04:49:18,971 epoch 77 - iter 15/50 - loss 2.45340130 - samples/sec: 76.36 - lr: 0.000586 +2021-03-26 04:49:20,915 epoch 77 - iter 20/50 - loss 2.32647941 - samples/sec: 82.38 - lr: 0.000586 +2021-03-26 04:49:22,837 epoch 77 - iter 25/50 - loss 2.29459015 - samples/sec: 83.34 - lr: 0.000586 +2021-03-26 04:49:24,741 epoch 77 - iter 30/50 - loss 2.29548467 - samples/sec: 84.12 - lr: 0.000586 +2021-03-26 04:49:26,796 epoch 77 - iter 35/50 - loss 2.29987510 - samples/sec: 77.94 - lr: 0.000586 +2021-03-26 04:49:28,659 epoch 77 - iter 40/50 - loss 2.34307029 - samples/sec: 86.02 - lr: 0.000586 +2021-03-26 04:49:30,601 epoch 77 - iter 45/50 - loss 2.45157159 - samples/sec: 82.45 - lr: 0.000586 +2021-03-26 04:49:32,374 epoch 77 - iter 50/50 - loss 2.42382906 - samples/sec: 90.34 - lr: 0.000586 +2021-03-26 04:49:32,375 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:49:32,375 EPOCH 77 done: loss 2.4238 - lr 0.0005859 +2021-03-26 04:49:33,174 DEV : loss 5.631059169769287 - score 0.9219 +2021-03-26 04:49:33,198 BAD EPOCHS (no improvement): 2 +2021-03-26 04:49:33,199 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:49:35,186 epoch 78 - iter 5/50 - loss 2.57229795 - samples/sec: 80.60 - lr: 0.000586 +2021-03-26 04:49:37,103 epoch 78 - iter 10/50 - loss 2.69179895 - samples/sec: 83.59 - lr: 0.000586 +2021-03-26 04:49:39,270 epoch 78 - iter 15/50 - loss 2.49594205 - samples/sec: 73.89 - lr: 0.000586 +2021-03-26 04:49:41,248 epoch 78 - iter 20/50 - loss 2.53078225 - samples/sec: 81.16 - lr: 0.000586 +2021-03-26 04:49:43,871 epoch 78 - iter 25/50 - loss 2.45043825 - samples/sec: 61.05 - lr: 0.000586 +2021-03-26 04:49:46,014 epoch 78 - iter 30/50 - loss 2.41474845 - samples/sec: 74.72 - lr: 0.000586 +2021-03-26 04:49:48,078 epoch 78 - iter 35/50 - loss 2.42673413 - samples/sec: 77.56 - lr: 0.000586 +2021-03-26 04:49:50,134 epoch 78 - iter 40/50 - loss 2.39748089 - samples/sec: 77.90 - lr: 0.000586 +2021-03-26 04:49:52,214 epoch 78 - iter 45/50 - loss 2.35223733 - samples/sec: 76.97 - lr: 0.000586 +2021-03-26 04:49:54,078 epoch 78 - iter 50/50 - loss 2.39032073 - samples/sec: 85.94 - lr: 0.000586 +2021-03-26 04:49:54,078 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:49:54,079 EPOCH 78 done: loss 2.3903 - lr 0.0005859 +2021-03-26 04:49:54,880 DEV : loss 5.631186485290527 - score 0.9219 +2021-03-26 04:49:54,905 BAD EPOCHS (no improvement): 3 +2021-03-26 04:49:54,906 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:49:56,860 epoch 79 - iter 5/50 - loss 2.32237132 - samples/sec: 81.96 - lr: 0.000586 +2021-03-26 04:49:58,699 epoch 79 - iter 10/50 - loss 2.59322809 - samples/sec: 87.15 - lr: 0.000586 +2021-03-26 04:50:00,731 epoch 79 - iter 15/50 - loss 2.58026641 - samples/sec: 78.79 - lr: 0.000586 +2021-03-26 04:50:02,869 epoch 79 - iter 20/50 - loss 2.56619343 - samples/sec: 74.92 - lr: 0.000586 +2021-03-26 04:50:05,041 epoch 79 - iter 25/50 - loss 2.51247024 - samples/sec: 73.74 - lr: 0.000586 +2021-03-26 04:50:06,903 epoch 79 - iter 30/50 - loss 2.44964033 - samples/sec: 86.01 - lr: 0.000586 +2021-03-26 04:50:08,977 epoch 79 - iter 35/50 - loss 2.48183150 - samples/sec: 77.23 - lr: 0.000586 +2021-03-26 04:50:11,005 epoch 79 - iter 40/50 - loss 2.50194048 - samples/sec: 78.93 - lr: 0.000586 +2021-03-26 04:50:12,895 epoch 79 - iter 45/50 - loss 2.50711123 - samples/sec: 84.78 - lr: 0.000586 +2021-03-26 04:50:14,643 epoch 79 - iter 50/50 - loss 2.52246664 - samples/sec: 91.65 - lr: 0.000586 +2021-03-26 04:50:14,644 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:50:14,644 EPOCH 79 done: loss 2.5225 - lr 0.0005859 +2021-03-26 04:50:15,449 DEV : loss 5.631038188934326 - score 0.9219 +2021-03-26 04:50:15,467 BAD EPOCHS (no improvement): 4 +2021-03-26 04:50:15,468 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:50:17,448 epoch 80 - iter 5/50 - loss 2.07633672 - samples/sec: 80.89 - lr: 0.000293 +2021-03-26 04:50:19,470 epoch 80 - iter 10/50 - loss 2.18866901 - samples/sec: 79.23 - lr: 0.000293 +2021-03-26 04:50:21,519 epoch 80 - iter 15/50 - loss 2.38815840 - samples/sec: 78.15 - lr: 0.000293 +2021-03-26 04:50:23,621 epoch 80 - iter 20/50 - loss 2.39637839 - samples/sec: 76.16 - lr: 0.000293 +2021-03-26 04:50:25,735 epoch 80 - iter 25/50 - loss 2.34935032 - samples/sec: 75.78 - lr: 0.000293 +2021-03-26 04:50:27,678 epoch 80 - iter 30/50 - loss 2.37969813 - samples/sec: 82.43 - lr: 0.000293 +2021-03-26 04:50:29,757 epoch 80 - iter 35/50 - loss 2.39405797 - samples/sec: 76.99 - lr: 0.000293 +2021-03-26 04:50:31,573 epoch 80 - iter 40/50 - loss 2.35879222 - samples/sec: 88.17 - lr: 0.000293 +2021-03-26 04:50:33,602 epoch 80 - iter 45/50 - loss 2.38336645 - samples/sec: 78.92 - lr: 0.000293 +2021-03-26 04:50:35,459 epoch 80 - iter 50/50 - loss 2.35637002 - samples/sec: 86.27 - lr: 0.000293 +2021-03-26 04:50:35,459 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:50:35,460 EPOCH 80 done: loss 2.3564 - lr 0.0002930 +2021-03-26 04:50:36,305 DEV : loss 5.630791664123535 - score 0.9219 +2021-03-26 04:50:36,331 BAD EPOCHS (no improvement): 1 +2021-03-26 04:50:36,331 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:50:38,254 epoch 81 - iter 5/50 - loss 2.48897581 - samples/sec: 83.31 - lr: 0.000293 +2021-03-26 04:50:40,041 epoch 81 - iter 10/50 - loss 2.25514839 - samples/sec: 89.62 - lr: 0.000293 +2021-03-26 04:50:42,071 epoch 81 - iter 15/50 - loss 2.26627936 - samples/sec: 78.90 - lr: 0.000293 +2021-03-26 04:50:43,990 epoch 81 - iter 20/50 - loss 2.30013109 - samples/sec: 83.47 - lr: 0.000293 +2021-03-26 04:50:45,953 epoch 81 - iter 25/50 - loss 2.23416708 - samples/sec: 81.57 - lr: 0.000293 +2021-03-26 04:50:47,965 epoch 81 - iter 30/50 - loss 2.24264871 - samples/sec: 79.60 - lr: 0.000293 +2021-03-26 04:50:49,973 epoch 81 - iter 35/50 - loss 2.27044076 - samples/sec: 79.79 - lr: 0.000293 +2021-03-26 04:50:51,941 epoch 81 - iter 40/50 - loss 2.25232525 - samples/sec: 81.36 - lr: 0.000293 +2021-03-26 04:50:53,968 epoch 81 - iter 45/50 - loss 2.24451494 - samples/sec: 78.99 - lr: 0.000293 +2021-03-26 04:50:55,794 epoch 81 - iter 50/50 - loss 2.26902369 - samples/sec: 87.71 - lr: 0.000293 +2021-03-26 04:50:55,795 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:50:55,795 EPOCH 81 done: loss 2.2690 - lr 0.0002930 +2021-03-26 04:50:56,602 DEV : loss 5.631392955780029 - score 0.9219 +2021-03-26 04:50:56,627 BAD EPOCHS (no improvement): 2 +2021-03-26 04:50:56,628 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:50:58,539 epoch 82 - iter 5/50 - loss 1.69027870 - samples/sec: 83.80 - lr: 0.000293 +2021-03-26 04:51:00,590 epoch 82 - iter 10/50 - loss 2.31625203 - samples/sec: 78.09 - lr: 0.000293 +2021-03-26 04:51:02,529 epoch 82 - iter 15/50 - loss 2.20893728 - samples/sec: 82.59 - lr: 0.000293 +2021-03-26 04:51:04,414 epoch 82 - iter 20/50 - loss 2.22896555 - samples/sec: 84.93 - lr: 0.000293 +2021-03-26 04:51:06,511 epoch 82 - iter 25/50 - loss 2.23054180 - samples/sec: 76.38 - lr: 0.000293 +2021-03-26 04:51:08,485 epoch 82 - iter 30/50 - loss 2.26120172 - samples/sec: 81.14 - lr: 0.000293 +2021-03-26 04:51:10,333 epoch 82 - iter 35/50 - loss 2.23025372 - samples/sec: 86.64 - lr: 0.000293 +2021-03-26 04:51:12,270 epoch 82 - iter 40/50 - loss 2.27022659 - samples/sec: 82.68 - lr: 0.000293 +2021-03-26 04:51:14,213 epoch 82 - iter 45/50 - loss 2.26758277 - samples/sec: 82.42 - lr: 0.000293 +2021-03-26 04:51:16,044 epoch 82 - iter 50/50 - loss 2.28401194 - samples/sec: 87.44 - lr: 0.000293 +2021-03-26 04:51:16,045 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:51:16,045 EPOCH 82 done: loss 2.2840 - lr 0.0002930 +2021-03-26 04:51:16,881 DEV : loss 5.632561683654785 - score 0.9219 +2021-03-26 04:51:16,907 BAD EPOCHS (no improvement): 3 +2021-03-26 04:51:16,908 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:51:19,242 epoch 83 - iter 5/50 - loss 2.13644736 - samples/sec: 68.62 - lr: 0.000293 +2021-03-26 04:51:21,256 epoch 83 - iter 10/50 - loss 2.38941020 - samples/sec: 79.51 - lr: 0.000293 +2021-03-26 04:51:23,131 epoch 83 - iter 15/50 - loss 2.38207877 - samples/sec: 85.41 - lr: 0.000293 +2021-03-26 04:51:25,093 epoch 83 - iter 20/50 - loss 2.34289131 - samples/sec: 81.61 - lr: 0.000293 +2021-03-26 04:51:26,872 epoch 83 - iter 25/50 - loss 2.32033530 - samples/sec: 90.10 - lr: 0.000293 +2021-03-26 04:51:28,995 epoch 83 - iter 30/50 - loss 2.27905010 - samples/sec: 75.43 - lr: 0.000293 +2021-03-26 04:51:30,992 epoch 83 - iter 35/50 - loss 2.28741706 - samples/sec: 80.22 - lr: 0.000293 +2021-03-26 04:51:32,890 epoch 83 - iter 40/50 - loss 2.29978756 - samples/sec: 84.39 - lr: 0.000293 +2021-03-26 04:51:34,879 epoch 83 - iter 45/50 - loss 2.33553922 - samples/sec: 80.52 - lr: 0.000293 +2021-03-26 04:51:36,738 epoch 83 - iter 50/50 - loss 2.34764516 - samples/sec: 86.15 - lr: 0.000293 +2021-03-26 04:51:36,739 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:51:36,739 EPOCH 83 done: loss 2.3476 - lr 0.0002930 +2021-03-26 04:51:37,557 DEV : loss 5.633981227874756 - score 0.9219 +2021-03-26 04:51:37,580 BAD EPOCHS (no improvement): 4 +2021-03-26 04:51:37,581 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:51:39,566 epoch 84 - iter 5/50 - loss 2.83537569 - samples/sec: 80.69 - lr: 0.000146 +2021-03-26 04:51:41,620 epoch 84 - iter 10/50 - loss 2.61301935 - samples/sec: 78.01 - lr: 0.000146 +2021-03-26 04:51:44,781 epoch 84 - iter 15/50 - loss 2.47842774 - samples/sec: 50.63 - lr: 0.000146 +2021-03-26 04:51:46,845 epoch 84 - iter 20/50 - loss 2.54841444 - samples/sec: 77.59 - lr: 0.000146 +2021-03-26 04:51:48,858 epoch 84 - iter 25/50 - loss 2.49058902 - samples/sec: 79.60 - lr: 0.000146 +2021-03-26 04:51:50,774 epoch 84 - iter 30/50 - loss 2.46437473 - samples/sec: 83.55 - lr: 0.000146 +2021-03-26 04:51:52,676 epoch 84 - iter 35/50 - loss 2.44853312 - samples/sec: 84.21 - lr: 0.000146 +2021-03-26 04:51:54,564 epoch 84 - iter 40/50 - loss 2.42928336 - samples/sec: 84.81 - lr: 0.000146 +2021-03-26 04:51:56,550 epoch 84 - iter 45/50 - loss 2.44210018 - samples/sec: 80.63 - lr: 0.000146 +2021-03-26 04:51:58,319 epoch 84 - iter 50/50 - loss 2.36999966 - samples/sec: 90.51 - lr: 0.000146 +2021-03-26 04:51:58,320 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:51:58,320 EPOCH 84 done: loss 2.3700 - lr 0.0001465 +2021-03-26 04:51:59,138 DEV : loss 5.633601665496826 - score 0.9219 +2021-03-26 04:51:59,158 BAD EPOCHS (no improvement): 1 +2021-03-26 04:51:59,159 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:52:00,936 epoch 85 - iter 5/50 - loss 2.53912203 - samples/sec: 90.07 - lr: 0.000146 +2021-03-26 04:52:02,884 epoch 85 - iter 10/50 - loss 2.55454174 - samples/sec: 82.22 - lr: 0.000146 +2021-03-26 04:52:05,097 epoch 85 - iter 15/50 - loss 2.37489824 - samples/sec: 72.38 - lr: 0.000146 +2021-03-26 04:52:07,291 epoch 85 - iter 20/50 - loss 2.44851355 - samples/sec: 72.95 - lr: 0.000146 +2021-03-26 04:52:09,311 epoch 85 - iter 25/50 - loss 2.45759084 - samples/sec: 79.33 - lr: 0.000146 +2021-03-26 04:52:11,666 epoch 85 - iter 30/50 - loss 2.37488166 - samples/sec: 67.99 - lr: 0.000146 +2021-03-26 04:52:13,863 epoch 85 - iter 35/50 - loss 2.32282099 - samples/sec: 72.92 - lr: 0.000146 +2021-03-26 04:52:15,805 epoch 85 - iter 40/50 - loss 2.35236027 - samples/sec: 82.48 - lr: 0.000146 +2021-03-26 04:52:17,833 epoch 85 - iter 45/50 - loss 2.34360903 - samples/sec: 78.96 - lr: 0.000146 +2021-03-26 04:52:19,618 epoch 85 - iter 50/50 - loss 2.32948160 - samples/sec: 89.72 - lr: 0.000146 +2021-03-26 04:52:19,619 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:52:19,619 EPOCH 85 done: loss 2.3295 - lr 0.0001465 +2021-03-26 04:52:20,430 DEV : loss 5.6335673332214355 - score 0.9219 +2021-03-26 04:52:20,454 BAD EPOCHS (no improvement): 2 +2021-03-26 04:52:20,454 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:52:22,441 epoch 86 - iter 5/50 - loss 2.27040057 - samples/sec: 80.61 - lr: 0.000146 +2021-03-26 04:52:24,461 epoch 86 - iter 10/50 - loss 2.20236114 - samples/sec: 79.26 - lr: 0.000146 +2021-03-26 04:52:26,490 epoch 86 - iter 15/50 - loss 2.20784380 - samples/sec: 78.93 - lr: 0.000146 +2021-03-26 04:52:28,401 epoch 86 - iter 20/50 - loss 2.25163724 - samples/sec: 83.80 - lr: 0.000146 +2021-03-26 04:52:30,217 epoch 86 - iter 25/50 - loss 2.24298689 - samples/sec: 88.23 - lr: 0.000146 +2021-03-26 04:52:32,301 epoch 86 - iter 30/50 - loss 2.31922689 - samples/sec: 76.85 - lr: 0.000146 +2021-03-26 04:52:34,212 epoch 86 - iter 35/50 - loss 2.33380191 - samples/sec: 83.76 - lr: 0.000146 +2021-03-26 04:52:36,177 epoch 86 - iter 40/50 - loss 2.40758548 - samples/sec: 81.50 - lr: 0.000146 +2021-03-26 04:52:38,150 epoch 86 - iter 45/50 - loss 2.38650574 - samples/sec: 81.17 - lr: 0.000146 +2021-03-26 04:52:39,975 epoch 86 - iter 50/50 - loss 2.44259843 - samples/sec: 87.79 - lr: 0.000146 +2021-03-26 04:52:39,976 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:52:39,976 EPOCH 86 done: loss 2.4426 - lr 0.0001465 +2021-03-26 04:52:40,774 DEV : loss 5.63382625579834 - score 0.9219 +2021-03-26 04:52:40,799 BAD EPOCHS (no improvement): 3 +2021-03-26 04:52:40,800 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:52:42,804 epoch 87 - iter 5/50 - loss 2.61555619 - samples/sec: 79.90 - lr: 0.000146 +2021-03-26 04:52:44,669 epoch 87 - iter 10/50 - loss 2.35265634 - samples/sec: 85.90 - lr: 0.000146 +2021-03-26 04:52:46,667 epoch 87 - iter 15/50 - loss 2.35643369 - samples/sec: 80.17 - lr: 0.000146 +2021-03-26 04:52:48,625 epoch 87 - iter 20/50 - loss 2.32868401 - samples/sec: 81.79 - lr: 0.000146 +2021-03-26 04:52:50,842 epoch 87 - iter 25/50 - loss 2.40707584 - samples/sec: 72.21 - lr: 0.000146 +2021-03-26 04:52:53,435 epoch 87 - iter 30/50 - loss 2.38372990 - samples/sec: 61.76 - lr: 0.000146 +2021-03-26 04:52:55,452 epoch 87 - iter 35/50 - loss 2.35168895 - samples/sec: 79.39 - lr: 0.000146 +2021-03-26 04:52:57,491 epoch 87 - iter 40/50 - loss 2.29494644 - samples/sec: 78.54 - lr: 0.000146 +2021-03-26 04:52:59,466 epoch 87 - iter 45/50 - loss 2.30548906 - samples/sec: 81.15 - lr: 0.000146 +2021-03-26 04:53:01,369 epoch 87 - iter 50/50 - loss 2.35296534 - samples/sec: 84.16 - lr: 0.000146 +2021-03-26 04:53:01,369 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:53:01,370 EPOCH 87 done: loss 2.3530 - lr 0.0001465 +2021-03-26 04:53:02,177 DEV : loss 5.633864402770996 - score 0.9219 +2021-03-26 04:53:02,201 BAD EPOCHS (no improvement): 4 +2021-03-26 04:53:02,202 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:53:02,203 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:53:02,203 learning rate too small - quitting training! +2021-03-26 04:53:02,203 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:53:11,683 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:53:11,684 Testing using best model ... +2021-03-26 04:53:11,685 loading file /home/tmp/megahedm/models/multipos/multipos_UDMADAR_4Diale-LEV_EGY_GLF_MGR__fasttext_flairbwfw__32__0.3_202103260417/best-model.pt +2021-03-26 04:53:19,342 0.914 +2021-03-26 04:53:19,343 +Results: +- F-score (micro): 0.9105 +- F-score (macro): 0.5251 +- Accuracy (incl. no class): 0.914 + +By class: + precision recall f1-score support + + PROPN 0.8824 0.9375 0.9091 16 + NUM 1.0000 0.9444 0.9714 18 + ADV 0.8785 0.8868 0.8826 106 + PART 0.9679 0.9679 0.9679 218 + VERB 0.9653 0.9586 0.9619 145 + ADP 0.9804 0.9709 0.9756 103 + DET 0.9434 0.9434 0.9434 53 + NOUN 0.9067 0.9577 0.9315 426 + CCONJ 0.9740 0.9868 0.9804 76 + ADJ 0.8699 0.8231 0.8458 130 + PUNCT 1.0000 1.0000 1.0000 26 + AUX 0.9524 0.9524 0.9524 42 + PRON 0.9706 0.9565 0.9635 207 + INTJ 0.8824 1.0000 0.9375 15 + SCONJ 0.9200 0.9583 0.9388 24 + CONJ 1.0000 1.0000 1.0000 40 + NOUN+PRON 0.7647 0.8667 0.8125 60 + PUNC 1.0000 1.0000 1.0000 177 + V+PRON 0.8305 0.8448 0.8376 58 + DET+NOUN 0.9222 0.9881 0.9540 84 + V 0.8642 0.8861 0.8750 79 + EOS 1.0000 1.0000 1.0000 70 + FOREIGN 1.0000 0.8571 0.9231 7 + MENTION 0.8947 1.0000 0.9444 17 + CONJ+PART 0.8889 0.7273 0.8000 11 + CONJ+NOUN 0.5000 0.8182 0.6207 11 + CONJ+ADJ 0.0000 0.0000 0.0000 3 + PREP+NOUN 0.8000 0.9524 0.8696 21 + ADJ+NSUFF 0.7297 0.8182 0.7714 33 + CONJ+PRON 1.0000 1.0000 1.0000 8 + PROG_PART+V 0.8710 0.8710 0.8710 31 + PART+PRON 1.0000 0.8095 0.8947 21 + FUT_PART 1.0000 1.0000 1.0000 3 + PREP 0.9846 0.9846 0.9846 65 + PRON+DET+NOUN+NSUFF 1.0000 1.0000 1.0000 1 + DET+ADJ+NSUFF 0.5000 1.0000 0.6667 2 + PREP+DET+NUM+NSUFF 1.0000 0.0000 0.0000 1 + PREP+DET+NOUN 0.8462 0.9167 0.8800 12 + NOUN+CASE 1.0000 0.8000 0.8889 5 + PREP+PART 1.0000 1.0000 1.0000 2 + NOUN+NSUFF 0.8393 0.8246 0.8319 57 + FUT_PART+V 0.8889 0.6154 0.7273 13 + CONJ+PREP 1.0000 0.8000 0.8889 5 + ADJ+PRON 0.6667 0.1818 0.2857 11 + V+PRON+PRON 0.7143 0.5556 0.6250 9 + CONJ+PREP+DET+NOUN 1.0000 1.0000 1.0000 1 + V+PRON+PREP+PRON 0.0000 0.0000 0.0000 3 + NOUN+NSUFF+NSUFF 1.0000 0.0000 0.0000 1 + DET+NOUN+NSUFF 0.8750 0.8235 0.8485 17 + CONJ+PREP+NOUN+PRON 1.0000 0.0000 0.0000 1 + CONJ+NOUN+PRON 0.0000 0.0000 0.0000 2 + PART+ADJ 1.0000 0.0000 0.0000 1 + PART+NOUN+PRON 0.0000 0.0000 0.0000 2 + CONJ+PREP+NOUN 1.0000 0.0000 0.0000 1 + URL 1.0000 1.0000 1.0000 6 + CONJ+FUT_PART 1.0000 0.0000 0.0000 1 + PREP+DET+NOUN+NSUFF 0.5000 0.5000 0.5000 2 + HASH 1.0000 1.0000 1.0000 13 + PREP+ADJ 1.0000 0.0000 0.0000 1 + EMOT 1.0000 0.9615 0.9804 26 + CONJ+FUT_PART+V 1.0000 0.0000 0.0000 2 + CONJ+NOUN+NSUFF 1.0000 0.6667 0.8000 3 + DET+ADJ 1.0000 0.4545 0.6250 11 + NOUN+NSUFF+PRON 0.5714 0.8000 0.6667 5 + PREP+PRON 0.9091 1.0000 0.9524 20 + CONJ+PREP+PART 1.0000 0.0000 0.0000 2 + V+PREP+PRON 0.8333 0.5556 0.6667 9 + PART+V 1.0000 0.0000 0.0000 1 + ADJ+PREP+PRON 0.0000 1.0000 0.0000 0 + PROG_PART+V+PREP+PRON 0.0000 0.0000 0.0000 3 + PROG_PART+V+PRON 0.7500 0.9375 0.8333 16 + CONJ+ADJ+CASE 1.0000 0.0000 0.0000 1 + CONJ+V+PRON+PREP+PRON 0.0000 1.0000 0.0000 0 + CONJ+PROG_PART+V 0.6667 1.0000 0.8000 2 + CONJ+PREP+V 1.0000 0.0000 0.0000 1 + PREP+V+PRON 1.0000 0.0000 0.0000 1 + DET+NOUN+NSUFF+NSUFF 1.0000 0.0000 0.0000 1 + PREP+NOUN+PRON 1.0000 0.4000 0.5714 5 + CONJ+DET+NOUN 1.0000 1.0000 1.0000 1 + CONJ+V 0.4000 1.0000 0.5714 2 + PREP+NOUN+NSUFF 0.5000 1.0000 0.6667 2 + V+NEG_PART 1.0000 0.0000 0.0000 1 + PROG_PART+V+PRON+PRON 0.0000 1.0000 0.0000 0 + CONJ+V+PRON 0.5000 0.5000 0.5000 2 + PREP+ADJ+NSUFF 1.0000 0.0000 0.0000 1 + PRON+DET+NOUN 1.0000 0.0000 0.0000 1 + PREP+DET 1.0000 0.0000 0.0000 1 + NOUN+PREP+PRON 1.0000 0.0000 0.0000 1 + PART+NOUN 1.0000 0.6000 0.7500 5 + CONJ+V+PREP+PRON 1.0000 0.0000 0.0000 1 + CONJ+ADJ+NSUFF 0.0000 0.0000 0.0000 1 + PART+PREP+PRON+NEG_PART 1.0000 0.0000 0.0000 1 + PART+V+NEG_PART 0.4000 0.4000 0.4000 5 + CONJ+PART+V+NEG_PART 1.0000 0.0000 0.0000 2 + CONJ+PART+PREP+NEG_PART 0.0000 1.0000 0.0000 0 + PART+V+PRON+NEG_PART 0.5000 0.8000 0.6154 5 + FUT_PART+V+PREP+PRON 1.0000 1.0000 1.0000 1 + PART+PREP+NEG_PART 1.0000 1.0000 1.0000 5 + FUT_PART+V+PRON+PRON 0.0000 0.0000 0.0000 1 + FUT_PART+V+PRON 0.0000 0.0000 0.0000 2 +PART+PROG_PART+V+PRON+NEG_PART 1.0000 0.0000 0.0000 1 + PART+NSUFF 1.0000 0.0000 0.0000 1 + PART+NOUN+NEG_PART 1.0000 1.0000 1.0000 1 + CONJ+PROG_PART+V+PRON 1.0000 1.0000 1.0000 2 + ADJ+NSUFF+PRON 1.0000 1.0000 1.0000 1 + PART+PROG_PART+V+NEG_PART 1.0000 0.0000 0.0000 3 + PROG_PART+V+NEG_PART 0.0000 1.0000 0.0000 0 + PART+PART+V+NEG_PART 1.0000 0.0000 0.0000 1 + NUM+NSUFF 1.0000 1.0000 1.0000 1 + CONJ+PART+V+PRON+NEG_PART 0.0000 1.0000 0.0000 0 + PART+PREP+PRON 1.0000 0.0000 0.0000 1 + CONJ+ADV+NSUFF 1.0000 0.0000 0.0000 1 + CONJ+ADV 0.0000 1.0000 0.0000 0 + CONJ+PART+ADJ 0.0000 1.0000 0.0000 0 + + micro avg 0.9105 0.9105 0.9105 2737 + macro avg 0.7807 0.6032 0.5251 2737 + weighted avg 0.9164 0.9105 0.9047 2737 + +2021-03-26 04:53:19,343 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:53:19,343 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:53:23,166 Reading data from ../../Datasets_adhoc/CSCS_corpus-GUC +2021-03-26 04:53:23,167 Train: ../../Datasets_adhoc/CSCS_corpus-GUC/all_participants.conllu +2021-03-26 04:53:23,167 Dev: None +2021-03-26 04:53:23,167 Test: None +2021-03-26 04:53:23,462 Reading data from ../../Datasets_adhoc/UD_MADAR +2021-03-26 04:53:23,463 Train: ../../Datasets_adhoc/UD_MADAR/ajp_madar-ud-test-edit.conllu +2021-03-26 04:53:23,463 Dev: None +2021-03-26 04:53:23,463 Test: None +2021-03-26 04:53:23,512 Reading data from ../../Datasets_adhoc/dialectal_arabic_resources-master +2021-03-26 04:53:23,513 Train: ../../Datasets_adhoc/dialectal_arabic_resources-master/seg_plus_pos_lev.txt +2021-03-26 04:53:23,513 Dev: None +2021-03-26 04:53:23,513 Test: None +2021-03-26 04:53:23,689 Reading data from ../../Datasets_adhoc/dialectal_arabic_resources-master +2021-03-26 04:53:23,689 Train: ../../Datasets_adhoc/dialectal_arabic_resources-master/seg_plus_pos_egy.txt +2021-03-26 04:53:23,690 Dev: None +2021-03-26 04:53:23,690 Test: None +2021-03-26 04:53:25,752 Reading data from ../../Datasets_adhoc/dialectal_arabic_resources-master +2021-03-26 04:53:25,753 Train: ../../Datasets_adhoc/dialectal_arabic_resources-master/seg_plus_pos_glf.txt +2021-03-26 04:53:25,753 Dev: None +2021-03-26 04:53:25,753 Test: None +2021-03-26 04:53:25,901 Reading data from ../../Datasets_adhoc/dialectal_arabic_resources-master +2021-03-26 04:53:25,901 Train: ../../Datasets_adhoc/dialectal_arabic_resources-master/seg_plus_pos_mgr.txt +2021-03-26 04:53:25,902 Dev: None +2021-03-26 04:53:25,902 Test: None +2021-03-26 04:53:26,053 Filtering long sentences +2021-03-26 04:53:26,092 MultiCorpus: 1574 train + 177 dev + 193 test sentences + - ColumnCorpus Corpus: 934 train + 104 dev + 115 test sentences + - ColumnCorpus Corpus: 81 train + 9 dev + 10 test sentences + - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences + - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences +2021-03-26 04:53:26,501 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:53:26,502 Model: "SequenceTagger( + (embeddings): StackedEmbeddings( + (list_embedding_0): WordEmbeddings('ar') + (list_embedding_1): FlairEmbeddings( + (lm): LanguageModel( + (drop): Dropout(p=0.1, inplace=False) + (encoder): Embedding(7125, 100) + (rnn): LSTM(100, 2048) + (decoder): Linear(in_features=2048, out_features=7125, bias=True) + ) + ) + (list_embedding_2): FlairEmbeddings( + (lm): LanguageModel( + (drop): Dropout(p=0.1, inplace=False) + (encoder): Embedding(7125, 100) + (rnn): LSTM(100, 2048) + (decoder): Linear(in_features=2048, out_features=7125, bias=True) + ) + ) + ) + (word_dropout): WordDropout(p=0.05) + (locked_dropout): LockedDropout(p=0.5) + (embedding2nn): Linear(in_features=4396, out_features=4396, bias=True) + (rnn): LSTM(4396, 256, batch_first=True, bidirectional=True) + (linear): Linear(in_features=512, out_features=206, bias=True) + (beta): 1.0 + (weights): None + (weight_tensor) None +)" +2021-03-26 04:53:26,502 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:53:26,502 Corpus: "MultiCorpus: 1574 train + 177 dev + 193 test sentences + - ColumnCorpus Corpus: 934 train + 104 dev + 115 test sentences + - ColumnCorpus Corpus: 81 train + 9 dev + 10 test sentences + - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences + - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences" +2021-03-26 04:53:26,503 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:53:26,503 Parameters: +2021-03-26 04:53:26,503 - learning_rate: "0.3" +2021-03-26 04:53:26,503 - mini_batch_size: "64" +2021-03-26 04:53:26,503 - patience: "3" +2021-03-26 04:53:26,504 - anneal_factor: "0.5" +2021-03-26 04:53:26,504 - max_epochs: "150" +2021-03-26 04:53:26,504 - shuffle: "True" +2021-03-26 04:53:26,504 - train_with_dev: "False" +2021-03-26 04:53:26,504 - batch_growth_annealing: "False" +2021-03-26 04:53:26,505 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:53:26,505 Model training base path: "/home/tmp/megahedm/models/multipos/multipos_UDMADAR_4Diale-LEV_EGY_GLF_MGR__fasttext_flairbwfw__64__0.3_202103260453" +2021-03-26 04:53:26,505 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:53:26,505 Device: cuda:0 +2021-03-26 04:53:26,505 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:53:26,506 Embeddings storage mode: cpu +2021-03-26 04:53:26,507 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:53:28,213 epoch 1 - iter 2/25 - loss 83.97468567 - samples/sec: 75.08 - lr: 0.300000 +2021-03-26 04:53:29,534 epoch 1 - iter 4/25 - loss 75.63530731 - samples/sec: 97.04 - lr: 0.300000 +2021-03-26 04:53:30,856 epoch 1 - iter 6/25 - loss 72.50162125 - samples/sec: 96.91 - lr: 0.300000 +2021-03-26 04:53:32,192 epoch 1 - iter 8/25 - loss 70.06266451 - samples/sec: 95.92 - lr: 0.300000 +2021-03-26 04:53:33,721 epoch 1 - iter 10/25 - loss 68.25133667 - samples/sec: 83.77 - lr: 0.300000 +2021-03-26 04:53:35,053 epoch 1 - iter 12/25 - loss 66.84262911 - samples/sec: 96.26 - lr: 0.300000 +2021-03-26 04:53:36,500 epoch 1 - iter 14/25 - loss 66.02898734 - samples/sec: 88.51 - lr: 0.300000 +2021-03-26 04:53:37,772 epoch 1 - iter 16/25 - loss 63.99918914 - samples/sec: 100.80 - lr: 0.300000 +2021-03-26 04:53:39,457 epoch 1 - iter 18/25 - loss 62.62216356 - samples/sec: 76.01 - lr: 0.300000 +2021-03-26 04:53:41,244 epoch 1 - iter 20/25 - loss 61.69286957 - samples/sec: 71.68 - lr: 0.300000 +2021-03-26 04:53:42,598 epoch 1 - iter 22/25 - loss 60.76663035 - samples/sec: 94.66 - lr: 0.300000 +2021-03-26 04:53:43,979 epoch 1 - iter 24/25 - loss 59.23835262 - samples/sec: 92.75 - lr: 0.300000 +2021-03-26 04:53:44,540 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:53:44,540 EPOCH 1 done: loss 58.6116 - lr 0.3000000 +2021-03-26 04:53:45,781 DEV : loss 42.79204559326172 - score 0.3643 +2021-03-26 04:53:45,806 BAD EPOCHS (no improvement): 0 +2021-03-26 04:53:55,344 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:53:56,735 epoch 2 - iter 2/25 - loss 43.24363708 - samples/sec: 92.21 - lr: 0.300000 +2021-03-26 04:53:58,021 epoch 2 - iter 4/25 - loss 40.99682331 - samples/sec: 99.68 - lr: 0.300000 +2021-03-26 04:53:59,501 epoch 2 - iter 6/25 - loss 40.01383527 - samples/sec: 86.57 - lr: 0.300000 +2021-03-26 04:54:00,815 epoch 2 - iter 8/25 - loss 40.72169018 - samples/sec: 97.57 - lr: 0.300000 +2021-03-26 04:54:02,099 epoch 2 - iter 10/25 - loss 40.42348213 - samples/sec: 99.81 - lr: 0.300000 +2021-03-26 04:54:03,096 epoch 2 - iter 12/25 - loss 39.97247760 - samples/sec: 128.64 - lr: 0.300000 +2021-03-26 04:54:04,152 epoch 2 - iter 14/25 - loss 39.15356309 - samples/sec: 121.35 - lr: 0.300000 +2021-03-26 04:54:05,258 epoch 2 - iter 16/25 - loss 38.90821433 - samples/sec: 116.00 - lr: 0.300000 +2021-03-26 04:54:06,347 epoch 2 - iter 18/25 - loss 38.21725549 - samples/sec: 117.69 - lr: 0.300000 +2021-03-26 04:54:07,491 epoch 2 - iter 20/25 - loss 37.46524897 - samples/sec: 112.16 - lr: 0.300000 +2021-03-26 04:54:08,417 epoch 2 - iter 22/25 - loss 36.69194898 - samples/sec: 138.43 - lr: 0.300000 +2021-03-26 04:54:09,446 epoch 2 - iter 24/25 - loss 36.02944136 - samples/sec: 124.54 - lr: 0.300000 +2021-03-26 04:54:09,833 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:54:09,833 EPOCH 2 done: loss 35.7397 - lr 0.3000000 +2021-03-26 04:54:10,594 DEV : loss 29.750354766845703 - score 0.5246 +2021-03-26 04:54:10,620 BAD EPOCHS (no improvement): 0 +2021-03-26 04:54:20,425 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:54:21,484 epoch 3 - iter 2/25 - loss 32.57054138 - samples/sec: 121.30 - lr: 0.300000 +2021-03-26 04:54:22,551 epoch 3 - iter 4/25 - loss 31.07599831 - samples/sec: 120.16 - lr: 0.300000 +2021-03-26 04:54:23,771 epoch 3 - iter 6/25 - loss 31.57134374 - samples/sec: 105.05 - lr: 0.300000 +2021-03-26 04:54:24,751 epoch 3 - iter 8/25 - loss 30.99461079 - samples/sec: 130.76 - lr: 0.300000 +2021-03-26 04:54:25,700 epoch 3 - iter 10/25 - loss 30.21249199 - samples/sec: 135.03 - lr: 0.300000 +2021-03-26 04:54:26,682 epoch 3 - iter 12/25 - loss 29.47812446 - samples/sec: 130.56 - lr: 0.300000 +2021-03-26 04:54:27,686 epoch 3 - iter 14/25 - loss 28.78943321 - samples/sec: 127.70 - lr: 0.300000 +2021-03-26 04:54:28,700 epoch 3 - iter 16/25 - loss 28.61816049 - samples/sec: 126.48 - lr: 0.300000 +2021-03-26 04:54:29,711 epoch 3 - iter 18/25 - loss 28.34320969 - samples/sec: 126.82 - lr: 0.300000 +2021-03-26 04:54:30,777 epoch 3 - iter 20/25 - loss 27.82139015 - samples/sec: 120.25 - lr: 0.300000 +2021-03-26 04:54:31,755 epoch 3 - iter 22/25 - loss 27.48131596 - samples/sec: 131.15 - lr: 0.300000 +2021-03-26 04:54:32,841 epoch 3 - iter 24/25 - loss 27.17227077 - samples/sec: 117.97 - lr: 0.300000 +2021-03-26 04:54:33,275 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:54:33,275 EPOCH 3 done: loss 26.8413 - lr 0.3000000 +2021-03-26 04:54:34,027 DEV : loss 19.97711753845215 - score 0.6439 +2021-03-26 04:54:34,052 BAD EPOCHS (no improvement): 0 +2021-03-26 04:54:43,808 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:54:45,019 epoch 4 - iter 2/25 - loss 19.83089352 - samples/sec: 105.94 - lr: 0.300000 +2021-03-26 04:54:46,080 epoch 4 - iter 4/25 - loss 19.36055803 - samples/sec: 120.83 - lr: 0.300000 +2021-03-26 04:54:47,235 epoch 4 - iter 6/25 - loss 20.59924157 - samples/sec: 111.03 - lr: 0.300000 +2021-03-26 04:54:48,384 epoch 4 - iter 8/25 - loss 20.88819456 - samples/sec: 111.60 - lr: 0.300000 +2021-03-26 04:54:49,409 epoch 4 - iter 10/25 - loss 21.39904594 - samples/sec: 125.20 - lr: 0.300000 +2021-03-26 04:54:50,527 epoch 4 - iter 12/25 - loss 21.46397877 - samples/sec: 114.64 - lr: 0.300000 +2021-03-26 04:54:51,584 epoch 4 - iter 14/25 - loss 21.64387935 - samples/sec: 121.24 - lr: 0.300000 +2021-03-26 04:54:52,581 epoch 4 - iter 16/25 - loss 21.79656446 - samples/sec: 128.51 - lr: 0.300000 +2021-03-26 04:54:53,606 epoch 4 - iter 18/25 - loss 22.06780073 - samples/sec: 125.06 - lr: 0.300000 +2021-03-26 04:54:54,615 epoch 4 - iter 20/25 - loss 21.88907843 - samples/sec: 127.22 - lr: 0.300000 +2021-03-26 04:54:55,736 epoch 4 - iter 22/25 - loss 21.68766655 - samples/sec: 114.33 - lr: 0.300000 +2021-03-26 04:54:56,755 epoch 4 - iter 24/25 - loss 21.61118587 - samples/sec: 125.98 - lr: 0.300000 +2021-03-26 04:54:57,265 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:54:57,265 EPOCH 4 done: loss 21.4647 - lr 0.3000000 +2021-03-26 04:54:58,036 DEV : loss 16.356971740722656 - score 0.7141 +2021-03-26 04:54:58,057 BAD EPOCHS (no improvement): 0 +2021-03-26 04:55:08,244 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:55:09,312 epoch 5 - iter 2/25 - loss 19.59161377 - samples/sec: 120.15 - lr: 0.300000 +2021-03-26 04:55:10,471 epoch 5 - iter 4/25 - loss 19.70559645 - samples/sec: 110.70 - lr: 0.300000 +2021-03-26 04:55:11,506 epoch 5 - iter 6/25 - loss 19.31401348 - samples/sec: 123.87 - lr: 0.300000 +2021-03-26 04:55:12,619 epoch 5 - iter 8/25 - loss 19.14877057 - samples/sec: 115.21 - lr: 0.300000 +2021-03-26 04:55:14,014 epoch 5 - iter 10/25 - loss 18.98953533 - samples/sec: 91.90 - lr: 0.300000 +2021-03-26 04:55:15,354 epoch 5 - iter 12/25 - loss 18.87965854 - samples/sec: 95.67 - lr: 0.300000 +2021-03-26 04:55:16,455 epoch 5 - iter 14/25 - loss 18.58909457 - samples/sec: 116.37 - lr: 0.300000 +2021-03-26 04:55:17,485 epoch 5 - iter 16/25 - loss 18.31753743 - samples/sec: 124.45 - lr: 0.300000 +2021-03-26 04:55:18,589 epoch 5 - iter 18/25 - loss 18.08833377 - samples/sec: 116.17 - lr: 0.300000 +2021-03-26 04:55:19,616 epoch 5 - iter 20/25 - loss 17.97942266 - samples/sec: 124.98 - lr: 0.300000 +2021-03-26 04:55:20,678 epoch 5 - iter 22/25 - loss 17.93916269 - samples/sec: 120.72 - lr: 0.300000 +2021-03-26 04:55:21,612 epoch 5 - iter 24/25 - loss 17.80934910 - samples/sec: 137.27 - lr: 0.300000 +2021-03-26 04:55:22,034 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:55:22,036 EPOCH 5 done: loss 17.8025 - lr 0.3000000 +2021-03-26 04:55:22,800 DEV : loss 13.64643669128418 - score 0.7617 +2021-03-26 04:55:22,825 BAD EPOCHS (no improvement): 0 +2021-03-26 04:55:32,596 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:55:33,686 epoch 6 - iter 2/25 - loss 15.59965611 - samples/sec: 117.74 - lr: 0.300000 +2021-03-26 04:55:34,792 epoch 6 - iter 4/25 - loss 16.48317146 - samples/sec: 115.95 - lr: 0.300000 +2021-03-26 04:55:35,785 epoch 6 - iter 6/25 - loss 16.92143854 - samples/sec: 129.11 - lr: 0.300000 +2021-03-26 04:55:36,888 epoch 6 - iter 8/25 - loss 16.72712636 - samples/sec: 116.07 - lr: 0.300000 +2021-03-26 04:55:37,882 epoch 6 - iter 10/25 - loss 16.70757866 - samples/sec: 129.16 - lr: 0.300000 +2021-03-26 04:55:38,963 epoch 6 - iter 12/25 - loss 16.57004619 - samples/sec: 118.55 - lr: 0.300000 +2021-03-26 04:55:40,075 epoch 6 - iter 14/25 - loss 16.66024719 - samples/sec: 115.27 - lr: 0.300000 +2021-03-26 04:55:41,105 epoch 6 - iter 16/25 - loss 16.44063133 - samples/sec: 124.54 - lr: 0.300000 +2021-03-26 04:55:42,329 epoch 6 - iter 18/25 - loss 16.27063513 - samples/sec: 104.70 - lr: 0.300000 +2021-03-26 04:55:43,481 epoch 6 - iter 20/25 - loss 16.08676958 - samples/sec: 111.24 - lr: 0.300000 +2021-03-26 04:55:44,554 epoch 6 - iter 22/25 - loss 15.83572531 - samples/sec: 119.47 - lr: 0.300000 +2021-03-26 04:55:45,590 epoch 6 - iter 24/25 - loss 15.76636434 - samples/sec: 123.78 - lr: 0.300000 +2021-03-26 04:55:45,975 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:55:45,975 EPOCH 6 done: loss 15.6612 - lr 0.3000000 +2021-03-26 04:55:46,744 DEV : loss 11.731863021850586 - score 0.7912 +2021-03-26 04:55:46,762 BAD EPOCHS (no improvement): 0 +2021-03-26 04:55:56,548 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:55:57,537 epoch 7 - iter 2/25 - loss 16.28380728 - samples/sec: 129.78 - lr: 0.300000 +2021-03-26 04:55:58,585 epoch 7 - iter 4/25 - loss 15.14429021 - samples/sec: 122.35 - lr: 0.300000 +2021-03-26 04:55:59,648 epoch 7 - iter 6/25 - loss 14.99506346 - samples/sec: 120.47 - lr: 0.300000 +2021-03-26 04:56:00,693 epoch 7 - iter 8/25 - loss 14.78533983 - samples/sec: 122.68 - lr: 0.300000 +2021-03-26 04:56:01,754 epoch 7 - iter 10/25 - loss 14.55287571 - samples/sec: 120.78 - lr: 0.300000 +2021-03-26 04:56:02,840 epoch 7 - iter 12/25 - loss 14.56592727 - samples/sec: 118.11 - lr: 0.300000 +2021-03-26 04:56:03,893 epoch 7 - iter 14/25 - loss 14.49070978 - samples/sec: 121.72 - lr: 0.300000 +2021-03-26 04:56:04,868 epoch 7 - iter 16/25 - loss 14.42348069 - samples/sec: 131.41 - lr: 0.300000 +2021-03-26 04:56:05,909 epoch 7 - iter 18/25 - loss 14.40482251 - samples/sec: 123.29 - lr: 0.300000 +2021-03-26 04:56:06,910 epoch 7 - iter 20/25 - loss 14.48805385 - samples/sec: 128.04 - lr: 0.300000 +2021-03-26 04:56:08,036 epoch 7 - iter 22/25 - loss 14.39600455 - samples/sec: 113.87 - lr: 0.300000 +2021-03-26 04:56:09,130 epoch 7 - iter 24/25 - loss 14.09759820 - samples/sec: 117.17 - lr: 0.300000 +2021-03-26 04:56:09,678 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:56:09,680 EPOCH 7 done: loss 14.0652 - lr 0.3000000 +2021-03-26 04:56:10,490 DEV : loss 11.188261985778809 - score 0.8053 +2021-03-26 04:56:10,518 BAD EPOCHS (no improvement): 0 +2021-03-26 04:56:20,507 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:56:21,589 epoch 8 - iter 2/25 - loss 12.58739758 - samples/sec: 118.76 - lr: 0.300000 +2021-03-26 04:56:22,604 epoch 8 - iter 4/25 - loss 13.04474783 - samples/sec: 126.36 - lr: 0.300000 +2021-03-26 04:56:23,632 epoch 8 - iter 6/25 - loss 12.55590312 - samples/sec: 124.70 - lr: 0.300000 +2021-03-26 04:56:24,546 epoch 8 - iter 8/25 - loss 12.62532973 - samples/sec: 140.18 - lr: 0.300000 +2021-03-26 04:56:25,643 epoch 8 - iter 10/25 - loss 12.26407042 - samples/sec: 116.98 - lr: 0.300000 +2021-03-26 04:56:26,633 epoch 8 - iter 12/25 - loss 12.92811076 - samples/sec: 129.49 - lr: 0.300000 +2021-03-26 04:56:27,553 epoch 8 - iter 14/25 - loss 12.84461852 - samples/sec: 139.37 - lr: 0.300000 +2021-03-26 04:56:28,491 epoch 8 - iter 16/25 - loss 12.82955539 - samples/sec: 136.64 - lr: 0.300000 +2021-03-26 04:56:29,583 epoch 8 - iter 18/25 - loss 12.95769893 - samples/sec: 117.41 - lr: 0.300000 +2021-03-26 04:56:30,550 epoch 8 - iter 20/25 - loss 12.84561110 - samples/sec: 133.25 - lr: 0.300000 +2021-03-26 04:56:31,578 epoch 8 - iter 22/25 - loss 12.85654727 - samples/sec: 124.80 - lr: 0.300000 +2021-03-26 04:56:32,956 epoch 8 - iter 24/25 - loss 12.81559539 - samples/sec: 93.12 - lr: 0.300000 +2021-03-26 04:56:33,390 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:56:33,392 EPOCH 8 done: loss 12.8214 - lr 0.3000000 +2021-03-26 04:56:34,251 DEV : loss 10.346247673034668 - score 0.8115 +2021-03-26 04:56:34,276 BAD EPOCHS (no improvement): 0 +2021-03-26 04:56:44,165 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:56:45,225 epoch 9 - iter 2/25 - loss 12.45041800 - samples/sec: 121.07 - lr: 0.300000 +2021-03-26 04:56:46,207 epoch 9 - iter 4/25 - loss 11.98195148 - samples/sec: 130.71 - lr: 0.300000 +2021-03-26 04:56:47,232 epoch 9 - iter 6/25 - loss 12.15954542 - samples/sec: 124.98 - lr: 0.300000 +2021-03-26 04:56:48,177 epoch 9 - iter 8/25 - loss 11.93493176 - samples/sec: 135.86 - lr: 0.300000 +2021-03-26 04:56:49,332 epoch 9 - iter 10/25 - loss 12.10410681 - samples/sec: 110.98 - lr: 0.300000 +2021-03-26 04:56:50,297 epoch 9 - iter 12/25 - loss 12.03921103 - samples/sec: 133.04 - lr: 0.300000 +2021-03-26 04:56:51,347 epoch 9 - iter 14/25 - loss 12.02120093 - samples/sec: 122.14 - lr: 0.300000 +2021-03-26 04:56:52,434 epoch 9 - iter 16/25 - loss 11.79854709 - samples/sec: 117.91 - lr: 0.300000 +2021-03-26 04:56:53,404 epoch 9 - iter 18/25 - loss 11.61538373 - samples/sec: 132.20 - lr: 0.300000 +2021-03-26 04:56:54,874 epoch 9 - iter 20/25 - loss 11.94448543 - samples/sec: 87.18 - lr: 0.300000 +2021-03-26 04:56:56,111 epoch 9 - iter 22/25 - loss 11.81401552 - samples/sec: 103.63 - lr: 0.300000 +2021-03-26 04:56:57,187 epoch 9 - iter 24/25 - loss 11.70084679 - samples/sec: 119.14 - lr: 0.300000 +2021-03-26 04:56:57,702 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:56:57,704 EPOCH 9 done: loss 11.6376 - lr 0.3000000 +2021-03-26 04:56:58,547 DEV : loss 9.364679336547852 - score 0.8449 +2021-03-26 04:56:58,572 BAD EPOCHS (no improvement): 0 +2021-03-26 04:57:08,513 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:57:09,634 epoch 10 - iter 2/25 - loss 10.44735384 - samples/sec: 114.43 - lr: 0.300000 +2021-03-26 04:57:10,698 epoch 10 - iter 4/25 - loss 10.18703008 - samples/sec: 120.58 - lr: 0.300000 +2021-03-26 04:57:11,751 epoch 10 - iter 6/25 - loss 10.51863972 - samples/sec: 121.84 - lr: 0.300000 +2021-03-26 04:57:12,786 epoch 10 - iter 8/25 - loss 10.90037954 - samples/sec: 123.85 - lr: 0.300000 +2021-03-26 04:57:13,822 epoch 10 - iter 10/25 - loss 11.50496826 - samples/sec: 123.63 - lr: 0.300000 +2021-03-26 04:57:14,889 epoch 10 - iter 12/25 - loss 11.35811400 - samples/sec: 120.19 - lr: 0.300000 +2021-03-26 04:57:15,945 epoch 10 - iter 14/25 - loss 11.25915929 - samples/sec: 121.32 - lr: 0.300000 +2021-03-26 04:57:16,881 epoch 10 - iter 16/25 - loss 11.20458829 - samples/sec: 136.99 - lr: 0.300000 +2021-03-26 04:57:17,882 epoch 10 - iter 18/25 - loss 11.15681087 - samples/sec: 128.10 - lr: 0.300000 +2021-03-26 04:57:18,788 epoch 10 - iter 20/25 - loss 11.15392790 - samples/sec: 141.52 - lr: 0.300000 +2021-03-26 04:57:19,925 epoch 10 - iter 22/25 - loss 11.13716615 - samples/sec: 112.77 - lr: 0.300000 +2021-03-26 04:57:20,924 epoch 10 - iter 24/25 - loss 11.03530443 - samples/sec: 128.44 - lr: 0.300000 +2021-03-26 04:57:21,328 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:57:21,329 EPOCH 10 done: loss 10.9343 - lr 0.3000000 +2021-03-26 04:57:22,082 DEV : loss 8.626761436462402 - score 0.8509 +2021-03-26 04:57:22,106 BAD EPOCHS (no improvement): 0 +2021-03-26 04:57:31,973 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:57:32,981 epoch 11 - iter 2/25 - loss 9.40605307 - samples/sec: 127.39 - lr: 0.300000 +2021-03-26 04:57:34,071 epoch 11 - iter 4/25 - loss 10.33374119 - samples/sec: 117.67 - lr: 0.300000 +2021-03-26 04:57:35,139 epoch 11 - iter 6/25 - loss 10.77863773 - samples/sec: 120.04 - lr: 0.300000 +2021-03-26 04:57:36,180 epoch 11 - iter 8/25 - loss 10.52217364 - samples/sec: 123.06 - lr: 0.300000 +2021-03-26 04:57:37,194 epoch 11 - iter 10/25 - loss 10.40267553 - samples/sec: 126.47 - lr: 0.300000 +2021-03-26 04:57:38,278 epoch 11 - iter 12/25 - loss 10.46414081 - samples/sec: 118.37 - lr: 0.300000 +2021-03-26 04:57:39,458 epoch 11 - iter 14/25 - loss 10.36640521 - samples/sec: 108.61 - lr: 0.300000 +2021-03-26 04:57:40,503 epoch 11 - iter 16/25 - loss 10.39492095 - samples/sec: 122.63 - lr: 0.300000 +2021-03-26 04:57:41,447 epoch 11 - iter 18/25 - loss 10.42191193 - samples/sec: 135.87 - lr: 0.300000 +2021-03-26 04:57:42,485 epoch 11 - iter 20/25 - loss 10.45917497 - samples/sec: 123.45 - lr: 0.300000 +2021-03-26 04:57:43,656 epoch 11 - iter 22/25 - loss 10.40492392 - samples/sec: 109.56 - lr: 0.300000 +2021-03-26 04:57:44,757 epoch 11 - iter 24/25 - loss 10.45924691 - samples/sec: 116.42 - lr: 0.300000 +2021-03-26 04:57:45,237 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:57:45,239 EPOCH 11 done: loss 10.4692 - lr 0.3000000 +2021-03-26 04:57:46,067 DEV : loss 8.071310997009277 - score 0.8623 +2021-03-26 04:57:46,101 BAD EPOCHS (no improvement): 0 +2021-03-26 04:57:55,909 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:57:57,009 epoch 12 - iter 2/25 - loss 9.19157362 - samples/sec: 116.70 - lr: 0.300000 +2021-03-26 04:57:58,023 epoch 12 - iter 4/25 - loss 10.71440196 - samples/sec: 126.52 - lr: 0.300000 +2021-03-26 04:57:59,154 epoch 12 - iter 6/25 - loss 10.32768408 - samples/sec: 113.33 - lr: 0.300000 +2021-03-26 04:58:00,159 epoch 12 - iter 8/25 - loss 10.20629978 - samples/sec: 127.54 - lr: 0.300000 +2021-03-26 04:58:01,389 epoch 12 - iter 10/25 - loss 9.83114662 - samples/sec: 104.13 - lr: 0.300000 +2021-03-26 04:58:02,493 epoch 12 - iter 12/25 - loss 9.89521742 - samples/sec: 116.14 - lr: 0.300000 +2021-03-26 04:58:03,608 epoch 12 - iter 14/25 - loss 9.87386608 - samples/sec: 114.93 - lr: 0.300000 +2021-03-26 04:58:04,675 epoch 12 - iter 16/25 - loss 9.74479634 - samples/sec: 120.12 - lr: 0.300000 +2021-03-26 04:58:05,716 epoch 12 - iter 18/25 - loss 9.72899850 - samples/sec: 123.32 - lr: 0.300000 +2021-03-26 04:58:06,762 epoch 12 - iter 20/25 - loss 9.79978871 - samples/sec: 122.56 - lr: 0.300000 +2021-03-26 04:58:07,844 epoch 12 - iter 22/25 - loss 9.69491967 - samples/sec: 118.46 - lr: 0.300000 +2021-03-26 04:58:08,816 epoch 12 - iter 24/25 - loss 9.54800685 - samples/sec: 131.85 - lr: 0.300000 +2021-03-26 04:58:09,262 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:58:09,263 EPOCH 12 done: loss 9.6618 - lr 0.3000000 +2021-03-26 04:58:10,044 DEV : loss 7.919666290283203 - score 0.8589 +2021-03-26 04:58:10,061 BAD EPOCHS (no improvement): 1 +2021-03-26 04:58:10,062 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:58:11,195 epoch 13 - iter 2/25 - loss 8.74641275 - samples/sec: 113.13 - lr: 0.300000 +2021-03-26 04:58:12,270 epoch 13 - iter 4/25 - loss 8.56014812 - samples/sec: 119.38 - lr: 0.300000 +2021-03-26 04:58:13,245 epoch 13 - iter 6/25 - loss 8.30085142 - samples/sec: 131.41 - lr: 0.300000 +2021-03-26 04:58:14,181 epoch 13 - iter 8/25 - loss 8.65799254 - samples/sec: 137.01 - lr: 0.300000 +2021-03-26 04:58:15,270 epoch 13 - iter 10/25 - loss 8.63084922 - samples/sec: 117.73 - lr: 0.300000 +2021-03-26 04:58:16,307 epoch 13 - iter 12/25 - loss 8.77607254 - samples/sec: 123.64 - lr: 0.300000 +2021-03-26 04:58:17,439 epoch 13 - iter 14/25 - loss 8.87740670 - samples/sec: 113.25 - lr: 0.300000 +2021-03-26 04:58:18,402 epoch 13 - iter 16/25 - loss 8.94392380 - samples/sec: 133.12 - lr: 0.300000 +2021-03-26 04:58:19,401 epoch 13 - iter 18/25 - loss 8.93109330 - samples/sec: 128.22 - lr: 0.300000 +2021-03-26 04:58:20,386 epoch 13 - iter 20/25 - loss 8.86445279 - samples/sec: 130.20 - lr: 0.300000 +2021-03-26 04:58:21,511 epoch 13 - iter 22/25 - loss 8.93233819 - samples/sec: 114.04 - lr: 0.300000 +2021-03-26 04:58:22,598 epoch 13 - iter 24/25 - loss 8.89520804 - samples/sec: 118.00 - lr: 0.300000 +2021-03-26 04:58:23,028 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:58:23,028 EPOCH 13 done: loss 8.9468 - lr 0.3000000 +2021-03-26 04:58:23,801 DEV : loss 7.6833624839782715 - score 0.8617 +2021-03-26 04:58:23,827 BAD EPOCHS (no improvement): 2 +2021-03-26 04:58:23,827 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:58:24,972 epoch 14 - iter 2/25 - loss 7.01421189 - samples/sec: 112.04 - lr: 0.300000 +2021-03-26 04:58:25,993 epoch 14 - iter 4/25 - loss 7.10888994 - samples/sec: 125.71 - lr: 0.300000 +2021-03-26 04:58:27,016 epoch 14 - iter 6/25 - loss 7.76643205 - samples/sec: 125.28 - lr: 0.300000 +2021-03-26 04:58:28,055 epoch 14 - iter 8/25 - loss 7.94494444 - samples/sec: 123.36 - lr: 0.300000 +2021-03-26 04:58:29,058 epoch 14 - iter 10/25 - loss 8.00102415 - samples/sec: 127.96 - lr: 0.300000 +2021-03-26 04:58:30,062 epoch 14 - iter 12/25 - loss 8.27631696 - samples/sec: 127.63 - lr: 0.300000 +2021-03-26 04:58:31,116 epoch 14 - iter 14/25 - loss 8.18886886 - samples/sec: 121.81 - lr: 0.300000 +2021-03-26 04:58:32,138 epoch 14 - iter 16/25 - loss 8.23848945 - samples/sec: 125.42 - lr: 0.300000 +2021-03-26 04:58:33,058 epoch 14 - iter 18/25 - loss 8.27528975 - samples/sec: 139.32 - lr: 0.300000 +2021-03-26 04:58:34,087 epoch 14 - iter 20/25 - loss 8.41562710 - samples/sec: 124.63 - lr: 0.300000 +2021-03-26 04:58:35,189 epoch 14 - iter 22/25 - loss 8.42097157 - samples/sec: 116.36 - lr: 0.300000 +2021-03-26 04:58:36,156 epoch 14 - iter 24/25 - loss 8.52386268 - samples/sec: 132.58 - lr: 0.300000 +2021-03-26 04:58:36,556 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:58:36,557 EPOCH 14 done: loss 8.5702 - lr 0.3000000 +2021-03-26 04:58:37,316 DEV : loss 7.460776329040527 - score 0.8689 +2021-03-26 04:58:37,342 BAD EPOCHS (no improvement): 0 +2021-03-26 04:58:47,205 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:58:48,451 epoch 15 - iter 2/25 - loss 8.41517854 - samples/sec: 102.99 - lr: 0.300000 +2021-03-26 04:58:49,597 epoch 15 - iter 4/25 - loss 8.74319327 - samples/sec: 111.78 - lr: 0.300000 +2021-03-26 04:58:50,617 epoch 15 - iter 6/25 - loss 8.38857420 - samples/sec: 125.83 - lr: 0.300000 +2021-03-26 04:58:51,743 epoch 15 - iter 8/25 - loss 8.21414596 - samples/sec: 113.80 - lr: 0.300000 +2021-03-26 04:58:52,739 epoch 15 - iter 10/25 - loss 8.17341042 - samples/sec: 128.92 - lr: 0.300000 +2021-03-26 04:58:53,723 epoch 15 - iter 12/25 - loss 8.06812867 - samples/sec: 130.47 - lr: 0.300000 +2021-03-26 04:58:54,790 epoch 15 - iter 14/25 - loss 8.05767250 - samples/sec: 120.11 - lr: 0.300000 +2021-03-26 04:58:55,810 epoch 15 - iter 16/25 - loss 7.91986740 - samples/sec: 125.62 - lr: 0.300000 +2021-03-26 04:58:56,808 epoch 15 - iter 18/25 - loss 8.05734306 - samples/sec: 128.59 - lr: 0.300000 +2021-03-26 04:58:57,799 epoch 15 - iter 20/25 - loss 8.08630199 - samples/sec: 129.33 - lr: 0.300000 +2021-03-26 04:58:58,976 epoch 15 - iter 22/25 - loss 8.16225732 - samples/sec: 108.90 - lr: 0.300000 +2021-03-26 04:59:00,036 epoch 15 - iter 24/25 - loss 8.11989534 - samples/sec: 120.91 - lr: 0.300000 +2021-03-26 04:59:00,437 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:59:00,437 EPOCH 15 done: loss 8.1712 - lr 0.3000000 +2021-03-26 04:59:01,181 DEV : loss 7.299462795257568 - score 0.8706 +2021-03-26 04:59:01,205 BAD EPOCHS (no improvement): 0 +2021-03-26 04:59:11,000 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:59:11,995 epoch 16 - iter 2/25 - loss 6.63340878 - samples/sec: 129.13 - lr: 0.300000 +2021-03-26 04:59:13,026 epoch 16 - iter 4/25 - loss 7.67077863 - samples/sec: 124.28 - lr: 0.300000 +2021-03-26 04:59:14,055 epoch 16 - iter 6/25 - loss 7.59115497 - samples/sec: 124.53 - lr: 0.300000 +2021-03-26 04:59:15,050 epoch 16 - iter 8/25 - loss 7.71391118 - samples/sec: 128.85 - lr: 0.300000 +2021-03-26 04:59:16,109 epoch 16 - iter 10/25 - loss 7.66146083 - samples/sec: 121.09 - lr: 0.300000 +2021-03-26 04:59:17,075 epoch 16 - iter 12/25 - loss 7.44220650 - samples/sec: 132.90 - lr: 0.300000 +2021-03-26 04:59:18,117 epoch 16 - iter 14/25 - loss 7.54290489 - samples/sec: 123.04 - lr: 0.300000 +2021-03-26 04:59:19,145 epoch 16 - iter 16/25 - loss 7.69426468 - samples/sec: 124.63 - lr: 0.300000 +2021-03-26 04:59:20,158 epoch 16 - iter 18/25 - loss 7.78613400 - samples/sec: 126.58 - lr: 0.300000 +2021-03-26 04:59:21,273 epoch 16 - iter 20/25 - loss 7.89610012 - samples/sec: 114.96 - lr: 0.300000 +2021-03-26 04:59:22,354 epoch 16 - iter 22/25 - loss 7.92611876 - samples/sec: 118.72 - lr: 0.300000 +2021-03-26 04:59:23,421 epoch 16 - iter 24/25 - loss 7.84850351 - samples/sec: 120.08 - lr: 0.300000 +2021-03-26 04:59:23,821 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:59:23,822 EPOCH 16 done: loss 7.8628 - lr 0.3000000 +2021-03-26 04:59:24,579 DEV : loss 7.393617153167725 - score 0.8748 +2021-03-26 04:59:24,604 BAD EPOCHS (no improvement): 0 +2021-03-26 04:59:34,384 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:59:35,451 epoch 17 - iter 2/25 - loss 8.34224653 - samples/sec: 120.25 - lr: 0.300000 +2021-03-26 04:59:36,452 epoch 17 - iter 4/25 - loss 7.54335344 - samples/sec: 128.08 - lr: 0.300000 +2021-03-26 04:59:37,404 epoch 17 - iter 6/25 - loss 7.42643118 - samples/sec: 134.66 - lr: 0.300000 +2021-03-26 04:59:38,517 epoch 17 - iter 8/25 - loss 7.16716546 - samples/sec: 115.23 - lr: 0.300000 +2021-03-26 04:59:39,689 epoch 17 - iter 10/25 - loss 7.25732112 - samples/sec: 109.28 - lr: 0.300000 +2021-03-26 04:59:40,758 epoch 17 - iter 12/25 - loss 7.16744749 - samples/sec: 119.97 - lr: 0.300000 +2021-03-26 04:59:41,835 epoch 17 - iter 14/25 - loss 7.25663158 - samples/sec: 118.92 - lr: 0.300000 +2021-03-26 04:59:42,817 epoch 17 - iter 16/25 - loss 7.41065523 - samples/sec: 130.45 - lr: 0.300000 +2021-03-26 04:59:43,875 epoch 17 - iter 18/25 - loss 7.50303170 - samples/sec: 121.18 - lr: 0.300000 +2021-03-26 04:59:44,911 epoch 17 - iter 20/25 - loss 7.51523950 - samples/sec: 123.69 - lr: 0.300000 +2021-03-26 04:59:46,000 epoch 17 - iter 22/25 - loss 7.52067937 - samples/sec: 117.71 - lr: 0.300000 +2021-03-26 04:59:46,976 epoch 17 - iter 24/25 - loss 7.43268249 - samples/sec: 131.49 - lr: 0.300000 +2021-03-26 04:59:47,387 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:59:47,388 EPOCH 17 done: loss 7.4542 - lr 0.3000000 +2021-03-26 04:59:48,170 DEV : loss 6.938924789428711 - score 0.881 +2021-03-26 04:59:48,194 BAD EPOCHS (no improvement): 0 +2021-03-26 04:59:57,881 ---------------------------------------------------------------------------------------------------- +2021-03-26 04:59:59,032 epoch 18 - iter 2/25 - loss 6.95253706 - samples/sec: 111.43 - lr: 0.300000 +2021-03-26 04:59:59,949 epoch 18 - iter 4/25 - loss 6.14172232 - samples/sec: 139.96 - lr: 0.300000 +2021-03-26 05:00:00,996 epoch 18 - iter 6/25 - loss 6.56162341 - samples/sec: 122.60 - lr: 0.300000 +2021-03-26 05:00:01,986 epoch 18 - iter 8/25 - loss 6.49303353 - samples/sec: 129.52 - lr: 0.300000 +2021-03-26 05:00:02,974 epoch 18 - iter 10/25 - loss 6.57700791 - samples/sec: 129.71 - lr: 0.300000 +2021-03-26 05:00:04,099 epoch 18 - iter 12/25 - loss 6.94862250 - samples/sec: 113.97 - lr: 0.300000 +2021-03-26 05:00:05,061 epoch 18 - iter 14/25 - loss 6.86058249 - samples/sec: 133.17 - lr: 0.300000 +2021-03-26 05:00:06,167 epoch 18 - iter 16/25 - loss 6.81615442 - samples/sec: 116.05 - lr: 0.300000 +2021-03-26 05:00:07,428 epoch 18 - iter 18/25 - loss 6.82973880 - samples/sec: 101.67 - lr: 0.300000 +2021-03-26 05:00:08,594 epoch 18 - iter 20/25 - loss 6.90358000 - samples/sec: 109.95 - lr: 0.300000 +2021-03-26 05:00:09,602 epoch 18 - iter 22/25 - loss 7.04449615 - samples/sec: 127.11 - lr: 0.300000 +2021-03-26 05:00:10,567 epoch 18 - iter 24/25 - loss 7.04610141 - samples/sec: 132.83 - lr: 0.300000 +2021-03-26 05:00:10,969 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:00:10,969 EPOCH 18 done: loss 7.0938 - lr 0.3000000 +2021-03-26 05:00:11,734 DEV : loss 6.572293758392334 - score 0.8845 +2021-03-26 05:00:11,758 BAD EPOCHS (no improvement): 0 +2021-03-26 05:00:21,476 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:00:22,656 epoch 19 - iter 2/25 - loss 6.50636125 - samples/sec: 108.71 - lr: 0.300000 +2021-03-26 05:00:23,684 epoch 19 - iter 4/25 - loss 6.44323480 - samples/sec: 124.77 - lr: 0.300000 +2021-03-26 05:00:24,684 epoch 19 - iter 6/25 - loss 6.69485251 - samples/sec: 128.42 - lr: 0.300000 +2021-03-26 05:00:25,731 epoch 19 - iter 8/25 - loss 6.91241974 - samples/sec: 122.59 - lr: 0.300000 +2021-03-26 05:00:26,727 epoch 19 - iter 10/25 - loss 6.92695212 - samples/sec: 128.70 - lr: 0.300000 +2021-03-26 05:00:27,693 epoch 19 - iter 12/25 - loss 6.99924886 - samples/sec: 132.68 - lr: 0.300000 +2021-03-26 05:00:28,765 epoch 19 - iter 14/25 - loss 6.83471465 - samples/sec: 119.55 - lr: 0.300000 +2021-03-26 05:00:29,737 epoch 19 - iter 16/25 - loss 7.05365744 - samples/sec: 131.98 - lr: 0.300000 +2021-03-26 05:00:30,739 epoch 19 - iter 18/25 - loss 7.09827262 - samples/sec: 127.87 - lr: 0.300000 +2021-03-26 05:00:31,759 epoch 19 - iter 20/25 - loss 7.02771170 - samples/sec: 125.77 - lr: 0.300000 +2021-03-26 05:00:32,731 epoch 19 - iter 22/25 - loss 7.03901434 - samples/sec: 131.91 - lr: 0.300000 +2021-03-26 05:00:33,745 epoch 19 - iter 24/25 - loss 7.00594433 - samples/sec: 126.41 - lr: 0.300000 +2021-03-26 05:00:34,200 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:00:34,200 EPOCH 19 done: loss 6.8745 - lr 0.3000000 +2021-03-26 05:00:34,975 DEV : loss 7.177085876464844 - score 0.8767 +2021-03-26 05:00:35,003 BAD EPOCHS (no improvement): 1 +2021-03-26 05:00:35,004 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:00:35,956 epoch 20 - iter 2/25 - loss 5.38716412 - samples/sec: 134.74 - lr: 0.300000 +2021-03-26 05:00:36,939 epoch 20 - iter 4/25 - loss 5.85706186 - samples/sec: 130.37 - lr: 0.300000 +2021-03-26 05:00:37,893 epoch 20 - iter 6/25 - loss 6.37915858 - samples/sec: 134.51 - lr: 0.300000 +2021-03-26 05:00:38,895 epoch 20 - iter 8/25 - loss 6.47841519 - samples/sec: 127.89 - lr: 0.300000 +2021-03-26 05:00:39,861 epoch 20 - iter 10/25 - loss 6.55083828 - samples/sec: 132.77 - lr: 0.300000 +2021-03-26 05:00:40,893 epoch 20 - iter 12/25 - loss 6.38643750 - samples/sec: 124.10 - lr: 0.300000 +2021-03-26 05:00:41,941 epoch 20 - iter 14/25 - loss 6.53920797 - samples/sec: 122.31 - lr: 0.300000 +2021-03-26 05:00:42,988 epoch 20 - iter 16/25 - loss 6.48416033 - samples/sec: 122.47 - lr: 0.300000 +2021-03-26 05:00:43,993 epoch 20 - iter 18/25 - loss 6.45129516 - samples/sec: 127.53 - lr: 0.300000 +2021-03-26 05:00:45,022 epoch 20 - iter 20/25 - loss 6.51733649 - samples/sec: 124.62 - lr: 0.300000 +2021-03-26 05:00:46,087 epoch 20 - iter 22/25 - loss 6.48917937 - samples/sec: 120.25 - lr: 0.300000 +2021-03-26 05:00:47,154 epoch 20 - iter 24/25 - loss 6.53120399 - samples/sec: 120.22 - lr: 0.300000 +2021-03-26 05:00:47,624 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:00:47,624 EPOCH 20 done: loss 6.5602 - lr 0.3000000 +2021-03-26 05:00:48,393 DEV : loss 6.406036376953125 - score 0.8849 +2021-03-26 05:00:48,419 BAD EPOCHS (no improvement): 0 +2021-03-26 05:00:57,975 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:00:58,986 epoch 21 - iter 2/25 - loss 5.07812023 - samples/sec: 126.91 - lr: 0.300000 +2021-03-26 05:00:59,931 epoch 21 - iter 4/25 - loss 5.87068987 - samples/sec: 135.69 - lr: 0.300000 +2021-03-26 05:01:00,899 epoch 21 - iter 6/25 - loss 5.90569671 - samples/sec: 132.46 - lr: 0.300000 +2021-03-26 05:01:02,123 epoch 21 - iter 8/25 - loss 5.89209396 - samples/sec: 104.74 - lr: 0.300000 +2021-03-26 05:01:03,354 epoch 21 - iter 10/25 - loss 5.82085629 - samples/sec: 104.07 - lr: 0.300000 +2021-03-26 05:01:04,387 epoch 21 - iter 12/25 - loss 6.16112550 - samples/sec: 124.28 - lr: 0.300000 +2021-03-26 05:01:05,424 epoch 21 - iter 14/25 - loss 6.25945626 - samples/sec: 123.54 - lr: 0.300000 +2021-03-26 05:01:06,523 epoch 21 - iter 16/25 - loss 6.27312291 - samples/sec: 116.64 - lr: 0.300000 +2021-03-26 05:01:07,581 epoch 21 - iter 18/25 - loss 6.25390930 - samples/sec: 121.21 - lr: 0.300000 +2021-03-26 05:01:08,642 epoch 21 - iter 20/25 - loss 6.37842085 - samples/sec: 120.83 - lr: 0.300000 +2021-03-26 05:01:09,671 epoch 21 - iter 22/25 - loss 6.37292236 - samples/sec: 124.55 - lr: 0.300000 +2021-03-26 05:01:10,834 epoch 21 - iter 24/25 - loss 6.36688979 - samples/sec: 110.18 - lr: 0.300000 +2021-03-26 05:01:11,319 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:01:11,320 EPOCH 21 done: loss 6.3817 - lr 0.3000000 +2021-03-26 05:01:12,084 DEV : loss 6.48774528503418 - score 0.8919 +2021-03-26 05:01:12,109 BAD EPOCHS (no improvement): 0 +2021-03-26 05:01:22,011 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:01:23,013 epoch 22 - iter 2/25 - loss 5.74258280 - samples/sec: 128.14 - lr: 0.300000 +2021-03-26 05:01:24,013 epoch 22 - iter 4/25 - loss 5.54190350 - samples/sec: 128.14 - lr: 0.300000 +2021-03-26 05:01:24,976 epoch 22 - iter 6/25 - loss 5.63025824 - samples/sec: 133.34 - lr: 0.300000 +2021-03-26 05:01:25,973 epoch 22 - iter 8/25 - loss 5.75732350 - samples/sec: 128.54 - lr: 0.300000 +2021-03-26 05:01:26,991 epoch 22 - iter 10/25 - loss 5.95047402 - samples/sec: 126.02 - lr: 0.300000 +2021-03-26 05:01:27,996 epoch 22 - iter 12/25 - loss 5.88446764 - samples/sec: 127.57 - lr: 0.300000 +2021-03-26 05:01:29,167 epoch 22 - iter 14/25 - loss 5.96128140 - samples/sec: 109.47 - lr: 0.300000 +2021-03-26 05:01:30,173 epoch 22 - iter 16/25 - loss 5.94008636 - samples/sec: 127.36 - lr: 0.300000 +2021-03-26 05:01:31,295 epoch 22 - iter 18/25 - loss 5.92830536 - samples/sec: 114.24 - lr: 0.300000 +2021-03-26 05:01:32,361 epoch 22 - iter 20/25 - loss 5.93459527 - samples/sec: 120.31 - lr: 0.300000 +2021-03-26 05:01:33,365 epoch 22 - iter 22/25 - loss 5.97066474 - samples/sec: 127.60 - lr: 0.300000 +2021-03-26 05:01:34,466 epoch 22 - iter 24/25 - loss 5.97097691 - samples/sec: 116.40 - lr: 0.300000 +2021-03-26 05:01:34,874 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:01:34,874 EPOCH 22 done: loss 6.0013 - lr 0.3000000 +2021-03-26 05:01:35,606 DEV : loss 6.178506374359131 - score 0.8919 +2021-03-26 05:01:35,631 BAD EPOCHS (no improvement): 0 +2021-03-26 05:01:45,379 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:01:46,594 epoch 23 - iter 2/25 - loss 5.67629218 - samples/sec: 105.53 - lr: 0.300000 +2021-03-26 05:01:47,589 epoch 23 - iter 4/25 - loss 5.56583500 - samples/sec: 129.01 - lr: 0.300000 +2021-03-26 05:01:48,595 epoch 23 - iter 6/25 - loss 5.36302352 - samples/sec: 127.60 - lr: 0.300000 +2021-03-26 05:01:49,597 epoch 23 - iter 8/25 - loss 5.49623519 - samples/sec: 127.93 - lr: 0.300000 +2021-03-26 05:01:50,643 epoch 23 - iter 10/25 - loss 5.41584888 - samples/sec: 122.65 - lr: 0.300000 +2021-03-26 05:01:51,595 epoch 23 - iter 12/25 - loss 5.47491487 - samples/sec: 134.79 - lr: 0.300000 +2021-03-26 05:01:52,616 epoch 23 - iter 14/25 - loss 5.70795202 - samples/sec: 125.53 - lr: 0.300000 +2021-03-26 05:01:53,650 epoch 23 - iter 16/25 - loss 5.80719429 - samples/sec: 123.95 - lr: 0.300000 +2021-03-26 05:01:54,808 epoch 23 - iter 18/25 - loss 5.87293715 - samples/sec: 110.66 - lr: 0.300000 +2021-03-26 05:01:55,896 epoch 23 - iter 20/25 - loss 5.88861842 - samples/sec: 117.84 - lr: 0.300000 +2021-03-26 05:01:56,873 epoch 23 - iter 22/25 - loss 5.97894348 - samples/sec: 131.40 - lr: 0.300000 +2021-03-26 05:01:57,893 epoch 23 - iter 24/25 - loss 5.95262996 - samples/sec: 125.57 - lr: 0.300000 +2021-03-26 05:01:58,309 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:01:58,309 EPOCH 23 done: loss 5.9302 - lr 0.3000000 +2021-03-26 05:01:59,065 DEV : loss 6.203141212463379 - score 0.8923 +2021-03-26 05:01:59,090 BAD EPOCHS (no improvement): 0 +2021-03-26 05:02:08,747 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:02:09,693 epoch 24 - iter 2/25 - loss 5.83838153 - samples/sec: 135.60 - lr: 0.300000 +2021-03-26 05:02:10,704 epoch 24 - iter 4/25 - loss 5.43874466 - samples/sec: 126.97 - lr: 0.300000 +2021-03-26 05:02:11,898 epoch 24 - iter 6/25 - loss 5.50941141 - samples/sec: 107.31 - lr: 0.300000 +2021-03-26 05:02:12,935 epoch 24 - iter 8/25 - loss 5.53578192 - samples/sec: 123.72 - lr: 0.300000 +2021-03-26 05:02:13,958 epoch 24 - iter 10/25 - loss 5.54665751 - samples/sec: 125.25 - lr: 0.300000 +2021-03-26 05:02:15,095 epoch 24 - iter 12/25 - loss 5.65865084 - samples/sec: 112.75 - lr: 0.300000 +2021-03-26 05:02:16,059 epoch 24 - iter 14/25 - loss 5.69921316 - samples/sec: 132.83 - lr: 0.300000 +2021-03-26 05:02:17,018 epoch 24 - iter 16/25 - loss 5.58560240 - samples/sec: 133.82 - lr: 0.300000 +2021-03-26 05:02:18,107 epoch 24 - iter 18/25 - loss 5.56636564 - samples/sec: 117.65 - lr: 0.300000 +2021-03-26 05:02:19,165 epoch 24 - iter 20/25 - loss 5.66292870 - samples/sec: 121.21 - lr: 0.300000 +2021-03-26 05:02:20,101 epoch 24 - iter 22/25 - loss 5.70732947 - samples/sec: 137.00 - lr: 0.300000 +2021-03-26 05:02:21,104 epoch 24 - iter 24/25 - loss 5.70914570 - samples/sec: 127.80 - lr: 0.300000 +2021-03-26 05:02:21,512 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:02:21,513 EPOCH 24 done: loss 5.6692 - lr 0.3000000 +2021-03-26 05:02:22,271 DEV : loss 6.174157619476318 - score 0.8896 +2021-03-26 05:02:22,295 BAD EPOCHS (no improvement): 1 +2021-03-26 05:02:22,296 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:02:23,296 epoch 25 - iter 2/25 - loss 5.88089275 - samples/sec: 128.16 - lr: 0.300000 +2021-03-26 05:02:24,282 epoch 25 - iter 4/25 - loss 5.79316306 - samples/sec: 130.07 - lr: 0.300000 +2021-03-26 05:02:25,277 epoch 25 - iter 6/25 - loss 5.90174588 - samples/sec: 128.83 - lr: 0.300000 +2021-03-26 05:02:26,328 epoch 25 - iter 8/25 - loss 5.62725395 - samples/sec: 121.91 - lr: 0.300000 +2021-03-26 05:02:27,533 epoch 25 - iter 10/25 - loss 5.43041258 - samples/sec: 106.42 - lr: 0.300000 +2021-03-26 05:02:28,504 epoch 25 - iter 12/25 - loss 5.60077206 - samples/sec: 131.97 - lr: 0.300000 +2021-03-26 05:02:29,625 epoch 25 - iter 14/25 - loss 5.79898020 - samples/sec: 114.35 - lr: 0.300000 +2021-03-26 05:02:30,630 epoch 25 - iter 16/25 - loss 5.70678404 - samples/sec: 127.65 - lr: 0.300000 +2021-03-26 05:02:31,573 epoch 25 - iter 18/25 - loss 5.63503326 - samples/sec: 135.94 - lr: 0.300000 +2021-03-26 05:02:32,604 epoch 25 - iter 20/25 - loss 5.63442178 - samples/sec: 124.30 - lr: 0.300000 +2021-03-26 05:02:33,687 epoch 25 - iter 22/25 - loss 5.58001575 - samples/sec: 118.33 - lr: 0.300000 +2021-03-26 05:02:34,799 epoch 25 - iter 24/25 - loss 5.57944733 - samples/sec: 115.26 - lr: 0.300000 +2021-03-26 05:02:35,209 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:02:35,210 EPOCH 25 done: loss 5.6108 - lr 0.3000000 +2021-03-26 05:02:35,984 DEV : loss 6.509549617767334 - score 0.8944 +2021-03-26 05:02:36,023 BAD EPOCHS (no improvement): 0 +2021-03-26 05:02:45,731 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:02:46,977 epoch 26 - iter 2/25 - loss 4.82402134 - samples/sec: 102.98 - lr: 0.300000 +2021-03-26 05:02:47,925 epoch 26 - iter 4/25 - loss 4.93847728 - samples/sec: 135.20 - lr: 0.300000 +2021-03-26 05:02:48,867 epoch 26 - iter 6/25 - loss 4.81588984 - samples/sec: 136.11 - lr: 0.300000 +2021-03-26 05:02:49,851 epoch 26 - iter 8/25 - loss 4.91123033 - samples/sec: 130.28 - lr: 0.300000 +2021-03-26 05:02:50,956 epoch 26 - iter 10/25 - loss 5.07624397 - samples/sec: 115.98 - lr: 0.300000 +2021-03-26 05:02:51,992 epoch 26 - iter 12/25 - loss 5.22970748 - samples/sec: 123.72 - lr: 0.300000 +2021-03-26 05:02:53,125 epoch 26 - iter 14/25 - loss 5.27702308 - samples/sec: 113.17 - lr: 0.300000 +2021-03-26 05:02:54,070 epoch 26 - iter 16/25 - loss 5.36155292 - samples/sec: 135.67 - lr: 0.300000 +2021-03-26 05:02:55,220 epoch 26 - iter 18/25 - loss 5.42107211 - samples/sec: 111.37 - lr: 0.300000 +2021-03-26 05:02:56,224 epoch 26 - iter 20/25 - loss 5.44072261 - samples/sec: 128.42 - lr: 0.300000 +2021-03-26 05:02:57,386 epoch 26 - iter 22/25 - loss 5.39594821 - samples/sec: 110.36 - lr: 0.300000 +2021-03-26 05:02:58,489 epoch 26 - iter 24/25 - loss 5.33946991 - samples/sec: 116.16 - lr: 0.300000 +2021-03-26 05:02:58,856 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:02:58,856 EPOCH 26 done: loss 5.3082 - lr 0.3000000 +2021-03-26 05:03:00,949 DEV : loss 6.412842750549316 - score 0.8935 +2021-03-26 05:03:00,966 BAD EPOCHS (no improvement): 1 +2021-03-26 05:03:00,967 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:03:01,938 epoch 27 - iter 2/25 - loss 5.76803780 - samples/sec: 132.05 - lr: 0.300000 +2021-03-26 05:03:02,874 epoch 27 - iter 4/25 - loss 5.79881656 - samples/sec: 136.92 - lr: 0.300000 +2021-03-26 05:03:03,970 epoch 27 - iter 6/25 - loss 5.63473813 - samples/sec: 116.95 - lr: 0.300000 +2021-03-26 05:03:05,029 epoch 27 - iter 8/25 - loss 5.36606169 - samples/sec: 121.11 - lr: 0.300000 +2021-03-26 05:03:06,019 epoch 27 - iter 10/25 - loss 5.34177275 - samples/sec: 129.56 - lr: 0.300000 +2021-03-26 05:03:07,115 epoch 27 - iter 12/25 - loss 5.12930922 - samples/sec: 116.96 - lr: 0.300000 +2021-03-26 05:03:08,042 epoch 27 - iter 14/25 - loss 5.18538295 - samples/sec: 138.48 - lr: 0.300000 +2021-03-26 05:03:09,057 epoch 27 - iter 16/25 - loss 5.25303575 - samples/sec: 126.31 - lr: 0.300000 +2021-03-26 05:03:10,077 epoch 27 - iter 18/25 - loss 5.25039638 - samples/sec: 125.59 - lr: 0.300000 +2021-03-26 05:03:11,030 epoch 27 - iter 20/25 - loss 5.24911377 - samples/sec: 134.58 - lr: 0.300000 +2021-03-26 05:03:11,958 epoch 27 - iter 22/25 - loss 5.26660614 - samples/sec: 138.14 - lr: 0.300000 +2021-03-26 05:03:12,919 epoch 27 - iter 24/25 - loss 5.24369309 - samples/sec: 133.49 - lr: 0.300000 +2021-03-26 05:03:13,378 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:03:13,379 EPOCH 27 done: loss 5.2047 - lr 0.3000000 +2021-03-26 05:03:14,128 DEV : loss 6.1164445877075195 - score 0.8931 +2021-03-26 05:03:14,153 BAD EPOCHS (no improvement): 2 +2021-03-26 05:03:14,153 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:03:15,200 epoch 28 - iter 2/25 - loss 4.54348111 - samples/sec: 122.45 - lr: 0.300000 +2021-03-26 05:03:16,213 epoch 28 - iter 4/25 - loss 4.49753290 - samples/sec: 126.69 - lr: 0.300000 +2021-03-26 05:03:17,257 epoch 28 - iter 6/25 - loss 4.49302026 - samples/sec: 122.77 - lr: 0.300000 +2021-03-26 05:03:18,217 epoch 28 - iter 8/25 - loss 4.60087708 - samples/sec: 133.58 - lr: 0.300000 +2021-03-26 05:03:19,189 epoch 28 - iter 10/25 - loss 4.63158011 - samples/sec: 131.77 - lr: 0.300000 +2021-03-26 05:03:20,103 epoch 28 - iter 12/25 - loss 4.91706479 - samples/sec: 140.25 - lr: 0.300000 +2021-03-26 05:03:21,115 epoch 28 - iter 14/25 - loss 5.02842866 - samples/sec: 126.71 - lr: 0.300000 +2021-03-26 05:03:22,361 epoch 28 - iter 16/25 - loss 5.06549254 - samples/sec: 102.89 - lr: 0.300000 +2021-03-26 05:03:23,390 epoch 28 - iter 18/25 - loss 5.10844946 - samples/sec: 124.53 - lr: 0.300000 +2021-03-26 05:03:24,395 epoch 28 - iter 20/25 - loss 5.04149504 - samples/sec: 127.52 - lr: 0.300000 +2021-03-26 05:03:25,527 epoch 28 - iter 22/25 - loss 5.04856537 - samples/sec: 113.23 - lr: 0.300000 +2021-03-26 05:03:26,533 epoch 28 - iter 24/25 - loss 5.06248623 - samples/sec: 127.57 - lr: 0.300000 +2021-03-26 05:03:26,959 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:03:26,960 EPOCH 28 done: loss 5.0528 - lr 0.3000000 +2021-03-26 05:03:27,725 DEV : loss 6.103049278259277 - score 0.8976 +2021-03-26 05:03:27,749 BAD EPOCHS (no improvement): 0 +2021-03-26 05:03:37,303 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:03:38,451 epoch 29 - iter 2/25 - loss 5.05517960 - samples/sec: 111.65 - lr: 0.300000 +2021-03-26 05:03:39,498 epoch 29 - iter 4/25 - loss 4.97563481 - samples/sec: 122.59 - lr: 0.300000 +2021-03-26 05:03:40,585 epoch 29 - iter 6/25 - loss 4.55425203 - samples/sec: 117.81 - lr: 0.300000 +2021-03-26 05:03:41,632 epoch 29 - iter 8/25 - loss 4.85152861 - samples/sec: 123.08 - lr: 0.300000 +2021-03-26 05:03:42,650 epoch 29 - iter 10/25 - loss 4.92433279 - samples/sec: 125.88 - lr: 0.300000 +2021-03-26 05:03:43,563 epoch 29 - iter 12/25 - loss 4.80069878 - samples/sec: 140.71 - lr: 0.300000 +2021-03-26 05:03:44,568 epoch 29 - iter 14/25 - loss 4.84453751 - samples/sec: 127.38 - lr: 0.300000 +2021-03-26 05:03:45,615 epoch 29 - iter 16/25 - loss 4.81823514 - samples/sec: 122.51 - lr: 0.300000 +2021-03-26 05:03:46,566 epoch 29 - iter 18/25 - loss 4.79057773 - samples/sec: 134.84 - lr: 0.300000 +2021-03-26 05:03:47,589 epoch 29 - iter 20/25 - loss 4.94235287 - samples/sec: 125.26 - lr: 0.300000 +2021-03-26 05:03:48,599 epoch 29 - iter 22/25 - loss 5.10795288 - samples/sec: 126.84 - lr: 0.300000 +2021-03-26 05:03:49,627 epoch 29 - iter 24/25 - loss 5.05398512 - samples/sec: 124.76 - lr: 0.300000 +2021-03-26 05:03:50,017 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:03:50,019 EPOCH 29 done: loss 5.0790 - lr 0.3000000 +2021-03-26 05:03:50,770 DEV : loss 6.255495071411133 - score 0.8982 +2021-03-26 05:03:50,796 BAD EPOCHS (no improvement): 0 +2021-03-26 05:04:00,750 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:04:01,870 epoch 30 - iter 2/25 - loss 3.96568513 - samples/sec: 114.60 - lr: 0.300000 +2021-03-26 05:04:02,933 epoch 30 - iter 4/25 - loss 4.38202250 - samples/sec: 120.73 - lr: 0.300000 +2021-03-26 05:04:03,852 epoch 30 - iter 6/25 - loss 4.26282032 - samples/sec: 139.72 - lr: 0.300000 +2021-03-26 05:04:04,888 epoch 30 - iter 8/25 - loss 4.35874718 - samples/sec: 123.69 - lr: 0.300000 +2021-03-26 05:04:05,801 epoch 30 - iter 10/25 - loss 4.36084557 - samples/sec: 140.40 - lr: 0.300000 +2021-03-26 05:04:06,790 epoch 30 - iter 12/25 - loss 4.43777708 - samples/sec: 129.54 - lr: 0.300000 +2021-03-26 05:04:07,807 epoch 30 - iter 14/25 - loss 4.34066623 - samples/sec: 126.10 - lr: 0.300000 +2021-03-26 05:04:08,783 epoch 30 - iter 16/25 - loss 4.34390225 - samples/sec: 131.35 - lr: 0.300000 +2021-03-26 05:04:09,779 epoch 30 - iter 18/25 - loss 4.47116148 - samples/sec: 128.85 - lr: 0.300000 +2021-03-26 05:04:10,815 epoch 30 - iter 20/25 - loss 4.57056917 - samples/sec: 123.76 - lr: 0.300000 +2021-03-26 05:04:11,991 epoch 30 - iter 22/25 - loss 4.66263837 - samples/sec: 109.11 - lr: 0.300000 +2021-03-26 05:04:13,097 epoch 30 - iter 24/25 - loss 4.72658858 - samples/sec: 115.89 - lr: 0.300000 +2021-03-26 05:04:13,528 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:04:13,529 EPOCH 30 done: loss 4.7034 - lr 0.3000000 +2021-03-26 05:04:14,287 DEV : loss 5.959897518157959 - score 0.9017 +2021-03-26 05:04:14,312 BAD EPOCHS (no improvement): 0 +2021-03-26 05:04:23,959 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:04:25,058 epoch 31 - iter 2/25 - loss 4.17308486 - samples/sec: 116.66 - lr: 0.300000 +2021-03-26 05:04:25,998 epoch 31 - iter 4/25 - loss 4.29541880 - samples/sec: 136.45 - lr: 0.300000 +2021-03-26 05:04:27,346 epoch 31 - iter 6/25 - loss 4.24963518 - samples/sec: 95.08 - lr: 0.300000 +2021-03-26 05:04:28,591 epoch 31 - iter 8/25 - loss 4.32354948 - samples/sec: 102.98 - lr: 0.300000 +2021-03-26 05:04:29,657 epoch 31 - iter 10/25 - loss 4.43648040 - samples/sec: 120.20 - lr: 0.300000 +2021-03-26 05:04:30,689 epoch 31 - iter 12/25 - loss 4.37655514 - samples/sec: 124.25 - lr: 0.300000 +2021-03-26 05:04:31,947 epoch 31 - iter 14/25 - loss 4.42733589 - samples/sec: 101.93 - lr: 0.300000 +2021-03-26 05:04:32,934 epoch 31 - iter 16/25 - loss 4.45457639 - samples/sec: 129.87 - lr: 0.300000 +2021-03-26 05:04:33,980 epoch 31 - iter 18/25 - loss 4.51251603 - samples/sec: 122.48 - lr: 0.300000 +2021-03-26 05:04:35,034 epoch 31 - iter 20/25 - loss 4.54872774 - samples/sec: 121.62 - lr: 0.300000 +2021-03-26 05:04:36,119 epoch 31 - iter 22/25 - loss 4.63483950 - samples/sec: 118.36 - lr: 0.300000 +2021-03-26 05:04:37,054 epoch 31 - iter 24/25 - loss 4.68269272 - samples/sec: 137.37 - lr: 0.300000 +2021-03-26 05:04:37,503 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:04:37,504 EPOCH 31 done: loss 4.6874 - lr 0.3000000 +2021-03-26 05:04:38,262 DEV : loss 6.077559471130371 - score 0.9038 +2021-03-26 05:04:38,282 BAD EPOCHS (no improvement): 0 +2021-03-26 05:04:47,950 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:04:48,988 epoch 32 - iter 2/25 - loss 5.09888327 - samples/sec: 123.58 - lr: 0.300000 +2021-03-26 05:04:49,981 epoch 32 - iter 4/25 - loss 5.04619509 - samples/sec: 129.18 - lr: 0.300000 +2021-03-26 05:04:51,035 epoch 32 - iter 6/25 - loss 4.74786079 - samples/sec: 121.64 - lr: 0.300000 +2021-03-26 05:04:52,048 epoch 32 - iter 8/25 - loss 4.76913765 - samples/sec: 126.53 - lr: 0.300000 +2021-03-26 05:04:52,962 epoch 32 - iter 10/25 - loss 4.60072637 - samples/sec: 140.27 - lr: 0.300000 +2021-03-26 05:04:53,981 epoch 32 - iter 12/25 - loss 4.57425706 - samples/sec: 125.71 - lr: 0.300000 +2021-03-26 05:04:54,919 epoch 32 - iter 14/25 - loss 4.58698058 - samples/sec: 136.65 - lr: 0.300000 +2021-03-26 05:04:55,956 epoch 32 - iter 16/25 - loss 4.55232823 - samples/sec: 123.64 - lr: 0.300000 +2021-03-26 05:04:56,900 epoch 32 - iter 18/25 - loss 4.48894657 - samples/sec: 135.83 - lr: 0.300000 +2021-03-26 05:04:58,000 epoch 32 - iter 20/25 - loss 4.55172141 - samples/sec: 116.53 - lr: 0.300000 +2021-03-26 05:04:58,967 epoch 32 - iter 22/25 - loss 4.54604585 - samples/sec: 132.47 - lr: 0.300000 +2021-03-26 05:04:59,905 epoch 32 - iter 24/25 - loss 4.52481222 - samples/sec: 136.66 - lr: 0.300000 +2021-03-26 05:05:00,313 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:05:00,314 EPOCH 32 done: loss 4.4789 - lr 0.3000000 +2021-03-26 05:05:01,059 DEV : loss 6.159491539001465 - score 0.8985 +2021-03-26 05:05:01,078 BAD EPOCHS (no improvement): 1 +2021-03-26 05:05:01,078 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:05:01,999 epoch 33 - iter 2/25 - loss 3.86642694 - samples/sec: 139.32 - lr: 0.300000 +2021-03-26 05:05:03,054 epoch 33 - iter 4/25 - loss 4.42698538 - samples/sec: 121.51 - lr: 0.300000 +2021-03-26 05:05:04,068 epoch 33 - iter 6/25 - loss 4.18415781 - samples/sec: 126.40 - lr: 0.300000 +2021-03-26 05:05:05,177 epoch 33 - iter 8/25 - loss 4.36480805 - samples/sec: 115.51 - lr: 0.300000 +2021-03-26 05:05:06,154 epoch 33 - iter 10/25 - loss 4.28511436 - samples/sec: 131.13 - lr: 0.300000 +2021-03-26 05:05:07,122 epoch 33 - iter 12/25 - loss 4.38803158 - samples/sec: 132.48 - lr: 0.300000 +2021-03-26 05:05:08,060 epoch 33 - iter 14/25 - loss 4.41996484 - samples/sec: 136.69 - lr: 0.300000 +2021-03-26 05:05:09,097 epoch 33 - iter 16/25 - loss 4.32795495 - samples/sec: 123.60 - lr: 0.300000 +2021-03-26 05:05:10,130 epoch 33 - iter 18/25 - loss 4.39254861 - samples/sec: 124.11 - lr: 0.300000 +2021-03-26 05:05:11,213 epoch 33 - iter 20/25 - loss 4.43446009 - samples/sec: 118.29 - lr: 0.300000 +2021-03-26 05:05:12,223 epoch 33 - iter 22/25 - loss 4.50511661 - samples/sec: 126.89 - lr: 0.300000 +2021-03-26 05:05:13,226 epoch 33 - iter 24/25 - loss 4.46563493 - samples/sec: 127.92 - lr: 0.300000 +2021-03-26 05:05:13,599 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:05:13,600 EPOCH 33 done: loss 4.4243 - lr 0.3000000 +2021-03-26 05:05:14,365 DEV : loss 6.339288234710693 - score 0.8939 +2021-03-26 05:05:14,387 BAD EPOCHS (no improvement): 2 +2021-03-26 05:05:14,388 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:05:15,475 epoch 34 - iter 2/25 - loss 3.79593897 - samples/sec: 117.93 - lr: 0.300000 +2021-03-26 05:05:16,628 epoch 34 - iter 4/25 - loss 4.36681736 - samples/sec: 111.14 - lr: 0.300000 +2021-03-26 05:05:17,738 epoch 34 - iter 6/25 - loss 4.18173194 - samples/sec: 115.66 - lr: 0.300000 +2021-03-26 05:05:18,795 epoch 34 - iter 8/25 - loss 4.21293190 - samples/sec: 121.30 - lr: 0.300000 +2021-03-26 05:05:19,875 epoch 34 - iter 10/25 - loss 4.08440013 - samples/sec: 118.62 - lr: 0.300000 +2021-03-26 05:05:20,904 epoch 34 - iter 12/25 - loss 4.21117107 - samples/sec: 124.54 - lr: 0.300000 +2021-03-26 05:05:21,982 epoch 34 - iter 14/25 - loss 4.21686561 - samples/sec: 118.88 - lr: 0.300000 +2021-03-26 05:05:22,980 epoch 34 - iter 16/25 - loss 4.20642594 - samples/sec: 128.56 - lr: 0.300000 +2021-03-26 05:05:24,036 epoch 34 - iter 18/25 - loss 4.21764451 - samples/sec: 121.44 - lr: 0.300000 +2021-03-26 05:05:25,148 epoch 34 - iter 20/25 - loss 4.20252523 - samples/sec: 115.30 - lr: 0.300000 +2021-03-26 05:05:26,171 epoch 34 - iter 22/25 - loss 4.26032110 - samples/sec: 125.37 - lr: 0.300000 +2021-03-26 05:05:27,258 epoch 34 - iter 24/25 - loss 4.23303785 - samples/sec: 117.90 - lr: 0.300000 +2021-03-26 05:05:27,672 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:05:27,673 EPOCH 34 done: loss 4.2135 - lr 0.3000000 +2021-03-26 05:05:28,422 DEV : loss 6.024802207946777 - score 0.9017 +2021-03-26 05:05:28,447 BAD EPOCHS (no improvement): 3 +2021-03-26 05:05:28,448 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:05:29,518 epoch 35 - iter 2/25 - loss 3.52251232 - samples/sec: 119.74 - lr: 0.300000 +2021-03-26 05:05:30,678 epoch 35 - iter 4/25 - loss 3.63564587 - samples/sec: 110.56 - lr: 0.300000 +2021-03-26 05:05:31,841 epoch 35 - iter 6/25 - loss 3.83008993 - samples/sec: 110.14 - lr: 0.300000 +2021-03-26 05:05:33,198 epoch 35 - iter 8/25 - loss 3.89975199 - samples/sec: 94.44 - lr: 0.300000 +2021-03-26 05:05:34,371 epoch 35 - iter 10/25 - loss 3.91242580 - samples/sec: 109.36 - lr: 0.300000 +2021-03-26 05:05:35,294 epoch 35 - iter 12/25 - loss 3.84081417 - samples/sec: 138.87 - lr: 0.300000 +2021-03-26 05:05:36,202 epoch 35 - iter 14/25 - loss 3.87259611 - samples/sec: 141.21 - lr: 0.300000 +2021-03-26 05:05:37,299 epoch 35 - iter 16/25 - loss 3.97355936 - samples/sec: 116.82 - lr: 0.300000 +2021-03-26 05:05:38,579 epoch 35 - iter 18/25 - loss 4.02878156 - samples/sec: 100.12 - lr: 0.300000 +2021-03-26 05:05:39,728 epoch 35 - iter 20/25 - loss 4.05654567 - samples/sec: 111.58 - lr: 0.300000 +2021-03-26 05:05:40,751 epoch 35 - iter 22/25 - loss 4.08147327 - samples/sec: 125.42 - lr: 0.300000 +2021-03-26 05:05:41,753 epoch 35 - iter 24/25 - loss 4.07749189 - samples/sec: 127.96 - lr: 0.300000 +2021-03-26 05:05:42,202 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:05:42,203 EPOCH 35 done: loss 4.0435 - lr 0.3000000 +2021-03-26 05:05:42,955 DEV : loss 6.006406784057617 - score 0.9026 +2021-03-26 05:05:42,976 BAD EPOCHS (no improvement): 4 +2021-03-26 05:05:42,977 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:05:43,943 epoch 36 - iter 2/25 - loss 3.29583621 - samples/sec: 132.70 - lr: 0.150000 +2021-03-26 05:05:45,019 epoch 36 - iter 4/25 - loss 3.62722439 - samples/sec: 119.20 - lr: 0.150000 +2021-03-26 05:05:46,059 epoch 36 - iter 6/25 - loss 3.59074521 - samples/sec: 123.38 - lr: 0.150000 +2021-03-26 05:05:47,147 epoch 36 - iter 8/25 - loss 3.63234273 - samples/sec: 117.75 - lr: 0.150000 +2021-03-26 05:05:48,360 epoch 36 - iter 10/25 - loss 3.72383468 - samples/sec: 105.67 - lr: 0.150000 +2021-03-26 05:05:49,397 epoch 36 - iter 12/25 - loss 3.77632584 - samples/sec: 123.76 - lr: 0.150000 +2021-03-26 05:05:50,348 epoch 36 - iter 14/25 - loss 3.86353767 - samples/sec: 134.94 - lr: 0.150000 +2021-03-26 05:05:51,284 epoch 36 - iter 16/25 - loss 3.91810854 - samples/sec: 137.10 - lr: 0.150000 +2021-03-26 05:05:52,391 epoch 36 - iter 18/25 - loss 3.85365452 - samples/sec: 115.85 - lr: 0.150000 +2021-03-26 05:05:53,380 epoch 36 - iter 20/25 - loss 3.83702444 - samples/sec: 129.71 - lr: 0.150000 +2021-03-26 05:05:54,475 epoch 36 - iter 22/25 - loss 3.82520224 - samples/sec: 117.11 - lr: 0.150000 +2021-03-26 05:05:55,555 epoch 36 - iter 24/25 - loss 3.77670812 - samples/sec: 118.77 - lr: 0.150000 +2021-03-26 05:05:55,938 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:05:55,938 EPOCH 36 done: loss 3.7885 - lr 0.1500000 +2021-03-26 05:05:56,689 DEV : loss 6.025773525238037 - score 0.9013 +2021-03-26 05:05:56,714 BAD EPOCHS (no improvement): 1 +2021-03-26 05:05:56,714 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:05:57,683 epoch 37 - iter 2/25 - loss 3.20568120 - samples/sec: 132.44 - lr: 0.150000 +2021-03-26 05:05:58,701 epoch 37 - iter 4/25 - loss 2.85169643 - samples/sec: 125.95 - lr: 0.150000 +2021-03-26 05:05:59,780 epoch 37 - iter 6/25 - loss 3.11694094 - samples/sec: 118.87 - lr: 0.150000 +2021-03-26 05:06:00,850 epoch 37 - iter 8/25 - loss 3.43193850 - samples/sec: 119.83 - lr: 0.150000 +2021-03-26 05:06:01,955 epoch 37 - iter 10/25 - loss 3.53840156 - samples/sec: 116.04 - lr: 0.150000 +2021-03-26 05:06:02,916 epoch 37 - iter 12/25 - loss 3.46605579 - samples/sec: 133.40 - lr: 0.150000 +2021-03-26 05:06:03,936 epoch 37 - iter 14/25 - loss 3.47069495 - samples/sec: 125.70 - lr: 0.150000 +2021-03-26 05:06:04,923 epoch 37 - iter 16/25 - loss 3.50348461 - samples/sec: 129.89 - lr: 0.150000 +2021-03-26 05:06:05,931 epoch 37 - iter 18/25 - loss 3.45909891 - samples/sec: 127.19 - lr: 0.150000 +2021-03-26 05:06:06,966 epoch 37 - iter 20/25 - loss 3.50957373 - samples/sec: 123.97 - lr: 0.150000 +2021-03-26 05:06:08,015 epoch 37 - iter 22/25 - loss 3.49967953 - samples/sec: 122.23 - lr: 0.150000 +2021-03-26 05:06:09,128 epoch 37 - iter 24/25 - loss 3.52551559 - samples/sec: 115.20 - lr: 0.150000 +2021-03-26 05:06:09,511 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:06:09,511 EPOCH 37 done: loss 3.5291 - lr 0.1500000 +2021-03-26 05:06:10,271 DEV : loss 5.989836692810059 - score 0.9054 +2021-03-26 05:06:10,296 BAD EPOCHS (no improvement): 0 +2021-03-26 05:06:20,120 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:06:21,127 epoch 38 - iter 2/25 - loss 3.95863259 - samples/sec: 127.34 - lr: 0.150000 +2021-03-26 05:06:22,137 epoch 38 - iter 4/25 - loss 3.60193974 - samples/sec: 126.85 - lr: 0.150000 +2021-03-26 05:06:23,140 epoch 38 - iter 6/25 - loss 3.31770647 - samples/sec: 127.85 - lr: 0.150000 +2021-03-26 05:06:24,141 epoch 38 - iter 8/25 - loss 3.34797689 - samples/sec: 128.04 - lr: 0.150000 +2021-03-26 05:06:25,187 epoch 38 - iter 10/25 - loss 3.29749174 - samples/sec: 122.57 - lr: 0.150000 +2021-03-26 05:06:26,342 epoch 38 - iter 12/25 - loss 3.26623889 - samples/sec: 111.07 - lr: 0.150000 +2021-03-26 05:06:27,446 epoch 38 - iter 14/25 - loss 3.33318295 - samples/sec: 116.26 - lr: 0.150000 +2021-03-26 05:06:28,490 epoch 38 - iter 16/25 - loss 3.37746768 - samples/sec: 122.81 - lr: 0.150000 +2021-03-26 05:06:29,449 epoch 38 - iter 18/25 - loss 3.37950455 - samples/sec: 133.70 - lr: 0.150000 +2021-03-26 05:06:30,652 epoch 38 - iter 20/25 - loss 3.36337690 - samples/sec: 106.49 - lr: 0.150000 +2021-03-26 05:06:31,791 epoch 38 - iter 22/25 - loss 3.31866727 - samples/sec: 112.52 - lr: 0.150000 +2021-03-26 05:06:32,804 epoch 38 - iter 24/25 - loss 3.36319639 - samples/sec: 126.51 - lr: 0.150000 +2021-03-26 05:06:33,246 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:06:33,248 EPOCH 38 done: loss 3.3952 - lr 0.1500000 +2021-03-26 05:06:34,037 DEV : loss 5.92034912109375 - score 0.9099 +2021-03-26 05:06:34,057 BAD EPOCHS (no improvement): 0 +2021-03-26 05:06:43,932 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:06:44,984 epoch 39 - iter 2/25 - loss 3.76524377 - samples/sec: 122.01 - lr: 0.150000 +2021-03-26 05:06:46,028 epoch 39 - iter 4/25 - loss 3.60626566 - samples/sec: 122.73 - lr: 0.150000 +2021-03-26 05:06:47,252 epoch 39 - iter 6/25 - loss 3.51154840 - samples/sec: 104.70 - lr: 0.150000 +2021-03-26 05:06:48,586 epoch 39 - iter 8/25 - loss 3.43775126 - samples/sec: 96.04 - lr: 0.150000 +2021-03-26 05:06:49,906 epoch 39 - iter 10/25 - loss 3.43897688 - samples/sec: 97.05 - lr: 0.150000 +2021-03-26 05:06:51,086 epoch 39 - iter 12/25 - loss 3.35562142 - samples/sec: 108.60 - lr: 0.150000 +2021-03-26 05:06:52,566 epoch 39 - iter 14/25 - loss 3.44103054 - samples/sec: 86.66 - lr: 0.150000 +2021-03-26 05:06:53,504 epoch 39 - iter 16/25 - loss 3.37010135 - samples/sec: 136.61 - lr: 0.150000 +2021-03-26 05:06:54,547 epoch 39 - iter 18/25 - loss 3.43329452 - samples/sec: 122.89 - lr: 0.150000 +2021-03-26 05:06:55,544 epoch 39 - iter 20/25 - loss 3.41735572 - samples/sec: 128.64 - lr: 0.150000 +2021-03-26 05:06:56,626 epoch 39 - iter 22/25 - loss 3.38750473 - samples/sec: 118.54 - lr: 0.150000 +2021-03-26 05:06:57,596 epoch 39 - iter 24/25 - loss 3.35841246 - samples/sec: 132.05 - lr: 0.150000 +2021-03-26 05:06:58,018 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:06:58,018 EPOCH 39 done: loss 3.3819 - lr 0.1500000 +2021-03-26 05:06:58,842 DEV : loss 5.770626068115234 - score 0.9104 +2021-03-26 05:06:58,865 BAD EPOCHS (no improvement): 0 +2021-03-26 05:07:08,812 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:07:09,950 epoch 40 - iter 2/25 - loss 3.43545771 - samples/sec: 112.83 - lr: 0.150000 +2021-03-26 05:07:11,157 epoch 40 - iter 4/25 - loss 3.34994274 - samples/sec: 106.27 - lr: 0.150000 +2021-03-26 05:07:12,161 epoch 40 - iter 6/25 - loss 3.34000186 - samples/sec: 127.67 - lr: 0.150000 +2021-03-26 05:07:13,157 epoch 40 - iter 8/25 - loss 3.21268988 - samples/sec: 128.82 - lr: 0.150000 +2021-03-26 05:07:14,151 epoch 40 - iter 10/25 - loss 3.27631202 - samples/sec: 128.79 - lr: 0.150000 +2021-03-26 05:07:15,085 epoch 40 - iter 12/25 - loss 3.40312175 - samples/sec: 137.31 - lr: 0.150000 +2021-03-26 05:07:16,061 epoch 40 - iter 14/25 - loss 3.38169825 - samples/sec: 131.38 - lr: 0.150000 +2021-03-26 05:07:17,056 epoch 40 - iter 16/25 - loss 3.36957914 - samples/sec: 128.78 - lr: 0.150000 +2021-03-26 05:07:18,091 epoch 40 - iter 18/25 - loss 3.34364626 - samples/sec: 123.75 - lr: 0.150000 +2021-03-26 05:07:19,111 epoch 40 - iter 20/25 - loss 3.28600924 - samples/sec: 125.76 - lr: 0.150000 +2021-03-26 05:07:20,175 epoch 40 - iter 22/25 - loss 3.27949734 - samples/sec: 120.46 - lr: 0.150000 +2021-03-26 05:07:21,140 epoch 40 - iter 24/25 - loss 3.24670504 - samples/sec: 133.07 - lr: 0.150000 +2021-03-26 05:07:21,528 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:07:21,529 EPOCH 40 done: loss 3.2736 - lr 0.1500000 +2021-03-26 05:07:22,266 DEV : loss 5.83436393737793 - score 0.9116 +2021-03-26 05:07:22,289 BAD EPOCHS (no improvement): 0 +2021-03-26 05:07:31,972 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:07:33,047 epoch 41 - iter 2/25 - loss 3.27510595 - samples/sec: 119.23 - lr: 0.150000 +2021-03-26 05:07:34,066 epoch 41 - iter 4/25 - loss 3.29064929 - samples/sec: 125.80 - lr: 0.150000 +2021-03-26 05:07:35,004 epoch 41 - iter 6/25 - loss 3.27098417 - samples/sec: 136.91 - lr: 0.150000 +2021-03-26 05:07:36,059 epoch 41 - iter 8/25 - loss 3.24700919 - samples/sec: 121.42 - lr: 0.150000 +2021-03-26 05:07:37,197 epoch 41 - iter 10/25 - loss 3.20525589 - samples/sec: 112.81 - lr: 0.150000 +2021-03-26 05:07:38,498 epoch 41 - iter 12/25 - loss 3.20550714 - samples/sec: 98.53 - lr: 0.150000 +2021-03-26 05:07:39,991 epoch 41 - iter 14/25 - loss 3.20661204 - samples/sec: 85.82 - lr: 0.150000 +2021-03-26 05:07:41,300 epoch 41 - iter 16/25 - loss 3.22897214 - samples/sec: 97.93 - lr: 0.150000 +2021-03-26 05:07:42,835 epoch 41 - iter 18/25 - loss 3.26956442 - samples/sec: 83.52 - lr: 0.150000 +2021-03-26 05:07:44,192 epoch 41 - iter 20/25 - loss 3.26194243 - samples/sec: 94.54 - lr: 0.150000 +2021-03-26 05:07:45,789 epoch 41 - iter 22/25 - loss 3.24391865 - samples/sec: 80.23 - lr: 0.150000 +2021-03-26 05:07:46,808 epoch 41 - iter 24/25 - loss 3.29610110 - samples/sec: 125.85 - lr: 0.150000 +2021-03-26 05:07:47,187 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:07:47,188 EPOCH 41 done: loss 3.3268 - lr 0.1500000 +2021-03-26 05:07:48,069 DEV : loss 5.8652191162109375 - score 0.9042 +2021-03-26 05:07:48,104 BAD EPOCHS (no improvement): 1 +2021-03-26 05:07:48,104 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:07:49,110 epoch 42 - iter 2/25 - loss 2.96759343 - samples/sec: 127.58 - lr: 0.150000 +2021-03-26 05:07:50,155 epoch 42 - iter 4/25 - loss 2.98074931 - samples/sec: 122.72 - lr: 0.150000 +2021-03-26 05:07:51,080 epoch 42 - iter 6/25 - loss 3.09812117 - samples/sec: 138.57 - lr: 0.150000 +2021-03-26 05:07:51,987 epoch 42 - iter 8/25 - loss 3.03092873 - samples/sec: 141.37 - lr: 0.150000 +2021-03-26 05:07:52,996 epoch 42 - iter 10/25 - loss 3.01937761 - samples/sec: 127.09 - lr: 0.150000 +2021-03-26 05:07:53,907 epoch 42 - iter 12/25 - loss 3.12206964 - samples/sec: 140.65 - lr: 0.150000 +2021-03-26 05:07:54,944 epoch 42 - iter 14/25 - loss 3.05444547 - samples/sec: 123.65 - lr: 0.150000 +2021-03-26 05:07:56,077 epoch 42 - iter 16/25 - loss 3.09004906 - samples/sec: 113.25 - lr: 0.150000 +2021-03-26 05:07:57,070 epoch 42 - iter 18/25 - loss 3.17406662 - samples/sec: 129.02 - lr: 0.150000 +2021-03-26 05:07:58,145 epoch 42 - iter 20/25 - loss 3.12834384 - samples/sec: 119.22 - lr: 0.150000 +2021-03-26 05:07:59,282 epoch 42 - iter 22/25 - loss 3.16719803 - samples/sec: 112.72 - lr: 0.150000 +2021-03-26 05:08:00,216 epoch 42 - iter 24/25 - loss 3.17727866 - samples/sec: 137.52 - lr: 0.150000 +2021-03-26 05:08:00,642 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:08:00,643 EPOCH 42 done: loss 3.2355 - lr 0.1500000 +2021-03-26 05:08:01,393 DEV : loss 6.138169765472412 - score 0.9087 +2021-03-26 05:08:01,418 BAD EPOCHS (no improvement): 2 +2021-03-26 05:08:01,419 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:08:02,314 epoch 43 - iter 2/25 - loss 2.71440363 - samples/sec: 143.30 - lr: 0.150000 +2021-03-26 05:08:03,386 epoch 43 - iter 4/25 - loss 2.60436940 - samples/sec: 119.47 - lr: 0.150000 +2021-03-26 05:08:04,515 epoch 43 - iter 6/25 - loss 2.90601961 - samples/sec: 113.71 - lr: 0.150000 +2021-03-26 05:08:05,603 epoch 43 - iter 8/25 - loss 2.87946507 - samples/sec: 117.88 - lr: 0.150000 +2021-03-26 05:08:06,562 epoch 43 - iter 10/25 - loss 2.92280562 - samples/sec: 133.72 - lr: 0.150000 +2021-03-26 05:08:07,605 epoch 43 - iter 12/25 - loss 3.02853823 - samples/sec: 122.97 - lr: 0.150000 +2021-03-26 05:08:08,542 epoch 43 - iter 14/25 - loss 3.05405590 - samples/sec: 136.82 - lr: 0.150000 +2021-03-26 05:08:09,481 epoch 43 - iter 16/25 - loss 3.07377841 - samples/sec: 136.51 - lr: 0.150000 +2021-03-26 05:08:10,552 epoch 43 - iter 18/25 - loss 3.16452719 - samples/sec: 119.71 - lr: 0.150000 +2021-03-26 05:08:11,590 epoch 43 - iter 20/25 - loss 3.12460047 - samples/sec: 123.47 - lr: 0.150000 +2021-03-26 05:08:12,565 epoch 43 - iter 22/25 - loss 3.13019865 - samples/sec: 131.42 - lr: 0.150000 +2021-03-26 05:08:13,455 epoch 43 - iter 24/25 - loss 3.10660581 - samples/sec: 144.13 - lr: 0.150000 +2021-03-26 05:08:13,917 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:08:13,918 EPOCH 43 done: loss 3.1132 - lr 0.1500000 +2021-03-26 05:08:14,652 DEV : loss 5.965920925140381 - score 0.905 +2021-03-26 05:08:14,669 BAD EPOCHS (no improvement): 3 +2021-03-26 05:08:14,669 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:08:15,501 epoch 44 - iter 2/25 - loss 2.74227560 - samples/sec: 154.12 - lr: 0.150000 +2021-03-26 05:08:16,485 epoch 44 - iter 4/25 - loss 2.89587241 - samples/sec: 130.33 - lr: 0.150000 +2021-03-26 05:08:17,504 epoch 44 - iter 6/25 - loss 2.87528515 - samples/sec: 125.87 - lr: 0.150000 +2021-03-26 05:08:18,483 epoch 44 - iter 8/25 - loss 2.86914650 - samples/sec: 130.94 - lr: 0.150000 +2021-03-26 05:08:19,471 epoch 44 - iter 10/25 - loss 2.98499634 - samples/sec: 129.74 - lr: 0.150000 +2021-03-26 05:08:20,410 epoch 44 - iter 12/25 - loss 3.04275517 - samples/sec: 136.50 - lr: 0.150000 +2021-03-26 05:08:21,499 epoch 44 - iter 14/25 - loss 3.04654218 - samples/sec: 117.85 - lr: 0.150000 +2021-03-26 05:08:22,529 epoch 44 - iter 16/25 - loss 3.04176466 - samples/sec: 124.36 - lr: 0.150000 +2021-03-26 05:08:23,525 epoch 44 - iter 18/25 - loss 3.02761053 - samples/sec: 128.73 - lr: 0.150000 +2021-03-26 05:08:24,552 epoch 44 - iter 20/25 - loss 3.06731745 - samples/sec: 124.84 - lr: 0.150000 +2021-03-26 05:08:25,503 epoch 44 - iter 22/25 - loss 3.11688727 - samples/sec: 134.78 - lr: 0.150000 +2021-03-26 05:08:26,522 epoch 44 - iter 24/25 - loss 3.14101426 - samples/sec: 125.77 - lr: 0.150000 +2021-03-26 05:08:26,918 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:08:26,919 EPOCH 44 done: loss 3.1204 - lr 0.1500000 +2021-03-26 05:08:27,670 DEV : loss 5.890631675720215 - score 0.9104 +2021-03-26 05:08:27,695 BAD EPOCHS (no improvement): 4 +2021-03-26 05:08:27,695 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:08:28,747 epoch 45 - iter 2/25 - loss 2.88748765 - samples/sec: 122.00 - lr: 0.075000 +2021-03-26 05:08:29,758 epoch 45 - iter 4/25 - loss 3.12572914 - samples/sec: 126.94 - lr: 0.075000 +2021-03-26 05:08:31,051 epoch 45 - iter 6/25 - loss 3.04913143 - samples/sec: 99.11 - lr: 0.075000 +2021-03-26 05:08:32,318 epoch 45 - iter 8/25 - loss 3.06582740 - samples/sec: 101.19 - lr: 0.075000 +2021-03-26 05:08:33,733 epoch 45 - iter 10/25 - loss 3.13125415 - samples/sec: 90.52 - lr: 0.075000 +2021-03-26 05:08:35,027 epoch 45 - iter 12/25 - loss 3.13420904 - samples/sec: 99.12 - lr: 0.075000 +2021-03-26 05:08:35,963 epoch 45 - iter 14/25 - loss 3.15804754 - samples/sec: 136.99 - lr: 0.075000 +2021-03-26 05:08:36,924 epoch 45 - iter 16/25 - loss 3.18488774 - samples/sec: 133.34 - lr: 0.075000 +2021-03-26 05:08:37,992 epoch 45 - iter 18/25 - loss 3.21598185 - samples/sec: 120.19 - lr: 0.075000 +2021-03-26 05:08:38,961 epoch 45 - iter 20/25 - loss 3.21181444 - samples/sec: 132.21 - lr: 0.075000 +2021-03-26 05:08:40,006 epoch 45 - iter 22/25 - loss 3.19924637 - samples/sec: 122.70 - lr: 0.075000 +2021-03-26 05:08:41,079 epoch 45 - iter 24/25 - loss 3.12326492 - samples/sec: 119.41 - lr: 0.075000 +2021-03-26 05:08:41,467 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:08:41,468 EPOCH 45 done: loss 3.1075 - lr 0.0750000 +2021-03-26 05:08:42,198 DEV : loss 5.875544548034668 - score 0.912 +2021-03-26 05:08:42,222 BAD EPOCHS (no improvement): 0 +2021-03-26 05:08:51,984 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:08:53,045 epoch 46 - iter 2/25 - loss 2.93449044 - samples/sec: 120.89 - lr: 0.075000 +2021-03-26 05:08:54,027 epoch 46 - iter 4/25 - loss 2.99039429 - samples/sec: 130.48 - lr: 0.075000 +2021-03-26 05:08:55,046 epoch 46 - iter 6/25 - loss 2.93938939 - samples/sec: 125.82 - lr: 0.075000 +2021-03-26 05:08:56,105 epoch 46 - iter 8/25 - loss 2.99548993 - samples/sec: 120.95 - lr: 0.075000 +2021-03-26 05:08:57,441 epoch 46 - iter 10/25 - loss 2.97969875 - samples/sec: 95.96 - lr: 0.075000 +2021-03-26 05:08:58,762 epoch 46 - iter 12/25 - loss 2.97719487 - samples/sec: 96.97 - lr: 0.075000 +2021-03-26 05:09:00,173 epoch 46 - iter 14/25 - loss 2.96926909 - samples/sec: 90.79 - lr: 0.075000 +2021-03-26 05:09:01,175 epoch 46 - iter 16/25 - loss 2.86651133 - samples/sec: 127.93 - lr: 0.075000 +2021-03-26 05:09:02,147 epoch 46 - iter 18/25 - loss 2.83233232 - samples/sec: 131.95 - lr: 0.075000 +2021-03-26 05:09:03,273 epoch 46 - iter 20/25 - loss 2.86771994 - samples/sec: 113.75 - lr: 0.075000 +2021-03-26 05:09:04,273 epoch 46 - iter 22/25 - loss 2.93586094 - samples/sec: 128.26 - lr: 0.075000 +2021-03-26 05:09:05,280 epoch 46 - iter 24/25 - loss 2.91423157 - samples/sec: 127.24 - lr: 0.075000 +2021-03-26 05:09:05,674 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:09:05,674 EPOCH 46 done: loss 2.9104 - lr 0.0750000 +2021-03-26 05:09:06,420 DEV : loss 5.901866436004639 - score 0.9128 +2021-03-26 05:09:06,444 BAD EPOCHS (no improvement): 0 +2021-03-26 05:09:16,208 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:09:17,307 epoch 47 - iter 2/25 - loss 2.89936328 - samples/sec: 116.74 - lr: 0.075000 +2021-03-26 05:09:18,420 epoch 47 - iter 4/25 - loss 3.05432254 - samples/sec: 115.20 - lr: 0.075000 +2021-03-26 05:09:19,609 epoch 47 - iter 6/25 - loss 3.02896949 - samples/sec: 107.74 - lr: 0.075000 +2021-03-26 05:09:20,605 epoch 47 - iter 8/25 - loss 2.99035889 - samples/sec: 128.88 - lr: 0.075000 +2021-03-26 05:09:21,620 epoch 47 - iter 10/25 - loss 2.95006714 - samples/sec: 126.38 - lr: 0.075000 +2021-03-26 05:09:22,681 epoch 47 - iter 12/25 - loss 2.90737903 - samples/sec: 120.81 - lr: 0.075000 +2021-03-26 05:09:23,646 epoch 47 - iter 14/25 - loss 2.91904158 - samples/sec: 133.04 - lr: 0.075000 +2021-03-26 05:09:24,556 epoch 47 - iter 16/25 - loss 2.85798267 - samples/sec: 140.88 - lr: 0.075000 +2021-03-26 05:09:25,453 epoch 47 - iter 18/25 - loss 2.82409436 - samples/sec: 142.76 - lr: 0.075000 +2021-03-26 05:09:26,450 epoch 47 - iter 20/25 - loss 2.83086318 - samples/sec: 128.60 - lr: 0.075000 +2021-03-26 05:09:27,466 epoch 47 - iter 22/25 - loss 2.79424383 - samples/sec: 126.07 - lr: 0.075000 +2021-03-26 05:09:28,477 epoch 47 - iter 24/25 - loss 2.80670158 - samples/sec: 126.97 - lr: 0.075000 +2021-03-26 05:09:29,006 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:09:29,008 EPOCH 47 done: loss 2.8199 - lr 0.0750000 +2021-03-26 05:09:29,759 DEV : loss 5.768811225891113 - score 0.9124 +2021-03-26 05:09:29,783 BAD EPOCHS (no improvement): 1 +2021-03-26 05:09:29,784 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:09:30,841 epoch 48 - iter 2/25 - loss 2.51504290 - samples/sec: 121.26 - lr: 0.075000 +2021-03-26 05:09:31,922 epoch 48 - iter 4/25 - loss 2.33881265 - samples/sec: 118.72 - lr: 0.075000 +2021-03-26 05:09:32,962 epoch 48 - iter 6/25 - loss 2.54652301 - samples/sec: 123.26 - lr: 0.075000 +2021-03-26 05:09:34,069 epoch 48 - iter 8/25 - loss 2.60233888 - samples/sec: 115.85 - lr: 0.075000 +2021-03-26 05:09:35,007 epoch 48 - iter 10/25 - loss 2.64935668 - samples/sec: 136.62 - lr: 0.075000 +2021-03-26 05:09:35,985 epoch 48 - iter 12/25 - loss 2.67004373 - samples/sec: 131.18 - lr: 0.075000 +2021-03-26 05:09:37,010 epoch 48 - iter 14/25 - loss 2.76849834 - samples/sec: 125.05 - lr: 0.075000 +2021-03-26 05:09:38,014 epoch 48 - iter 16/25 - loss 2.82034701 - samples/sec: 127.64 - lr: 0.075000 +2021-03-26 05:09:38,976 epoch 48 - iter 18/25 - loss 2.83146665 - samples/sec: 133.26 - lr: 0.075000 +2021-03-26 05:09:39,936 epoch 48 - iter 20/25 - loss 2.83332582 - samples/sec: 133.66 - lr: 0.075000 +2021-03-26 05:09:40,992 epoch 48 - iter 22/25 - loss 2.85925512 - samples/sec: 121.42 - lr: 0.075000 +2021-03-26 05:09:42,018 epoch 48 - iter 24/25 - loss 2.82686347 - samples/sec: 125.17 - lr: 0.075000 +2021-03-26 05:09:42,375 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:09:42,377 EPOCH 48 done: loss 2.8162 - lr 0.0750000 +2021-03-26 05:09:43,143 DEV : loss 5.849610328674316 - score 0.9165 +2021-03-26 05:09:43,161 BAD EPOCHS (no improvement): 0 +2021-03-26 05:09:53,064 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:09:54,097 epoch 49 - iter 2/25 - loss 3.24604166 - samples/sec: 124.25 - lr: 0.075000 +2021-03-26 05:09:55,183 epoch 49 - iter 4/25 - loss 3.21493500 - samples/sec: 118.06 - lr: 0.075000 +2021-03-26 05:09:56,179 epoch 49 - iter 6/25 - loss 2.91566018 - samples/sec: 128.78 - lr: 0.075000 +2021-03-26 05:09:57,183 epoch 49 - iter 8/25 - loss 2.90339556 - samples/sec: 127.74 - lr: 0.075000 +2021-03-26 05:09:58,302 epoch 49 - iter 10/25 - loss 2.77145948 - samples/sec: 114.58 - lr: 0.075000 +2021-03-26 05:09:59,481 epoch 49 - iter 12/25 - loss 2.80427039 - samples/sec: 108.89 - lr: 0.075000 +2021-03-26 05:10:00,481 epoch 49 - iter 14/25 - loss 2.79233439 - samples/sec: 128.23 - lr: 0.075000 +2021-03-26 05:10:01,534 epoch 49 - iter 16/25 - loss 2.72604090 - samples/sec: 121.80 - lr: 0.075000 +2021-03-26 05:10:02,728 epoch 49 - iter 18/25 - loss 2.72750786 - samples/sec: 107.31 - lr: 0.075000 +2021-03-26 05:10:03,735 epoch 49 - iter 20/25 - loss 2.73197596 - samples/sec: 127.33 - lr: 0.075000 +2021-03-26 05:10:04,679 epoch 49 - iter 22/25 - loss 2.74085398 - samples/sec: 135.80 - lr: 0.075000 +2021-03-26 05:10:05,680 epoch 49 - iter 24/25 - loss 2.73969566 - samples/sec: 128.00 - lr: 0.075000 +2021-03-26 05:10:06,075 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:10:06,075 EPOCH 49 done: loss 2.7766 - lr 0.0750000 +2021-03-26 05:10:06,816 DEV : loss 5.883216857910156 - score 0.9157 +2021-03-26 05:10:06,840 BAD EPOCHS (no improvement): 1 +2021-03-26 05:10:06,841 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:10:07,848 epoch 50 - iter 2/25 - loss 2.59146047 - samples/sec: 127.38 - lr: 0.075000 +2021-03-26 05:10:08,849 epoch 50 - iter 4/25 - loss 2.85189342 - samples/sec: 128.01 - lr: 0.075000 +2021-03-26 05:10:09,838 epoch 50 - iter 6/25 - loss 2.85971487 - samples/sec: 129.57 - lr: 0.075000 +2021-03-26 05:10:10,796 epoch 50 - iter 8/25 - loss 2.81490350 - samples/sec: 133.84 - lr: 0.075000 +2021-03-26 05:10:11,810 epoch 50 - iter 10/25 - loss 2.72035291 - samples/sec: 126.40 - lr: 0.075000 +2021-03-26 05:10:12,878 epoch 50 - iter 12/25 - loss 2.77295995 - samples/sec: 120.02 - lr: 0.075000 +2021-03-26 05:10:13,955 epoch 50 - iter 14/25 - loss 2.76417232 - samples/sec: 119.49 - lr: 0.075000 +2021-03-26 05:10:14,902 epoch 50 - iter 16/25 - loss 2.77499604 - samples/sec: 135.33 - lr: 0.075000 +2021-03-26 05:10:15,913 epoch 50 - iter 18/25 - loss 2.73042422 - samples/sec: 126.79 - lr: 0.075000 +2021-03-26 05:10:16,936 epoch 50 - iter 20/25 - loss 2.72530985 - samples/sec: 125.36 - lr: 0.075000 +2021-03-26 05:10:18,015 epoch 50 - iter 22/25 - loss 2.68665273 - samples/sec: 118.91 - lr: 0.075000 +2021-03-26 05:10:19,058 epoch 50 - iter 24/25 - loss 2.68635307 - samples/sec: 122.84 - lr: 0.075000 +2021-03-26 05:10:19,451 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:10:19,452 EPOCH 50 done: loss 2.6908 - lr 0.0750000 +2021-03-26 05:10:20,196 DEV : loss 5.870783805847168 - score 0.9165 +2021-03-26 05:10:20,222 BAD EPOCHS (no improvement): 2 +2021-03-26 05:10:20,223 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:10:21,245 epoch 51 - iter 2/25 - loss 3.00540733 - samples/sec: 125.46 - lr: 0.075000 +2021-03-26 05:10:22,211 epoch 51 - iter 4/25 - loss 2.98089063 - samples/sec: 132.85 - lr: 0.075000 +2021-03-26 05:10:23,317 epoch 51 - iter 6/25 - loss 2.87133745 - samples/sec: 115.83 - lr: 0.075000 +2021-03-26 05:10:24,245 epoch 51 - iter 8/25 - loss 2.79685122 - samples/sec: 138.23 - lr: 0.075000 +2021-03-26 05:10:25,314 epoch 51 - iter 10/25 - loss 2.83986566 - samples/sec: 119.88 - lr: 0.075000 +2021-03-26 05:10:26,240 epoch 51 - iter 12/25 - loss 2.70246488 - samples/sec: 138.42 - lr: 0.075000 +2021-03-26 05:10:27,172 epoch 51 - iter 14/25 - loss 2.68542244 - samples/sec: 137.51 - lr: 0.075000 +2021-03-26 05:10:28,293 epoch 51 - iter 16/25 - loss 2.66649586 - samples/sec: 114.35 - lr: 0.075000 +2021-03-26 05:10:29,616 epoch 51 - iter 18/25 - loss 2.69753993 - samples/sec: 96.85 - lr: 0.075000 +2021-03-26 05:10:30,640 epoch 51 - iter 20/25 - loss 2.73083348 - samples/sec: 125.16 - lr: 0.075000 +2021-03-26 05:10:31,801 epoch 51 - iter 22/25 - loss 2.74151724 - samples/sec: 110.31 - lr: 0.075000 +2021-03-26 05:10:32,894 epoch 51 - iter 24/25 - loss 2.78909466 - samples/sec: 117.31 - lr: 0.075000 +2021-03-26 05:10:33,297 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:10:33,298 EPOCH 51 done: loss 2.7559 - lr 0.0750000 +2021-03-26 05:10:34,050 DEV : loss 5.804508209228516 - score 0.9157 +2021-03-26 05:10:34,068 BAD EPOCHS (no improvement): 3 +2021-03-26 05:10:34,068 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:10:35,051 epoch 52 - iter 2/25 - loss 2.59395206 - samples/sec: 130.52 - lr: 0.075000 +2021-03-26 05:10:36,047 epoch 52 - iter 4/25 - loss 2.64464074 - samples/sec: 128.60 - lr: 0.075000 +2021-03-26 05:10:36,977 epoch 52 - iter 6/25 - loss 2.63424472 - samples/sec: 137.92 - lr: 0.075000 +2021-03-26 05:10:38,067 epoch 52 - iter 8/25 - loss 2.66879421 - samples/sec: 117.59 - lr: 0.075000 +2021-03-26 05:10:39,227 epoch 52 - iter 10/25 - loss 2.63585150 - samples/sec: 110.48 - lr: 0.075000 +2021-03-26 05:10:40,216 epoch 52 - iter 12/25 - loss 2.56680995 - samples/sec: 129.67 - lr: 0.075000 +2021-03-26 05:10:41,237 epoch 52 - iter 14/25 - loss 2.54881712 - samples/sec: 125.51 - lr: 0.075000 +2021-03-26 05:10:42,260 epoch 52 - iter 16/25 - loss 2.60447754 - samples/sec: 125.39 - lr: 0.075000 +2021-03-26 05:10:43,231 epoch 52 - iter 18/25 - loss 2.55686861 - samples/sec: 131.86 - lr: 0.075000 +2021-03-26 05:10:44,205 epoch 52 - iter 20/25 - loss 2.60568520 - samples/sec: 131.73 - lr: 0.075000 +2021-03-26 05:10:45,252 epoch 52 - iter 22/25 - loss 2.67642180 - samples/sec: 122.38 - lr: 0.075000 +2021-03-26 05:10:46,209 epoch 52 - iter 24/25 - loss 2.69854917 - samples/sec: 133.98 - lr: 0.075000 +2021-03-26 05:10:46,588 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:10:46,589 EPOCH 52 done: loss 2.6946 - lr 0.0750000 +2021-03-26 05:10:47,361 DEV : loss 5.859862804412842 - score 0.9128 +2021-03-26 05:10:47,389 BAD EPOCHS (no improvement): 4 +2021-03-26 05:10:47,390 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:10:48,466 epoch 53 - iter 2/25 - loss 2.85855150 - samples/sec: 119.12 - lr: 0.037500 +2021-03-26 05:10:49,499 epoch 53 - iter 4/25 - loss 2.82207429 - samples/sec: 124.04 - lr: 0.037500 +2021-03-26 05:10:50,589 epoch 53 - iter 6/25 - loss 2.66309059 - samples/sec: 117.62 - lr: 0.037500 +2021-03-26 05:10:51,582 epoch 53 - iter 8/25 - loss 2.65608153 - samples/sec: 129.17 - lr: 0.037500 +2021-03-26 05:10:52,583 epoch 53 - iter 10/25 - loss 2.80065906 - samples/sec: 128.28 - lr: 0.037500 +2021-03-26 05:10:53,621 epoch 53 - iter 12/25 - loss 2.76256857 - samples/sec: 123.54 - lr: 0.037500 +2021-03-26 05:10:54,591 epoch 53 - iter 14/25 - loss 2.71554373 - samples/sec: 132.25 - lr: 0.037500 +2021-03-26 05:10:55,563 epoch 53 - iter 16/25 - loss 2.73098251 - samples/sec: 131.95 - lr: 0.037500 +2021-03-26 05:10:56,763 epoch 53 - iter 18/25 - loss 2.69363379 - samples/sec: 106.77 - lr: 0.037500 +2021-03-26 05:10:57,722 epoch 53 - iter 20/25 - loss 2.69553999 - samples/sec: 133.64 - lr: 0.037500 +2021-03-26 05:10:58,774 epoch 53 - iter 22/25 - loss 2.71568731 - samples/sec: 121.87 - lr: 0.037500 +2021-03-26 05:10:59,737 epoch 53 - iter 24/25 - loss 2.75015450 - samples/sec: 133.29 - lr: 0.037500 +2021-03-26 05:11:00,161 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:11:00,162 EPOCH 53 done: loss 2.7943 - lr 0.0375000 +2021-03-26 05:11:00,943 DEV : loss 5.84169864654541 - score 0.9153 +2021-03-26 05:11:00,962 BAD EPOCHS (no improvement): 1 +2021-03-26 05:11:00,963 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:11:01,985 epoch 54 - iter 2/25 - loss 2.60842669 - samples/sec: 125.45 - lr: 0.037500 +2021-03-26 05:11:03,055 epoch 54 - iter 4/25 - loss 2.65413553 - samples/sec: 119.73 - lr: 0.037500 +2021-03-26 05:11:03,993 epoch 54 - iter 6/25 - loss 2.61845867 - samples/sec: 136.63 - lr: 0.037500 +2021-03-26 05:11:05,132 epoch 54 - iter 8/25 - loss 2.58603658 - samples/sec: 112.68 - lr: 0.037500 +2021-03-26 05:11:06,201 epoch 54 - iter 10/25 - loss 2.65403033 - samples/sec: 119.94 - lr: 0.037500 +2021-03-26 05:11:07,283 epoch 54 - iter 12/25 - loss 2.73958167 - samples/sec: 118.39 - lr: 0.037500 +2021-03-26 05:11:08,408 epoch 54 - iter 14/25 - loss 2.76906408 - samples/sec: 114.00 - lr: 0.037500 +2021-03-26 05:11:09,636 epoch 54 - iter 16/25 - loss 2.74741621 - samples/sec: 104.33 - lr: 0.037500 +2021-03-26 05:11:10,794 epoch 54 - iter 18/25 - loss 2.73005085 - samples/sec: 110.87 - lr: 0.037500 +2021-03-26 05:11:11,738 epoch 54 - iter 20/25 - loss 2.76427827 - samples/sec: 135.69 - lr: 0.037500 +2021-03-26 05:11:12,781 epoch 54 - iter 22/25 - loss 2.76628489 - samples/sec: 122.98 - lr: 0.037500 +2021-03-26 05:11:13,868 epoch 54 - iter 24/25 - loss 2.78265012 - samples/sec: 117.86 - lr: 0.037500 +2021-03-26 05:11:14,243 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:11:14,244 EPOCH 54 done: loss 2.7432 - lr 0.0375000 +2021-03-26 05:11:15,075 DEV : loss 5.859203338623047 - score 0.9132 +2021-03-26 05:11:15,103 BAD EPOCHS (no improvement): 2 +2021-03-26 05:11:15,104 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:11:16,164 epoch 55 - iter 2/25 - loss 2.60760987 - samples/sec: 120.93 - lr: 0.037500 +2021-03-26 05:11:17,322 epoch 55 - iter 4/25 - loss 2.61803091 - samples/sec: 110.63 - lr: 0.037500 +2021-03-26 05:11:18,328 epoch 55 - iter 6/25 - loss 2.67942242 - samples/sec: 127.42 - lr: 0.037500 +2021-03-26 05:11:19,411 epoch 55 - iter 8/25 - loss 2.65748721 - samples/sec: 118.38 - lr: 0.037500 +2021-03-26 05:11:20,659 epoch 55 - iter 10/25 - loss 2.59346483 - samples/sec: 102.72 - lr: 0.037500 +2021-03-26 05:11:21,786 epoch 55 - iter 12/25 - loss 2.66532530 - samples/sec: 113.73 - lr: 0.037500 +2021-03-26 05:11:22,685 epoch 55 - iter 14/25 - loss 2.65365284 - samples/sec: 143.41 - lr: 0.037500 +2021-03-26 05:11:23,646 epoch 55 - iter 16/25 - loss 2.70386864 - samples/sec: 133.36 - lr: 0.037500 +2021-03-26 05:11:24,700 epoch 55 - iter 18/25 - loss 2.70395010 - samples/sec: 121.62 - lr: 0.037500 +2021-03-26 05:11:25,598 epoch 55 - iter 20/25 - loss 2.66016315 - samples/sec: 142.75 - lr: 0.037500 +2021-03-26 05:11:26,611 epoch 55 - iter 22/25 - loss 2.69519200 - samples/sec: 126.71 - lr: 0.037500 +2021-03-26 05:11:27,617 epoch 55 - iter 24/25 - loss 2.66808570 - samples/sec: 127.43 - lr: 0.037500 +2021-03-26 05:11:28,000 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:11:28,001 EPOCH 55 done: loss 2.6832 - lr 0.0375000 +2021-03-26 05:11:28,754 DEV : loss 5.843713283538818 - score 0.9141 +2021-03-26 05:11:28,772 BAD EPOCHS (no improvement): 3 +2021-03-26 05:11:28,772 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:11:29,808 epoch 56 - iter 2/25 - loss 2.50143147 - samples/sec: 123.76 - lr: 0.037500 +2021-03-26 05:11:30,889 epoch 56 - iter 4/25 - loss 2.65481919 - samples/sec: 118.62 - lr: 0.037500 +2021-03-26 05:11:32,017 epoch 56 - iter 6/25 - loss 2.60150150 - samples/sec: 113.60 - lr: 0.037500 +2021-03-26 05:11:33,057 epoch 56 - iter 8/25 - loss 2.60987782 - samples/sec: 123.20 - lr: 0.037500 +2021-03-26 05:11:33,940 epoch 56 - iter 10/25 - loss 2.58933198 - samples/sec: 145.08 - lr: 0.037500 +2021-03-26 05:11:34,885 epoch 56 - iter 12/25 - loss 2.58792406 - samples/sec: 135.73 - lr: 0.037500 +2021-03-26 05:11:35,833 epoch 56 - iter 14/25 - loss 2.56192599 - samples/sec: 135.15 - lr: 0.037500 +2021-03-26 05:11:36,891 epoch 56 - iter 16/25 - loss 2.60430720 - samples/sec: 121.24 - lr: 0.037500 +2021-03-26 05:11:37,917 epoch 56 - iter 18/25 - loss 2.60791723 - samples/sec: 124.84 - lr: 0.037500 +2021-03-26 05:11:38,932 epoch 56 - iter 20/25 - loss 2.63114707 - samples/sec: 126.37 - lr: 0.037500 +2021-03-26 05:11:39,958 epoch 56 - iter 22/25 - loss 2.62431546 - samples/sec: 124.85 - lr: 0.037500 +2021-03-26 05:11:40,929 epoch 56 - iter 24/25 - loss 2.63437254 - samples/sec: 132.04 - lr: 0.037500 +2021-03-26 05:11:41,364 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:11:41,365 EPOCH 56 done: loss 2.6203 - lr 0.0375000 +2021-03-26 05:11:42,109 DEV : loss 5.871529579162598 - score 0.9149 +2021-03-26 05:11:42,134 BAD EPOCHS (no improvement): 4 +2021-03-26 05:11:42,135 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:11:43,263 epoch 57 - iter 2/25 - loss 2.50472903 - samples/sec: 113.60 - lr: 0.018750 +2021-03-26 05:11:44,255 epoch 57 - iter 4/25 - loss 2.49861062 - samples/sec: 129.41 - lr: 0.018750 +2021-03-26 05:11:45,263 epoch 57 - iter 6/25 - loss 2.46727121 - samples/sec: 127.14 - lr: 0.018750 +2021-03-26 05:11:46,321 epoch 57 - iter 8/25 - loss 2.36317959 - samples/sec: 121.23 - lr: 0.018750 +2021-03-26 05:11:47,310 epoch 57 - iter 10/25 - loss 2.46769011 - samples/sec: 129.66 - lr: 0.018750 +2021-03-26 05:11:48,323 epoch 57 - iter 12/25 - loss 2.48527586 - samples/sec: 126.71 - lr: 0.018750 +2021-03-26 05:11:49,358 epoch 57 - iter 14/25 - loss 2.50677931 - samples/sec: 123.84 - lr: 0.018750 +2021-03-26 05:11:50,425 epoch 57 - iter 16/25 - loss 2.46721734 - samples/sec: 120.02 - lr: 0.018750 +2021-03-26 05:11:51,317 epoch 57 - iter 18/25 - loss 2.47471072 - samples/sec: 143.74 - lr: 0.018750 +2021-03-26 05:11:52,350 epoch 57 - iter 20/25 - loss 2.49596874 - samples/sec: 124.22 - lr: 0.018750 +2021-03-26 05:11:53,357 epoch 57 - iter 22/25 - loss 2.52411051 - samples/sec: 127.30 - lr: 0.018750 +2021-03-26 05:11:54,282 epoch 57 - iter 24/25 - loss 2.46456073 - samples/sec: 138.49 - lr: 0.018750 +2021-03-26 05:11:54,675 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:11:54,675 EPOCH 57 done: loss 2.4844 - lr 0.0187500 +2021-03-26 05:11:55,418 DEV : loss 5.8934326171875 - score 0.9141 +2021-03-26 05:11:55,443 BAD EPOCHS (no improvement): 1 +2021-03-26 05:11:55,443 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:11:56,448 epoch 58 - iter 2/25 - loss 2.22876573 - samples/sec: 127.67 - lr: 0.018750 +2021-03-26 05:11:57,451 epoch 58 - iter 4/25 - loss 2.52895451 - samples/sec: 127.87 - lr: 0.018750 +2021-03-26 05:11:58,483 epoch 58 - iter 6/25 - loss 2.48068396 - samples/sec: 124.18 - lr: 0.018750 +2021-03-26 05:11:59,489 epoch 58 - iter 8/25 - loss 2.41449891 - samples/sec: 127.56 - lr: 0.018750 +2021-03-26 05:12:00,499 epoch 58 - iter 10/25 - loss 2.40381507 - samples/sec: 127.00 - lr: 0.018750 +2021-03-26 05:12:01,522 epoch 58 - iter 12/25 - loss 2.42167565 - samples/sec: 125.30 - lr: 0.018750 +2021-03-26 05:12:02,549 epoch 58 - iter 14/25 - loss 2.52895261 - samples/sec: 124.93 - lr: 0.018750 +2021-03-26 05:12:03,595 epoch 58 - iter 16/25 - loss 2.56519403 - samples/sec: 122.76 - lr: 0.018750 +2021-03-26 05:12:04,581 epoch 58 - iter 18/25 - loss 2.54275488 - samples/sec: 129.98 - lr: 0.018750 +2021-03-26 05:12:05,694 epoch 58 - iter 20/25 - loss 2.51648801 - samples/sec: 115.12 - lr: 0.018750 +2021-03-26 05:12:06,939 epoch 58 - iter 22/25 - loss 2.52750042 - samples/sec: 102.93 - lr: 0.018750 +2021-03-26 05:12:07,900 epoch 58 - iter 24/25 - loss 2.53430993 - samples/sec: 133.41 - lr: 0.018750 +2021-03-26 05:12:08,329 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:12:08,330 EPOCH 58 done: loss 2.5247 - lr 0.0187500 +2021-03-26 05:12:09,139 DEV : loss 5.869925498962402 - score 0.9153 +2021-03-26 05:12:09,163 BAD EPOCHS (no improvement): 2 +2021-03-26 05:12:09,164 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:12:10,200 epoch 59 - iter 2/25 - loss 2.34672618 - samples/sec: 123.70 - lr: 0.018750 +2021-03-26 05:12:11,324 epoch 59 - iter 4/25 - loss 2.17811352 - samples/sec: 114.08 - lr: 0.018750 +2021-03-26 05:12:12,332 epoch 59 - iter 6/25 - loss 2.23459947 - samples/sec: 127.22 - lr: 0.018750 +2021-03-26 05:12:13,388 epoch 59 - iter 8/25 - loss 2.42494446 - samples/sec: 121.54 - lr: 0.018750 +2021-03-26 05:12:14,448 epoch 59 - iter 10/25 - loss 2.44249034 - samples/sec: 120.92 - lr: 0.018750 +2021-03-26 05:12:15,374 epoch 59 - iter 12/25 - loss 2.42561487 - samples/sec: 138.50 - lr: 0.018750 +2021-03-26 05:12:16,325 epoch 59 - iter 14/25 - loss 2.37253753 - samples/sec: 134.70 - lr: 0.018750 +2021-03-26 05:12:17,346 epoch 59 - iter 16/25 - loss 2.36989839 - samples/sec: 125.65 - lr: 0.018750 +2021-03-26 05:12:18,355 epoch 59 - iter 18/25 - loss 2.37962128 - samples/sec: 127.11 - lr: 0.018750 +2021-03-26 05:12:19,338 epoch 59 - iter 20/25 - loss 2.39560186 - samples/sec: 130.44 - lr: 0.018750 +2021-03-26 05:12:20,283 epoch 59 - iter 22/25 - loss 2.39514251 - samples/sec: 135.67 - lr: 0.018750 +2021-03-26 05:12:21,443 epoch 59 - iter 24/25 - loss 2.44506241 - samples/sec: 110.53 - lr: 0.018750 +2021-03-26 05:12:21,938 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:12:21,939 EPOCH 59 done: loss 2.4598 - lr 0.0187500 +2021-03-26 05:12:22,694 DEV : loss 5.848148345947266 - score 0.9153 +2021-03-26 05:12:22,718 BAD EPOCHS (no improvement): 3 +2021-03-26 05:12:22,719 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:12:23,910 epoch 60 - iter 2/25 - loss 3.09506989 - samples/sec: 107.58 - lr: 0.018750 +2021-03-26 05:12:26,192 epoch 60 - iter 4/25 - loss 2.75254214 - samples/sec: 56.14 - lr: 0.018750 +2021-03-26 05:12:27,279 epoch 60 - iter 6/25 - loss 2.84813265 - samples/sec: 117.88 - lr: 0.018750 +2021-03-26 05:12:28,297 epoch 60 - iter 8/25 - loss 2.69154215 - samples/sec: 125.84 - lr: 0.018750 +2021-03-26 05:12:29,232 epoch 60 - iter 10/25 - loss 2.61896937 - samples/sec: 137.16 - lr: 0.018750 +2021-03-26 05:12:30,252 epoch 60 - iter 12/25 - loss 2.55158060 - samples/sec: 125.64 - lr: 0.018750 +2021-03-26 05:12:31,363 epoch 60 - iter 14/25 - loss 2.55699714 - samples/sec: 115.47 - lr: 0.018750 +2021-03-26 05:12:32,463 epoch 60 - iter 16/25 - loss 2.57129791 - samples/sec: 116.69 - lr: 0.018750 +2021-03-26 05:12:33,580 epoch 60 - iter 18/25 - loss 2.60079477 - samples/sec: 114.69 - lr: 0.018750 +2021-03-26 05:12:34,567 epoch 60 - iter 20/25 - loss 2.59394053 - samples/sec: 130.11 - lr: 0.018750 +2021-03-26 05:12:35,746 epoch 60 - iter 22/25 - loss 2.63753519 - samples/sec: 108.65 - lr: 0.018750 +2021-03-26 05:12:36,862 epoch 60 - iter 24/25 - loss 2.60684501 - samples/sec: 114.85 - lr: 0.018750 +2021-03-26 05:12:37,274 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:12:37,274 EPOCH 60 done: loss 2.6101 - lr 0.0187500 +2021-03-26 05:12:38,056 DEV : loss 5.879796028137207 - score 0.9165 +2021-03-26 05:12:38,074 BAD EPOCHS (no improvement): 4 +2021-03-26 05:12:38,075 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:12:39,090 epoch 61 - iter 2/25 - loss 2.78632987 - samples/sec: 126.29 - lr: 0.009375 +2021-03-26 05:12:40,059 epoch 61 - iter 4/25 - loss 2.51424420 - samples/sec: 132.54 - lr: 0.009375 +2021-03-26 05:12:40,989 epoch 61 - iter 6/25 - loss 2.56003817 - samples/sec: 137.81 - lr: 0.009375 +2021-03-26 05:12:41,999 epoch 61 - iter 8/25 - loss 2.39456305 - samples/sec: 127.01 - lr: 0.009375 +2021-03-26 05:12:42,951 epoch 61 - iter 10/25 - loss 2.40821805 - samples/sec: 134.69 - lr: 0.009375 +2021-03-26 05:12:43,956 epoch 61 - iter 12/25 - loss 2.53350836 - samples/sec: 127.51 - lr: 0.009375 +2021-03-26 05:12:44,919 epoch 61 - iter 14/25 - loss 2.53313923 - samples/sec: 133.12 - lr: 0.009375 +2021-03-26 05:12:45,990 epoch 61 - iter 16/25 - loss 2.51551811 - samples/sec: 119.72 - lr: 0.009375 +2021-03-26 05:12:47,052 epoch 61 - iter 18/25 - loss 2.51872024 - samples/sec: 120.76 - lr: 0.009375 +2021-03-26 05:12:48,033 epoch 61 - iter 20/25 - loss 2.50309972 - samples/sec: 130.59 - lr: 0.009375 +2021-03-26 05:12:49,082 epoch 61 - iter 22/25 - loss 2.48143146 - samples/sec: 122.17 - lr: 0.009375 +2021-03-26 05:12:50,012 epoch 61 - iter 24/25 - loss 2.45347960 - samples/sec: 138.08 - lr: 0.009375 +2021-03-26 05:12:50,468 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:12:50,468 EPOCH 61 done: loss 2.4569 - lr 0.0093750 +2021-03-26 05:12:51,225 DEV : loss 5.87412166595459 - score 0.9161 +2021-03-26 05:12:51,250 BAD EPOCHS (no improvement): 1 +2021-03-26 05:12:51,250 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:12:52,234 epoch 62 - iter 2/25 - loss 2.94621849 - samples/sec: 130.36 - lr: 0.009375 +2021-03-26 05:12:53,488 epoch 62 - iter 4/25 - loss 2.98037022 - samples/sec: 102.16 - lr: 0.009375 +2021-03-26 05:12:54,500 epoch 62 - iter 6/25 - loss 2.73766508 - samples/sec: 126.65 - lr: 0.009375 +2021-03-26 05:12:55,659 epoch 62 - iter 8/25 - loss 2.73090996 - samples/sec: 110.57 - lr: 0.009375 +2021-03-26 05:12:56,756 epoch 62 - iter 10/25 - loss 2.75914966 - samples/sec: 116.86 - lr: 0.009375 +2021-03-26 05:12:57,696 epoch 62 - iter 12/25 - loss 2.66616900 - samples/sec: 136.38 - lr: 0.009375 +2021-03-26 05:12:58,648 epoch 62 - iter 14/25 - loss 2.62130229 - samples/sec: 134.58 - lr: 0.009375 +2021-03-26 05:12:59,693 epoch 62 - iter 16/25 - loss 2.59688076 - samples/sec: 122.69 - lr: 0.009375 +2021-03-26 05:13:00,661 epoch 62 - iter 18/25 - loss 2.63257953 - samples/sec: 132.59 - lr: 0.009375 +2021-03-26 05:13:01,796 epoch 62 - iter 20/25 - loss 2.62566335 - samples/sec: 112.96 - lr: 0.009375 +2021-03-26 05:13:02,774 epoch 62 - iter 22/25 - loss 2.58263093 - samples/sec: 131.28 - lr: 0.009375 +2021-03-26 05:13:03,781 epoch 62 - iter 24/25 - loss 2.53657119 - samples/sec: 127.23 - lr: 0.009375 +2021-03-26 05:13:04,209 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:13:04,209 EPOCH 62 done: loss 2.5635 - lr 0.0093750 +2021-03-26 05:13:05,058 DEV : loss 5.863091945648193 - score 0.9165 +2021-03-26 05:13:05,087 BAD EPOCHS (no improvement): 2 +2021-03-26 05:13:05,088 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:13:06,122 epoch 63 - iter 2/25 - loss 3.11381161 - samples/sec: 123.94 - lr: 0.009375 +2021-03-26 05:13:07,116 epoch 63 - iter 4/25 - loss 2.85715598 - samples/sec: 129.11 - lr: 0.009375 +2021-03-26 05:13:08,228 epoch 63 - iter 6/25 - loss 2.82112515 - samples/sec: 115.23 - lr: 0.009375 +2021-03-26 05:13:09,259 epoch 63 - iter 8/25 - loss 2.75335273 - samples/sec: 124.37 - lr: 0.009375 +2021-03-26 05:13:10,286 epoch 63 - iter 10/25 - loss 2.71493399 - samples/sec: 124.77 - lr: 0.009375 +2021-03-26 05:13:11,242 epoch 63 - iter 12/25 - loss 2.60437027 - samples/sec: 134.13 - lr: 0.009375 +2021-03-26 05:13:12,288 epoch 63 - iter 14/25 - loss 2.62287546 - samples/sec: 122.53 - lr: 0.009375 +2021-03-26 05:13:13,354 epoch 63 - iter 16/25 - loss 2.64130027 - samples/sec: 120.16 - lr: 0.009375 +2021-03-26 05:13:14,451 epoch 63 - iter 18/25 - loss 2.67891765 - samples/sec: 116.81 - lr: 0.009375 +2021-03-26 05:13:15,483 epoch 63 - iter 20/25 - loss 2.69270143 - samples/sec: 124.19 - lr: 0.009375 +2021-03-26 05:13:16,393 epoch 63 - iter 22/25 - loss 2.65427635 - samples/sec: 141.12 - lr: 0.009375 +2021-03-26 05:13:17,485 epoch 63 - iter 24/25 - loss 2.63275237 - samples/sec: 117.50 - lr: 0.009375 +2021-03-26 05:13:17,966 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:13:17,967 EPOCH 63 done: loss 2.5749 - lr 0.0093750 +2021-03-26 05:13:18,819 DEV : loss 5.869942665100098 - score 0.9161 +2021-03-26 05:13:18,855 BAD EPOCHS (no improvement): 3 +2021-03-26 05:13:18,855 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:13:19,939 epoch 64 - iter 2/25 - loss 2.58264530 - samples/sec: 118.29 - lr: 0.009375 +2021-03-26 05:13:20,939 epoch 64 - iter 4/25 - loss 2.61912549 - samples/sec: 128.18 - lr: 0.009375 +2021-03-26 05:13:22,026 epoch 64 - iter 6/25 - loss 2.56370831 - samples/sec: 117.86 - lr: 0.009375 +2021-03-26 05:13:23,047 epoch 64 - iter 8/25 - loss 2.60431099 - samples/sec: 125.72 - lr: 0.009375 +2021-03-26 05:13:24,057 epoch 64 - iter 10/25 - loss 2.60761056 - samples/sec: 126.93 - lr: 0.009375 +2021-03-26 05:13:25,106 epoch 64 - iter 12/25 - loss 2.72470246 - samples/sec: 122.07 - lr: 0.009375 +2021-03-26 05:13:26,033 epoch 64 - iter 14/25 - loss 2.66825621 - samples/sec: 138.53 - lr: 0.009375 +2021-03-26 05:13:27,101 epoch 64 - iter 16/25 - loss 2.60022439 - samples/sec: 120.00 - lr: 0.009375 +2021-03-26 05:13:28,074 epoch 64 - iter 18/25 - loss 2.60289984 - samples/sec: 131.76 - lr: 0.009375 +2021-03-26 05:13:29,159 epoch 64 - iter 20/25 - loss 2.58121090 - samples/sec: 118.53 - lr: 0.009375 +2021-03-26 05:13:30,114 epoch 64 - iter 22/25 - loss 2.57763591 - samples/sec: 134.19 - lr: 0.009375 +2021-03-26 05:13:31,172 epoch 64 - iter 24/25 - loss 2.54952568 - samples/sec: 121.21 - lr: 0.009375 +2021-03-26 05:13:31,584 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:13:31,584 EPOCH 64 done: loss 2.5479 - lr 0.0093750 +2021-03-26 05:13:32,339 DEV : loss 5.880767822265625 - score 0.9153 +2021-03-26 05:13:32,360 BAD EPOCHS (no improvement): 4 +2021-03-26 05:13:32,360 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:13:33,437 epoch 65 - iter 2/25 - loss 2.48567939 - samples/sec: 119.00 - lr: 0.004687 +2021-03-26 05:13:34,449 epoch 65 - iter 4/25 - loss 2.40336627 - samples/sec: 126.63 - lr: 0.004687 +2021-03-26 05:13:35,517 epoch 65 - iter 6/25 - loss 2.40060925 - samples/sec: 120.07 - lr: 0.004687 +2021-03-26 05:13:36,655 epoch 65 - iter 8/25 - loss 2.41814449 - samples/sec: 112.69 - lr: 0.004687 +2021-03-26 05:13:37,690 epoch 65 - iter 10/25 - loss 2.37873857 - samples/sec: 123.99 - lr: 0.004687 +2021-03-26 05:13:38,751 epoch 65 - iter 12/25 - loss 2.45327397 - samples/sec: 120.76 - lr: 0.004687 +2021-03-26 05:13:39,797 epoch 65 - iter 14/25 - loss 2.53240495 - samples/sec: 122.67 - lr: 0.004687 +2021-03-26 05:13:40,798 epoch 65 - iter 16/25 - loss 2.51361752 - samples/sec: 128.07 - lr: 0.004687 +2021-03-26 05:13:41,785 epoch 65 - iter 18/25 - loss 2.53079632 - samples/sec: 129.86 - lr: 0.004687 +2021-03-26 05:13:42,803 epoch 65 - iter 20/25 - loss 2.53446673 - samples/sec: 125.99 - lr: 0.004687 +2021-03-26 05:13:43,922 epoch 65 - iter 22/25 - loss 2.52693764 - samples/sec: 114.57 - lr: 0.004687 +2021-03-26 05:13:44,886 epoch 65 - iter 24/25 - loss 2.52916571 - samples/sec: 133.22 - lr: 0.004687 +2021-03-26 05:13:45,382 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:13:45,383 EPOCH 65 done: loss 2.5173 - lr 0.0046875 +2021-03-26 05:13:46,142 DEV : loss 5.894221305847168 - score 0.9145 +2021-03-26 05:13:46,161 BAD EPOCHS (no improvement): 1 +2021-03-26 05:13:46,161 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:13:47,203 epoch 66 - iter 2/25 - loss 2.66583157 - samples/sec: 122.99 - lr: 0.004687 +2021-03-26 05:13:48,255 epoch 66 - iter 4/25 - loss 2.67538059 - samples/sec: 122.04 - lr: 0.004687 +2021-03-26 05:13:49,417 epoch 66 - iter 6/25 - loss 2.65424959 - samples/sec: 110.23 - lr: 0.004687 +2021-03-26 05:13:50,456 epoch 66 - iter 8/25 - loss 2.47670087 - samples/sec: 123.34 - lr: 0.004687 +2021-03-26 05:13:51,585 epoch 66 - iter 10/25 - loss 2.41680605 - samples/sec: 113.51 - lr: 0.004687 +2021-03-26 05:13:52,562 epoch 66 - iter 12/25 - loss 2.40879808 - samples/sec: 131.25 - lr: 0.004687 +2021-03-26 05:13:53,556 epoch 66 - iter 14/25 - loss 2.41184230 - samples/sec: 128.90 - lr: 0.004687 +2021-03-26 05:13:54,512 epoch 66 - iter 16/25 - loss 2.43771230 - samples/sec: 134.14 - lr: 0.004687 +2021-03-26 05:13:55,522 epoch 66 - iter 18/25 - loss 2.44391036 - samples/sec: 126.81 - lr: 0.004687 +2021-03-26 05:13:56,665 epoch 66 - iter 20/25 - loss 2.44685799 - samples/sec: 112.12 - lr: 0.004687 +2021-03-26 05:13:57,641 epoch 66 - iter 22/25 - loss 2.45734792 - samples/sec: 131.33 - lr: 0.004687 +2021-03-26 05:13:58,658 epoch 66 - iter 24/25 - loss 2.47441368 - samples/sec: 126.08 - lr: 0.004687 +2021-03-26 05:13:59,212 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:13:59,213 EPOCH 66 done: loss 2.4660 - lr 0.0046875 +2021-03-26 05:13:59,990 DEV : loss 5.8916521072387695 - score 0.9145 +2021-03-26 05:14:00,015 BAD EPOCHS (no improvement): 2 +2021-03-26 05:14:00,015 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:14:01,057 epoch 67 - iter 2/25 - loss 2.12723714 - samples/sec: 123.10 - lr: 0.004687 +2021-03-26 05:14:02,138 epoch 67 - iter 4/25 - loss 2.38154635 - samples/sec: 118.62 - lr: 0.004687 +2021-03-26 05:14:03,340 epoch 67 - iter 6/25 - loss 2.29852877 - samples/sec: 106.67 - lr: 0.004687 +2021-03-26 05:14:04,341 epoch 67 - iter 8/25 - loss 2.45876713 - samples/sec: 127.97 - lr: 0.004687 +2021-03-26 05:14:05,262 epoch 67 - iter 10/25 - loss 2.47288162 - samples/sec: 139.40 - lr: 0.004687 +2021-03-26 05:14:06,179 epoch 67 - iter 12/25 - loss 2.52856566 - samples/sec: 139.82 - lr: 0.004687 +2021-03-26 05:14:07,310 epoch 67 - iter 14/25 - loss 2.53270457 - samples/sec: 113.46 - lr: 0.004687 +2021-03-26 05:14:08,291 epoch 67 - iter 16/25 - loss 2.51299741 - samples/sec: 131.29 - lr: 0.004687 +2021-03-26 05:14:09,357 epoch 67 - iter 18/25 - loss 2.54054016 - samples/sec: 120.20 - lr: 0.004687 +2021-03-26 05:14:10,623 epoch 67 - iter 20/25 - loss 2.52876206 - samples/sec: 101.22 - lr: 0.004687 +2021-03-26 05:14:11,603 epoch 67 - iter 22/25 - loss 2.48196209 - samples/sec: 130.77 - lr: 0.004687 +2021-03-26 05:14:12,712 epoch 67 - iter 24/25 - loss 2.50243033 - samples/sec: 115.67 - lr: 0.004687 +2021-03-26 05:14:13,202 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:14:13,204 EPOCH 67 done: loss 2.4775 - lr 0.0046875 +2021-03-26 05:14:13,970 DEV : loss 5.894336700439453 - score 0.9149 +2021-03-26 05:14:13,994 BAD EPOCHS (no improvement): 3 +2021-03-26 05:14:13,995 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:14:14,997 epoch 68 - iter 2/25 - loss 2.99291348 - samples/sec: 127.91 - lr: 0.004687 +2021-03-26 05:14:16,162 epoch 68 - iter 4/25 - loss 2.62084621 - samples/sec: 110.09 - lr: 0.004687 +2021-03-26 05:14:17,201 epoch 68 - iter 6/25 - loss 2.47720150 - samples/sec: 123.29 - lr: 0.004687 +2021-03-26 05:14:18,203 epoch 68 - iter 8/25 - loss 2.42659986 - samples/sec: 127.96 - lr: 0.004687 +2021-03-26 05:14:19,273 epoch 68 - iter 10/25 - loss 2.46118934 - samples/sec: 119.76 - lr: 0.004687 +2021-03-26 05:14:20,279 epoch 68 - iter 12/25 - loss 2.50323576 - samples/sec: 127.52 - lr: 0.004687 +2021-03-26 05:14:21,324 epoch 68 - iter 14/25 - loss 2.51408729 - samples/sec: 122.60 - lr: 0.004687 +2021-03-26 05:14:22,284 epoch 68 - iter 16/25 - loss 2.47996278 - samples/sec: 133.74 - lr: 0.004687 +2021-03-26 05:14:23,295 epoch 68 - iter 18/25 - loss 2.43154002 - samples/sec: 126.92 - lr: 0.004687 +2021-03-26 05:14:24,251 epoch 68 - iter 20/25 - loss 2.43471692 - samples/sec: 134.03 - lr: 0.004687 +2021-03-26 05:14:25,238 epoch 68 - iter 22/25 - loss 2.39672428 - samples/sec: 129.89 - lr: 0.004687 +2021-03-26 05:14:26,290 epoch 68 - iter 24/25 - loss 2.39018219 - samples/sec: 121.87 - lr: 0.004687 +2021-03-26 05:14:26,797 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:14:26,798 EPOCH 68 done: loss 2.3901 - lr 0.0046875 +2021-03-26 05:14:27,532 DEV : loss 5.884322166442871 - score 0.9149 +2021-03-26 05:14:27,557 BAD EPOCHS (no improvement): 4 +2021-03-26 05:14:27,558 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:14:28,546 epoch 69 - iter 2/25 - loss 2.71395886 - samples/sec: 129.73 - lr: 0.002344 +2021-03-26 05:14:29,499 epoch 69 - iter 4/25 - loss 2.59460217 - samples/sec: 134.51 - lr: 0.002344 +2021-03-26 05:14:30,626 epoch 69 - iter 6/25 - loss 2.48051115 - samples/sec: 113.74 - lr: 0.002344 +2021-03-26 05:14:31,639 epoch 69 - iter 8/25 - loss 2.50417805 - samples/sec: 126.57 - lr: 0.002344 +2021-03-26 05:14:32,660 epoch 69 - iter 10/25 - loss 2.46161354 - samples/sec: 125.52 - lr: 0.002344 +2021-03-26 05:14:33,644 epoch 69 - iter 12/25 - loss 2.41734342 - samples/sec: 130.14 - lr: 0.002344 +2021-03-26 05:14:34,611 epoch 69 - iter 14/25 - loss 2.47696517 - samples/sec: 132.53 - lr: 0.002344 +2021-03-26 05:14:35,670 epoch 69 - iter 16/25 - loss 2.49385166 - samples/sec: 121.04 - lr: 0.002344 +2021-03-26 05:14:36,768 epoch 69 - iter 18/25 - loss 2.45438148 - samples/sec: 116.73 - lr: 0.002344 +2021-03-26 05:14:37,702 epoch 69 - iter 20/25 - loss 2.42570816 - samples/sec: 137.20 - lr: 0.002344 +2021-03-26 05:14:38,752 epoch 69 - iter 22/25 - loss 2.39344043 - samples/sec: 122.11 - lr: 0.002344 +2021-03-26 05:14:39,838 epoch 69 - iter 24/25 - loss 2.40441241 - samples/sec: 118.11 - lr: 0.002344 +2021-03-26 05:14:40,355 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:14:40,356 EPOCH 69 done: loss 2.3998 - lr 0.0023437 +2021-03-26 05:14:41,098 DEV : loss 5.891051769256592 - score 0.9145 +2021-03-26 05:14:41,122 BAD EPOCHS (no improvement): 1 +2021-03-26 05:14:41,123 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:14:42,187 epoch 70 - iter 2/25 - loss 2.27894652 - samples/sec: 120.46 - lr: 0.002344 +2021-03-26 05:14:43,232 epoch 70 - iter 4/25 - loss 2.45882046 - samples/sec: 122.67 - lr: 0.002344 +2021-03-26 05:14:44,185 epoch 70 - iter 6/25 - loss 2.50836623 - samples/sec: 134.52 - lr: 0.002344 +2021-03-26 05:14:45,178 epoch 70 - iter 8/25 - loss 2.55092400 - samples/sec: 129.08 - lr: 0.002344 +2021-03-26 05:14:46,199 epoch 70 - iter 10/25 - loss 2.50842555 - samples/sec: 125.50 - lr: 0.002344 +2021-03-26 05:14:47,232 epoch 70 - iter 12/25 - loss 2.42263022 - samples/sec: 124.23 - lr: 0.002344 +2021-03-26 05:14:48,402 epoch 70 - iter 14/25 - loss 2.45893973 - samples/sec: 109.54 - lr: 0.002344 +2021-03-26 05:14:49,451 epoch 70 - iter 16/25 - loss 2.55384215 - samples/sec: 122.11 - lr: 0.002344 +2021-03-26 05:14:50,418 epoch 70 - iter 18/25 - loss 2.51961670 - samples/sec: 132.62 - lr: 0.002344 +2021-03-26 05:14:51,354 epoch 70 - iter 20/25 - loss 2.51997625 - samples/sec: 137.17 - lr: 0.002344 +2021-03-26 05:14:52,302 epoch 70 - iter 22/25 - loss 2.48214970 - samples/sec: 135.22 - lr: 0.002344 +2021-03-26 05:14:53,306 epoch 70 - iter 24/25 - loss 2.48843359 - samples/sec: 127.55 - lr: 0.002344 +2021-03-26 05:14:53,738 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:14:53,738 EPOCH 70 done: loss 2.4984 - lr 0.0023437 +2021-03-26 05:14:54,464 DEV : loss 5.888722896575928 - score 0.9149 +2021-03-26 05:14:54,488 BAD EPOCHS (no improvement): 2 +2021-03-26 05:14:54,489 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:14:55,538 epoch 71 - iter 2/25 - loss 3.02102542 - samples/sec: 122.14 - lr: 0.002344 +2021-03-26 05:14:56,471 epoch 71 - iter 4/25 - loss 2.67315236 - samples/sec: 137.54 - lr: 0.002344 +2021-03-26 05:14:57,539 epoch 71 - iter 6/25 - loss 2.68371644 - samples/sec: 120.02 - lr: 0.002344 +2021-03-26 05:14:58,609 epoch 71 - iter 8/25 - loss 2.78057234 - samples/sec: 119.82 - lr: 0.002344 +2021-03-26 05:14:59,655 epoch 71 - iter 10/25 - loss 2.71441923 - samples/sec: 122.49 - lr: 0.002344 +2021-03-26 05:15:00,624 epoch 71 - iter 12/25 - loss 2.62355559 - samples/sec: 132.38 - lr: 0.002344 +2021-03-26 05:15:01,638 epoch 71 - iter 14/25 - loss 2.59343140 - samples/sec: 126.45 - lr: 0.002344 +2021-03-26 05:15:02,641 epoch 71 - iter 16/25 - loss 2.58739737 - samples/sec: 127.71 - lr: 0.002344 +2021-03-26 05:15:03,735 epoch 71 - iter 18/25 - loss 2.53276908 - samples/sec: 117.21 - lr: 0.002344 +2021-03-26 05:15:04,761 epoch 71 - iter 20/25 - loss 2.48124036 - samples/sec: 124.86 - lr: 0.002344 +2021-03-26 05:15:05,992 epoch 71 - iter 22/25 - loss 2.48307181 - samples/sec: 104.05 - lr: 0.002344 +2021-03-26 05:15:07,021 epoch 71 - iter 24/25 - loss 2.51121307 - samples/sec: 124.58 - lr: 0.002344 +2021-03-26 05:15:07,383 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:15:07,383 EPOCH 71 done: loss 2.5028 - lr 0.0023437 +2021-03-26 05:15:08,126 DEV : loss 5.893047332763672 - score 0.9149 +2021-03-26 05:15:08,150 BAD EPOCHS (no improvement): 3 +2021-03-26 05:15:08,151 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:15:09,154 epoch 72 - iter 2/25 - loss 2.53216600 - samples/sec: 127.79 - lr: 0.002344 +2021-03-26 05:15:10,199 epoch 72 - iter 4/25 - loss 2.36523569 - samples/sec: 122.60 - lr: 0.002344 +2021-03-26 05:15:11,199 epoch 72 - iter 6/25 - loss 2.33067107 - samples/sec: 128.31 - lr: 0.002344 +2021-03-26 05:15:12,208 epoch 72 - iter 8/25 - loss 2.29951569 - samples/sec: 127.19 - lr: 0.002344 +2021-03-26 05:15:13,256 epoch 72 - iter 10/25 - loss 2.18574475 - samples/sec: 122.38 - lr: 0.002344 +2021-03-26 05:15:14,362 epoch 72 - iter 12/25 - loss 2.24115007 - samples/sec: 115.77 - lr: 0.002344 +2021-03-26 05:15:15,302 epoch 72 - iter 14/25 - loss 2.28182931 - samples/sec: 136.40 - lr: 0.002344 +2021-03-26 05:15:16,284 epoch 72 - iter 16/25 - loss 2.26094202 - samples/sec: 130.60 - lr: 0.002344 +2021-03-26 05:15:17,276 epoch 72 - iter 18/25 - loss 2.30384613 - samples/sec: 129.11 - lr: 0.002344 +2021-03-26 05:15:18,324 epoch 72 - iter 20/25 - loss 2.31356340 - samples/sec: 122.35 - lr: 0.002344 +2021-03-26 05:15:19,379 epoch 72 - iter 22/25 - loss 2.33352863 - samples/sec: 121.49 - lr: 0.002344 +2021-03-26 05:15:20,391 epoch 72 - iter 24/25 - loss 2.37726304 - samples/sec: 126.74 - lr: 0.002344 +2021-03-26 05:15:20,819 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:15:20,819 EPOCH 72 done: loss 2.3748 - lr 0.0023437 +2021-03-26 05:15:21,559 DEV : loss 5.89636754989624 - score 0.9145 +2021-03-26 05:15:21,580 BAD EPOCHS (no improvement): 4 +2021-03-26 05:15:21,581 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:15:22,608 epoch 73 - iter 2/25 - loss 2.45547676 - samples/sec: 124.87 - lr: 0.001172 +2021-03-26 05:15:23,609 epoch 73 - iter 4/25 - loss 2.33488351 - samples/sec: 128.13 - lr: 0.001172 +2021-03-26 05:15:24,542 epoch 73 - iter 6/25 - loss 2.34714580 - samples/sec: 137.56 - lr: 0.001172 +2021-03-26 05:15:25,591 epoch 73 - iter 8/25 - loss 2.31484377 - samples/sec: 122.14 - lr: 0.001172 +2021-03-26 05:15:26,516 epoch 73 - iter 10/25 - loss 2.39485235 - samples/sec: 138.56 - lr: 0.001172 +2021-03-26 05:15:27,651 epoch 73 - iter 12/25 - loss 2.43526415 - samples/sec: 113.01 - lr: 0.001172 +2021-03-26 05:15:28,614 epoch 73 - iter 14/25 - loss 2.40032681 - samples/sec: 133.36 - lr: 0.001172 +2021-03-26 05:15:29,614 epoch 73 - iter 16/25 - loss 2.45623024 - samples/sec: 128.13 - lr: 0.001172 +2021-03-26 05:15:30,586 epoch 73 - iter 18/25 - loss 2.48155908 - samples/sec: 131.99 - lr: 0.001172 +2021-03-26 05:15:31,609 epoch 73 - iter 20/25 - loss 2.49938724 - samples/sec: 125.33 - lr: 0.001172 +2021-03-26 05:15:32,665 epoch 73 - iter 22/25 - loss 2.49379253 - samples/sec: 121.28 - lr: 0.001172 +2021-03-26 05:15:33,706 epoch 73 - iter 24/25 - loss 2.46238670 - samples/sec: 123.33 - lr: 0.001172 +2021-03-26 05:15:34,108 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:15:34,110 EPOCH 73 done: loss 2.4599 - lr 0.0011719 +2021-03-26 05:15:34,864 DEV : loss 5.895561218261719 - score 0.9153 +2021-03-26 05:15:34,888 BAD EPOCHS (no improvement): 1 +2021-03-26 05:15:34,888 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:15:35,937 epoch 74 - iter 2/25 - loss 2.04502749 - samples/sec: 122.26 - lr: 0.001172 +2021-03-26 05:15:36,960 epoch 74 - iter 4/25 - loss 2.12893891 - samples/sec: 125.32 - lr: 0.001172 +2021-03-26 05:15:37,977 epoch 74 - iter 6/25 - loss 2.39949350 - samples/sec: 125.98 - lr: 0.001172 +2021-03-26 05:15:38,946 epoch 74 - iter 8/25 - loss 2.45375013 - samples/sec: 132.30 - lr: 0.001172 +2021-03-26 05:15:39,907 epoch 74 - iter 10/25 - loss 2.48227482 - samples/sec: 133.42 - lr: 0.001172 +2021-03-26 05:15:40,863 epoch 74 - iter 12/25 - loss 2.49502762 - samples/sec: 134.06 - lr: 0.001172 +2021-03-26 05:15:41,847 epoch 74 - iter 14/25 - loss 2.54132497 - samples/sec: 130.24 - lr: 0.001172 +2021-03-26 05:15:42,813 epoch 74 - iter 16/25 - loss 2.54034129 - samples/sec: 132.68 - lr: 0.001172 +2021-03-26 05:15:43,850 epoch 74 - iter 18/25 - loss 2.51002742 - samples/sec: 123.57 - lr: 0.001172 +2021-03-26 05:15:44,895 epoch 74 - iter 20/25 - loss 2.48838024 - samples/sec: 122.82 - lr: 0.001172 +2021-03-26 05:15:45,960 epoch 74 - iter 22/25 - loss 2.52372887 - samples/sec: 120.32 - lr: 0.001172 +2021-03-26 05:15:47,085 epoch 74 - iter 24/25 - loss 2.50319789 - samples/sec: 113.89 - lr: 0.001172 +2021-03-26 05:15:47,526 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:15:47,527 EPOCH 74 done: loss 2.5035 - lr 0.0011719 +2021-03-26 05:15:48,279 DEV : loss 5.892360210418701 - score 0.9145 +2021-03-26 05:15:48,304 BAD EPOCHS (no improvement): 2 +2021-03-26 05:15:48,305 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:15:49,330 epoch 75 - iter 2/25 - loss 2.31683660 - samples/sec: 125.14 - lr: 0.001172 +2021-03-26 05:15:50,392 epoch 75 - iter 4/25 - loss 2.51113093 - samples/sec: 120.76 - lr: 0.001172 +2021-03-26 05:15:51,367 epoch 75 - iter 6/25 - loss 2.62654793 - samples/sec: 131.42 - lr: 0.001172 +2021-03-26 05:15:52,388 epoch 75 - iter 8/25 - loss 2.57659081 - samples/sec: 125.51 - lr: 0.001172 +2021-03-26 05:15:53,485 epoch 75 - iter 10/25 - loss 2.54353738 - samples/sec: 116.86 - lr: 0.001172 +2021-03-26 05:15:54,555 epoch 75 - iter 12/25 - loss 2.55309633 - samples/sec: 119.92 - lr: 0.001172 +2021-03-26 05:15:55,570 epoch 75 - iter 14/25 - loss 2.56957877 - samples/sec: 126.40 - lr: 0.001172 +2021-03-26 05:15:56,552 epoch 75 - iter 16/25 - loss 2.58506122 - samples/sec: 130.60 - lr: 0.001172 +2021-03-26 05:15:57,625 epoch 75 - iter 18/25 - loss 2.55468255 - samples/sec: 119.44 - lr: 0.001172 +2021-03-26 05:15:58,646 epoch 75 - iter 20/25 - loss 2.54961177 - samples/sec: 125.73 - lr: 0.001172 +2021-03-26 05:15:59,622 epoch 75 - iter 22/25 - loss 2.54893047 - samples/sec: 131.40 - lr: 0.001172 +2021-03-26 05:16:00,537 epoch 75 - iter 24/25 - loss 2.52259524 - samples/sec: 140.18 - lr: 0.001172 +2021-03-26 05:16:00,912 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:16:00,913 EPOCH 75 done: loss 2.5293 - lr 0.0011719 +2021-03-26 05:16:01,658 DEV : loss 5.892311096191406 - score 0.9145 +2021-03-26 05:16:01,678 BAD EPOCHS (no improvement): 3 +2021-03-26 05:16:01,678 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:16:02,688 epoch 76 - iter 2/25 - loss 2.29732823 - samples/sec: 127.04 - lr: 0.001172 +2021-03-26 05:16:03,637 epoch 76 - iter 4/25 - loss 2.20136350 - samples/sec: 135.14 - lr: 0.001172 +2021-03-26 05:16:04,571 epoch 76 - iter 6/25 - loss 2.35438251 - samples/sec: 137.34 - lr: 0.001172 +2021-03-26 05:16:05,574 epoch 76 - iter 8/25 - loss 2.44377515 - samples/sec: 127.92 - lr: 0.001172 +2021-03-26 05:16:06,774 epoch 76 - iter 10/25 - loss 2.59143996 - samples/sec: 106.79 - lr: 0.001172 +2021-03-26 05:16:07,862 epoch 76 - iter 12/25 - loss 2.51938006 - samples/sec: 117.82 - lr: 0.001172 +2021-03-26 05:16:08,889 epoch 76 - iter 14/25 - loss 2.47963315 - samples/sec: 124.86 - lr: 0.001172 +2021-03-26 05:16:09,845 epoch 76 - iter 16/25 - loss 2.48009091 - samples/sec: 134.32 - lr: 0.001172 +2021-03-26 05:16:10,790 epoch 76 - iter 18/25 - loss 2.46930445 - samples/sec: 135.56 - lr: 0.001172 +2021-03-26 05:16:11,735 epoch 76 - iter 20/25 - loss 2.44070104 - samples/sec: 135.72 - lr: 0.001172 +2021-03-26 05:16:12,732 epoch 76 - iter 22/25 - loss 2.45098142 - samples/sec: 128.53 - lr: 0.001172 +2021-03-26 05:16:13,835 epoch 76 - iter 24/25 - loss 2.46575201 - samples/sec: 116.25 - lr: 0.001172 +2021-03-26 05:16:14,244 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:16:14,245 EPOCH 76 done: loss 2.4803 - lr 0.0011719 +2021-03-26 05:16:14,978 DEV : loss 5.895107269287109 - score 0.9153 +2021-03-26 05:16:15,002 BAD EPOCHS (no improvement): 4 +2021-03-26 05:16:15,003 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:16:16,058 epoch 77 - iter 2/25 - loss 2.50079191 - samples/sec: 121.56 - lr: 0.000586 +2021-03-26 05:16:17,007 epoch 77 - iter 4/25 - loss 2.34794635 - samples/sec: 135.02 - lr: 0.000586 +2021-03-26 05:16:18,040 epoch 77 - iter 6/25 - loss 2.35746503 - samples/sec: 124.12 - lr: 0.000586 +2021-03-26 05:16:18,979 epoch 77 - iter 8/25 - loss 2.38873762 - samples/sec: 136.50 - lr: 0.000586 +2021-03-26 05:16:20,061 epoch 77 - iter 10/25 - loss 2.47826626 - samples/sec: 118.39 - lr: 0.000586 +2021-03-26 05:16:21,140 epoch 77 - iter 12/25 - loss 2.39502103 - samples/sec: 118.86 - lr: 0.000586 +2021-03-26 05:16:22,180 epoch 77 - iter 14/25 - loss 2.39116090 - samples/sec: 123.17 - lr: 0.000586 +2021-03-26 05:16:23,213 epoch 77 - iter 16/25 - loss 2.40831371 - samples/sec: 124.12 - lr: 0.000586 +2021-03-26 05:16:24,201 epoch 77 - iter 18/25 - loss 2.46315281 - samples/sec: 129.72 - lr: 0.000586 +2021-03-26 05:16:25,258 epoch 77 - iter 20/25 - loss 2.53796518 - samples/sec: 121.30 - lr: 0.000586 +2021-03-26 05:16:26,328 epoch 77 - iter 22/25 - loss 2.50689035 - samples/sec: 119.73 - lr: 0.000586 +2021-03-26 05:16:27,297 epoch 77 - iter 24/25 - loss 2.49591765 - samples/sec: 132.42 - lr: 0.000586 +2021-03-26 05:16:27,655 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:16:27,656 EPOCH 77 done: loss 2.4828 - lr 0.0005859 +2021-03-26 05:16:28,391 DEV : loss 5.895717620849609 - score 0.9153 +2021-03-26 05:16:28,416 BAD EPOCHS (no improvement): 1 +2021-03-26 05:16:28,416 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:16:29,466 epoch 78 - iter 2/25 - loss 2.57285833 - samples/sec: 122.15 - lr: 0.000586 +2021-03-26 05:16:30,481 epoch 78 - iter 4/25 - loss 2.78017128 - samples/sec: 126.44 - lr: 0.000586 +2021-03-26 05:16:31,399 epoch 78 - iter 6/25 - loss 2.68961970 - samples/sec: 139.73 - lr: 0.000586 +2021-03-26 05:16:32,386 epoch 78 - iter 8/25 - loss 2.59651193 - samples/sec: 129.87 - lr: 0.000586 +2021-03-26 05:16:33,413 epoch 78 - iter 10/25 - loss 2.53945725 - samples/sec: 124.83 - lr: 0.000586 +2021-03-26 05:16:34,389 epoch 78 - iter 12/25 - loss 2.55630420 - samples/sec: 131.41 - lr: 0.000586 +2021-03-26 05:16:35,423 epoch 78 - iter 14/25 - loss 2.51264097 - samples/sec: 124.17 - lr: 0.000586 +2021-03-26 05:16:36,391 epoch 78 - iter 16/25 - loss 2.52606347 - samples/sec: 132.54 - lr: 0.000586 +2021-03-26 05:16:37,456 epoch 78 - iter 18/25 - loss 2.55921306 - samples/sec: 120.33 - lr: 0.000586 +2021-03-26 05:16:38,424 epoch 78 - iter 20/25 - loss 2.57019910 - samples/sec: 132.37 - lr: 0.000586 +2021-03-26 05:16:39,406 epoch 78 - iter 22/25 - loss 2.58772639 - samples/sec: 130.54 - lr: 0.000586 +2021-03-26 05:16:40,495 epoch 78 - iter 24/25 - loss 2.58014371 - samples/sec: 117.67 - lr: 0.000586 +2021-03-26 05:16:40,987 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:16:40,987 EPOCH 78 done: loss 2.5614 - lr 0.0005859 +2021-03-26 05:16:41,754 DEV : loss 5.895739555358887 - score 0.9145 +2021-03-26 05:16:41,771 BAD EPOCHS (no improvement): 2 +2021-03-26 05:16:41,772 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:16:42,890 epoch 79 - iter 2/25 - loss 2.56692624 - samples/sec: 114.65 - lr: 0.000586 +2021-03-26 05:16:43,923 epoch 79 - iter 4/25 - loss 2.41068244 - samples/sec: 124.04 - lr: 0.000586 +2021-03-26 05:16:44,935 epoch 79 - iter 6/25 - loss 2.32666834 - samples/sec: 126.65 - lr: 0.000586 +2021-03-26 05:16:45,878 epoch 79 - iter 8/25 - loss 2.32099563 - samples/sec: 135.88 - lr: 0.000586 +2021-03-26 05:16:47,046 epoch 79 - iter 10/25 - loss 2.39023049 - samples/sec: 109.85 - lr: 0.000586 +2021-03-26 05:16:48,037 epoch 79 - iter 12/25 - loss 2.40536650 - samples/sec: 129.47 - lr: 0.000586 +2021-03-26 05:16:49,053 epoch 79 - iter 14/25 - loss 2.34349969 - samples/sec: 126.15 - lr: 0.000586 +2021-03-26 05:16:50,086 epoch 79 - iter 16/25 - loss 2.35110615 - samples/sec: 124.14 - lr: 0.000586 +2021-03-26 05:16:51,225 epoch 79 - iter 18/25 - loss 2.35502756 - samples/sec: 112.50 - lr: 0.000586 +2021-03-26 05:16:52,310 epoch 79 - iter 20/25 - loss 2.37096562 - samples/sec: 118.04 - lr: 0.000586 +2021-03-26 05:16:53,567 epoch 79 - iter 22/25 - loss 2.38549516 - samples/sec: 102.01 - lr: 0.000586 +2021-03-26 05:16:54,747 epoch 79 - iter 24/25 - loss 2.43781242 - samples/sec: 108.60 - lr: 0.000586 +2021-03-26 05:16:55,188 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:16:55,190 EPOCH 79 done: loss 2.4295 - lr 0.0005859 +2021-03-26 05:16:56,051 DEV : loss 5.895864009857178 - score 0.9145 +2021-03-26 05:16:56,074 BAD EPOCHS (no improvement): 3 +2021-03-26 05:16:56,075 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:16:57,105 epoch 80 - iter 2/25 - loss 2.79264188 - samples/sec: 124.50 - lr: 0.000586 +2021-03-26 05:16:58,139 epoch 80 - iter 4/25 - loss 2.76948452 - samples/sec: 123.99 - lr: 0.000586 +2021-03-26 05:16:59,172 epoch 80 - iter 6/25 - loss 2.66697685 - samples/sec: 123.99 - lr: 0.000586 +2021-03-26 05:17:00,172 epoch 80 - iter 8/25 - loss 2.63417286 - samples/sec: 128.34 - lr: 0.000586 +2021-03-26 05:17:01,246 epoch 80 - iter 10/25 - loss 2.61623344 - samples/sec: 119.38 - lr: 0.000586 +2021-03-26 05:17:02,219 epoch 80 - iter 12/25 - loss 2.60252472 - samples/sec: 131.81 - lr: 0.000586 +2021-03-26 05:17:03,270 epoch 80 - iter 14/25 - loss 2.58438189 - samples/sec: 121.86 - lr: 0.000586 +2021-03-26 05:17:04,387 epoch 80 - iter 16/25 - loss 2.54688759 - samples/sec: 114.80 - lr: 0.000586 +2021-03-26 05:17:05,376 epoch 80 - iter 18/25 - loss 2.59013311 - samples/sec: 130.15 - lr: 0.000586 +2021-03-26 05:17:06,459 epoch 80 - iter 20/25 - loss 2.53017637 - samples/sec: 118.32 - lr: 0.000586 +2021-03-26 05:17:07,503 epoch 80 - iter 22/25 - loss 2.49253048 - samples/sec: 122.79 - lr: 0.000586 +2021-03-26 05:17:08,425 epoch 80 - iter 24/25 - loss 2.48274991 - samples/sec: 139.14 - lr: 0.000586 +2021-03-26 05:17:08,858 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:17:08,859 EPOCH 80 done: loss 2.4664 - lr 0.0005859 +2021-03-26 05:17:09,695 DEV : loss 5.894993782043457 - score 0.9149 +2021-03-26 05:17:09,730 BAD EPOCHS (no improvement): 4 +2021-03-26 05:17:09,731 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:17:10,817 epoch 81 - iter 2/25 - loss 2.55748820 - samples/sec: 118.03 - lr: 0.000293 +2021-03-26 05:17:11,991 epoch 81 - iter 4/25 - loss 2.57345551 - samples/sec: 109.24 - lr: 0.000293 +2021-03-26 05:17:13,067 epoch 81 - iter 6/25 - loss 2.54840223 - samples/sec: 119.13 - lr: 0.000293 +2021-03-26 05:17:14,169 epoch 81 - iter 8/25 - loss 2.46514797 - samples/sec: 116.34 - lr: 0.000293 +2021-03-26 05:17:15,292 epoch 81 - iter 10/25 - loss 2.34481575 - samples/sec: 114.29 - lr: 0.000293 +2021-03-26 05:17:16,417 epoch 81 - iter 12/25 - loss 2.35811732 - samples/sec: 114.02 - lr: 0.000293 +2021-03-26 05:17:17,416 epoch 81 - iter 14/25 - loss 2.29000000 - samples/sec: 128.35 - lr: 0.000293 +2021-03-26 05:17:18,363 epoch 81 - iter 16/25 - loss 2.31538082 - samples/sec: 135.39 - lr: 0.000293 +2021-03-26 05:17:19,453 epoch 81 - iter 18/25 - loss 2.32631816 - samples/sec: 117.56 - lr: 0.000293 +2021-03-26 05:17:20,445 epoch 81 - iter 20/25 - loss 2.39415182 - samples/sec: 129.27 - lr: 0.000293 +2021-03-26 05:17:21,415 epoch 81 - iter 22/25 - loss 2.41178614 - samples/sec: 132.20 - lr: 0.000293 +2021-03-26 05:17:22,378 epoch 81 - iter 24/25 - loss 2.42476410 - samples/sec: 133.28 - lr: 0.000293 +2021-03-26 05:17:22,787 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:17:22,788 EPOCH 81 done: loss 2.4483 - lr 0.0002930 +2021-03-26 05:17:23,546 DEV : loss 5.89434814453125 - score 0.9149 +2021-03-26 05:17:23,566 BAD EPOCHS (no improvement): 1 +2021-03-26 05:17:23,566 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:17:24,574 epoch 82 - iter 2/25 - loss 2.53205180 - samples/sec: 127.22 - lr: 0.000293 +2021-03-26 05:17:25,609 epoch 82 - iter 4/25 - loss 2.74800831 - samples/sec: 123.83 - lr: 0.000293 +2021-03-26 05:17:26,804 epoch 82 - iter 6/25 - loss 2.64432458 - samples/sec: 107.28 - lr: 0.000293 +2021-03-26 05:17:27,892 epoch 82 - iter 8/25 - loss 2.65834966 - samples/sec: 117.75 - lr: 0.000293 +2021-03-26 05:17:28,872 epoch 82 - iter 10/25 - loss 2.64527481 - samples/sec: 130.87 - lr: 0.000293 +2021-03-26 05:17:29,960 epoch 82 - iter 12/25 - loss 2.59270751 - samples/sec: 117.89 - lr: 0.000293 +2021-03-26 05:17:31,073 epoch 82 - iter 14/25 - loss 2.53665956 - samples/sec: 115.22 - lr: 0.000293 +2021-03-26 05:17:32,158 epoch 82 - iter 16/25 - loss 2.57464282 - samples/sec: 118.07 - lr: 0.000293 +2021-03-26 05:17:33,226 epoch 82 - iter 18/25 - loss 2.56387957 - samples/sec: 120.00 - lr: 0.000293 +2021-03-26 05:17:34,313 epoch 82 - iter 20/25 - loss 2.53260517 - samples/sec: 117.98 - lr: 0.000293 +2021-03-26 05:17:35,219 epoch 82 - iter 22/25 - loss 2.50258301 - samples/sec: 141.54 - lr: 0.000293 +2021-03-26 05:17:36,244 epoch 82 - iter 24/25 - loss 2.49314828 - samples/sec: 124.97 - lr: 0.000293 +2021-03-26 05:17:36,689 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:17:36,690 EPOCH 82 done: loss 2.4795 - lr 0.0002930 +2021-03-26 05:17:37,457 DEV : loss 5.894433498382568 - score 0.9149 +2021-03-26 05:17:37,475 BAD EPOCHS (no improvement): 2 +2021-03-26 05:17:37,475 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:17:38,444 epoch 83 - iter 2/25 - loss 2.64499736 - samples/sec: 132.45 - lr: 0.000293 +2021-03-26 05:17:39,536 epoch 83 - iter 4/25 - loss 2.47873867 - samples/sec: 117.36 - lr: 0.000293 +2021-03-26 05:17:40,528 epoch 83 - iter 6/25 - loss 2.45670923 - samples/sec: 129.37 - lr: 0.000293 +2021-03-26 05:17:41,576 epoch 83 - iter 8/25 - loss 2.44804201 - samples/sec: 122.37 - lr: 0.000293 +2021-03-26 05:17:42,539 epoch 83 - iter 10/25 - loss 2.41737645 - samples/sec: 133.09 - lr: 0.000293 +2021-03-26 05:17:43,639 epoch 83 - iter 12/25 - loss 2.54113217 - samples/sec: 116.60 - lr: 0.000293 +2021-03-26 05:17:44,607 epoch 83 - iter 14/25 - loss 2.51169845 - samples/sec: 132.33 - lr: 0.000293 +2021-03-26 05:17:45,556 epoch 83 - iter 16/25 - loss 2.45529628 - samples/sec: 135.07 - lr: 0.000293 +2021-03-26 05:17:46,709 epoch 83 - iter 18/25 - loss 2.44203492 - samples/sec: 111.10 - lr: 0.000293 +2021-03-26 05:17:47,775 epoch 83 - iter 20/25 - loss 2.44877447 - samples/sec: 120.43 - lr: 0.000293 +2021-03-26 05:17:48,799 epoch 83 - iter 22/25 - loss 2.43847547 - samples/sec: 125.11 - lr: 0.000293 +2021-03-26 05:17:49,865 epoch 83 - iter 24/25 - loss 2.41514853 - samples/sec: 120.24 - lr: 0.000293 +2021-03-26 05:17:50,383 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:17:50,384 EPOCH 83 done: loss 2.4312 - lr 0.0002930 +2021-03-26 05:17:51,148 DEV : loss 5.894194602966309 - score 0.9149 +2021-03-26 05:17:51,173 BAD EPOCHS (no improvement): 3 +2021-03-26 05:17:51,173 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:17:52,185 epoch 84 - iter 2/25 - loss 2.68529451 - samples/sec: 126.80 - lr: 0.000293 +2021-03-26 05:17:53,304 epoch 84 - iter 4/25 - loss 2.73050523 - samples/sec: 114.46 - lr: 0.000293 +2021-03-26 05:17:54,349 epoch 84 - iter 6/25 - loss 2.87833659 - samples/sec: 122.86 - lr: 0.000293 +2021-03-26 05:17:55,398 epoch 84 - iter 8/25 - loss 2.77546391 - samples/sec: 122.19 - lr: 0.000293 +2021-03-26 05:17:56,412 epoch 84 - iter 10/25 - loss 2.66991322 - samples/sec: 126.45 - lr: 0.000293 +2021-03-26 05:17:57,450 epoch 84 - iter 12/25 - loss 2.60607652 - samples/sec: 123.46 - lr: 0.000293 +2021-03-26 05:17:58,487 epoch 84 - iter 14/25 - loss 2.59675641 - samples/sec: 123.65 - lr: 0.000293 +2021-03-26 05:17:59,547 epoch 84 - iter 16/25 - loss 2.57955612 - samples/sec: 121.00 - lr: 0.000293 +2021-03-26 05:18:00,699 epoch 84 - iter 18/25 - loss 2.57765302 - samples/sec: 111.25 - lr: 0.000293 +2021-03-26 05:18:01,740 epoch 84 - iter 20/25 - loss 2.54013187 - samples/sec: 123.33 - lr: 0.000293 +2021-03-26 05:18:02,736 epoch 84 - iter 22/25 - loss 2.53648597 - samples/sec: 128.65 - lr: 0.000293 +2021-03-26 05:18:03,676 epoch 84 - iter 24/25 - loss 2.51152799 - samples/sec: 136.43 - lr: 0.000293 +2021-03-26 05:18:04,070 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:18:04,071 EPOCH 84 done: loss 2.5123 - lr 0.0002930 +2021-03-26 05:18:04,829 DEV : loss 5.8933563232421875 - score 0.9153 +2021-03-26 05:18:04,853 BAD EPOCHS (no improvement): 4 +2021-03-26 05:18:04,854 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:18:05,828 epoch 85 - iter 2/25 - loss 1.89106965 - samples/sec: 131.72 - lr: 0.000146 +2021-03-26 05:18:06,827 epoch 85 - iter 4/25 - loss 2.02825582 - samples/sec: 128.21 - lr: 0.000146 +2021-03-26 05:18:07,851 epoch 85 - iter 6/25 - loss 2.21381227 - samples/sec: 125.20 - lr: 0.000146 +2021-03-26 05:18:08,779 epoch 85 - iter 8/25 - loss 2.27279621 - samples/sec: 138.20 - lr: 0.000146 +2021-03-26 05:18:09,736 epoch 85 - iter 10/25 - loss 2.23479022 - samples/sec: 134.07 - lr: 0.000146 +2021-03-26 05:18:10,719 epoch 85 - iter 12/25 - loss 2.35115473 - samples/sec: 130.41 - lr: 0.000146 +2021-03-26 05:18:11,775 epoch 85 - iter 14/25 - loss 2.36515174 - samples/sec: 121.38 - lr: 0.000146 +2021-03-26 05:18:12,720 epoch 85 - iter 16/25 - loss 2.35759029 - samples/sec: 135.56 - lr: 0.000146 +2021-03-26 05:18:13,776 epoch 85 - iter 18/25 - loss 2.45850852 - samples/sec: 121.35 - lr: 0.000146 +2021-03-26 05:18:14,894 epoch 85 - iter 20/25 - loss 2.40584831 - samples/sec: 114.71 - lr: 0.000146 +2021-03-26 05:18:15,826 epoch 85 - iter 22/25 - loss 2.40154030 - samples/sec: 137.61 - lr: 0.000146 +2021-03-26 05:18:16,847 epoch 85 - iter 24/25 - loss 2.39086893 - samples/sec: 125.51 - lr: 0.000146 +2021-03-26 05:18:17,362 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:18:17,362 EPOCH 85 done: loss 2.3930 - lr 0.0001465 +2021-03-26 05:18:18,105 DEV : loss 5.893399238586426 - score 0.9157 +2021-03-26 05:18:18,129 BAD EPOCHS (no improvement): 1 +2021-03-26 05:18:18,130 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:18:19,065 epoch 86 - iter 2/25 - loss 2.54171717 - samples/sec: 137.16 - lr: 0.000146 +2021-03-26 05:18:19,991 epoch 86 - iter 4/25 - loss 2.48442131 - samples/sec: 138.64 - lr: 0.000146 +2021-03-26 05:18:21,000 epoch 86 - iter 6/25 - loss 2.51889253 - samples/sec: 127.10 - lr: 0.000146 +2021-03-26 05:18:21,992 epoch 86 - iter 8/25 - loss 2.56522778 - samples/sec: 129.29 - lr: 0.000146 +2021-03-26 05:18:23,055 epoch 86 - iter 10/25 - loss 2.58503950 - samples/sec: 120.51 - lr: 0.000146 +2021-03-26 05:18:24,115 epoch 86 - iter 12/25 - loss 2.57443593 - samples/sec: 120.91 - lr: 0.000146 +2021-03-26 05:18:25,045 epoch 86 - iter 14/25 - loss 2.51936642 - samples/sec: 138.08 - lr: 0.000146 +2021-03-26 05:18:26,061 epoch 86 - iter 16/25 - loss 2.50782646 - samples/sec: 126.09 - lr: 0.000146 +2021-03-26 05:18:27,083 epoch 86 - iter 18/25 - loss 2.51853973 - samples/sec: 125.46 - lr: 0.000146 +2021-03-26 05:18:28,032 epoch 86 - iter 20/25 - loss 2.46419308 - samples/sec: 135.59 - lr: 0.000146 +2021-03-26 05:18:29,187 epoch 86 - iter 22/25 - loss 2.43465135 - samples/sec: 111.02 - lr: 0.000146 +2021-03-26 05:18:30,187 epoch 86 - iter 24/25 - loss 2.45168869 - samples/sec: 128.31 - lr: 0.000146 +2021-03-26 05:18:30,669 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:18:30,669 EPOCH 86 done: loss 2.4679 - lr 0.0001465 +2021-03-26 05:18:31,446 DEV : loss 5.893395900726318 - score 0.9153 +2021-03-26 05:18:31,463 BAD EPOCHS (no improvement): 2 +2021-03-26 05:18:31,464 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:18:32,535 epoch 87 - iter 2/25 - loss 2.91735208 - samples/sec: 119.70 - lr: 0.000146 +2021-03-26 05:18:33,557 epoch 87 - iter 4/25 - loss 2.84264398 - samples/sec: 125.54 - lr: 0.000146 +2021-03-26 05:18:34,544 epoch 87 - iter 6/25 - loss 2.56568110 - samples/sec: 129.80 - lr: 0.000146 +2021-03-26 05:18:35,558 epoch 87 - iter 8/25 - loss 2.63990408 - samples/sec: 126.46 - lr: 0.000146 +2021-03-26 05:18:36,455 epoch 87 - iter 10/25 - loss 2.56502144 - samples/sec: 142.83 - lr: 0.000146 +2021-03-26 05:18:37,387 epoch 87 - iter 12/25 - loss 2.53154775 - samples/sec: 137.50 - lr: 0.000146 +2021-03-26 05:18:38,397 epoch 87 - iter 14/25 - loss 2.54436823 - samples/sec: 126.86 - lr: 0.000146 +2021-03-26 05:18:39,401 epoch 87 - iter 16/25 - loss 2.61185640 - samples/sec: 127.69 - lr: 0.000146 +2021-03-26 05:18:40,370 epoch 87 - iter 18/25 - loss 2.54913453 - samples/sec: 132.51 - lr: 0.000146 +2021-03-26 05:18:41,463 epoch 87 - iter 20/25 - loss 2.50478936 - samples/sec: 117.20 - lr: 0.000146 +2021-03-26 05:18:42,489 epoch 87 - iter 22/25 - loss 2.52266684 - samples/sec: 124.89 - lr: 0.000146 +2021-03-26 05:18:43,496 epoch 87 - iter 24/25 - loss 2.56201922 - samples/sec: 127.35 - lr: 0.000146 +2021-03-26 05:18:43,856 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:18:43,856 EPOCH 87 done: loss 2.5356 - lr 0.0001465 +2021-03-26 05:18:44,595 DEV : loss 5.893139839172363 - score 0.9153 +2021-03-26 05:18:44,620 BAD EPOCHS (no improvement): 3 +2021-03-26 05:18:44,621 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:18:45,646 epoch 88 - iter 2/25 - loss 2.54361033 - samples/sec: 125.02 - lr: 0.000146 +2021-03-26 05:18:46,620 epoch 88 - iter 4/25 - loss 2.52765530 - samples/sec: 131.69 - lr: 0.000146 +2021-03-26 05:18:47,691 epoch 88 - iter 6/25 - loss 2.64498627 - samples/sec: 119.75 - lr: 0.000146 +2021-03-26 05:18:48,676 epoch 88 - iter 8/25 - loss 2.57745424 - samples/sec: 130.13 - lr: 0.000146 +2021-03-26 05:18:49,701 epoch 88 - iter 10/25 - loss 2.45820544 - samples/sec: 125.23 - lr: 0.000146 +2021-03-26 05:18:50,694 epoch 88 - iter 12/25 - loss 2.43848614 - samples/sec: 129.11 - lr: 0.000146 +2021-03-26 05:18:51,837 epoch 88 - iter 14/25 - loss 2.40640150 - samples/sec: 112.61 - lr: 0.000146 +2021-03-26 05:18:52,925 epoch 88 - iter 16/25 - loss 2.45308697 - samples/sec: 117.86 - lr: 0.000146 +2021-03-26 05:18:53,985 epoch 88 - iter 18/25 - loss 2.43245529 - samples/sec: 120.87 - lr: 0.000146 +2021-03-26 05:18:54,902 epoch 88 - iter 20/25 - loss 2.46451479 - samples/sec: 139.94 - lr: 0.000146 +2021-03-26 05:18:55,882 epoch 88 - iter 22/25 - loss 2.47260539 - samples/sec: 130.79 - lr: 0.000146 +2021-03-26 05:18:57,029 epoch 88 - iter 24/25 - loss 2.48033151 - samples/sec: 111.84 - lr: 0.000146 +2021-03-26 05:18:57,437 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:18:57,438 EPOCH 88 done: loss 2.4450 - lr 0.0001465 +2021-03-26 05:18:58,197 DEV : loss 5.893143653869629 - score 0.9153 +2021-03-26 05:18:58,222 BAD EPOCHS (no improvement): 4 +2021-03-26 05:18:58,223 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:18:58,223 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:18:58,224 learning rate too small - quitting training! +2021-03-26 05:18:58,224 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:19:07,624 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:19:07,624 Testing using best model ... +2021-03-26 05:19:07,625 loading file /home/tmp/megahedm/models/multipos/multipos_UDMADAR_4Diale-LEV_EGY_GLF_MGR__fasttext_flairbwfw__64__0.3_202103260453/best-model.pt +2021-03-26 05:19:15,030 0.9089 +2021-03-26 05:19:15,030 +Results: +- F-score (micro): 0.9058 +- F-score (macro): 0.5975 +- Accuracy (incl. no class): 0.9089 + +By class: + precision recall f1-score support + + SCONJ 0.9697 0.9412 0.9552 34 + PRON 0.9453 0.9744 0.9596 195 + ADP 0.9604 0.9604 0.9604 101 + AUX 0.9118 0.9688 0.9394 32 + VERB 0.9615 0.9058 0.9328 138 + ADV 0.9160 0.8934 0.9046 122 + PART 0.9583 0.9684 0.9634 190 + NOUN 0.9224 0.9378 0.9300 418 + CCONJ 1.0000 0.9438 0.9711 89 + ADJ 0.9205 0.8710 0.8950 93 + DET 0.9545 0.8936 0.9231 47 + NUM 0.9048 0.7917 0.8444 24 + PUNCT 1.0000 1.0000 1.0000 30 + PROPN 0.8000 0.8000 0.8000 20 + INTJ 1.0000 1.0000 1.0000 19 + X 1.0000 0.0000 0.0000 1 + FOREIGN 1.0000 1.0000 1.0000 3 + MENTION 1.0000 1.0000 1.0000 20 + NOUN+NSUFF 0.8000 0.9180 0.8550 61 + ADJ+NSUFF 0.7500 0.6562 0.7000 32 + CONJ+NOUN+NSUFF 1.0000 1.0000 1.0000 1 + DET+NOUN 0.9726 0.9861 0.9793 72 + V 0.8500 0.8500 0.8500 80 + PREP+DET+NOUN 0.6923 0.9000 0.7826 10 + CONJ+PART 1.0000 1.0000 1.0000 9 + PREP+V 1.0000 0.0000 0.0000 2 + CONJ+V 0.7857 0.8462 0.8148 13 + PREP 0.9219 0.9833 0.9516 60 + DET+NOUN+NSUFF 0.9167 0.9167 0.9167 24 + DET+ADJ+NSUFF 0.6667 0.8000 0.7273 5 + EOS 1.0000 1.0000 1.0000 70 + NOUN+PRON 0.7901 0.8533 0.8205 75 + V+PRON 0.7541 0.8070 0.7797 57 + CONJ+NOUN+PRON 0.7500 0.5000 0.6000 6 + PUNC 0.9919 1.0000 0.9960 123 + V+PRON+PRON 0.5455 0.5000 0.5217 12 + CONJ+V+PREP+PRON 0.0000 0.0000 0.0000 2 + CONJ+V+PRON 0.4444 0.8000 0.5714 5 + PROG_PART+V 0.7500 0.8462 0.7952 39 + PART+PRON 1.0000 0.9444 0.9714 18 + CONJ 1.0000 1.0000 1.0000 44 + V+PREP+PRON 0.3333 0.4000 0.3636 5 + PART+V+PRON 1.0000 1.0000 1.0000 1 + CONJ+PRON 1.0000 1.0000 1.0000 6 + PRON+DET+NOUN 1.0000 0.2500 0.4000 4 + PRON+DET+NOUN+NSUFF 0.3333 1.0000 0.5000 1 + PREP+PART+PRON 1.0000 0.8000 0.8889 5 + PROG_PART+V+PRON 0.6000 0.9231 0.7273 13 + PREP+DET+ADJ 1.0000 0.0000 0.0000 3 + PREP+NOUN 0.7500 0.6000 0.6667 10 + PREP+NOUN+PRON 0.5000 0.1667 0.2500 6 + PREP+DET+NOUN+NSUFF 1.0000 0.5000 0.6667 2 + HASH 0.9688 1.0000 0.9841 31 + FUT_PART+V 0.8571 0.5455 0.6667 11 + PREP+PRON 0.8947 0.8500 0.8718 20 + PREP+PRON+DET 1.0000 0.0000 0.0000 1 + CONJ+DET+NOUN 1.0000 1.0000 1.0000 6 + NOUN+PREP+PRON 1.0000 0.0000 0.0000 1 + V+PRON+PREP+PRON 0.0000 0.0000 0.0000 1 + NOUN+NSUFF+PRON 0.5385 0.7000 0.6087 10 + FUT_PART 1.0000 1.0000 1.0000 5 + CONJ+ADJ 1.0000 1.0000 1.0000 1 + PREP+NOUN+NSUFF+PRON 0.6667 0.6667 0.6667 3 + PART+V 0.0000 0.0000 0.0000 1 + PART+NOUN+PRON 1.0000 0.0000 0.0000 1 + PROG_PART+V+PREP+PRON 1.0000 0.0000 0.0000 2 + ADJ+PRON 0.6667 0.8000 0.7273 5 + EMOT 1.0000 1.0000 1.0000 10 + CONJ+FUT_PART+V 1.0000 0.0000 0.0000 3 + CONJ+PROG_PART+V 0.2500 1.0000 0.4000 1 + PREP+NOUN+NSUFF 1.0000 1.0000 1.0000 1 + PART+V+NEG_PART 0.5000 0.5000 0.5000 4 + CONJ+PREP+PRON 1.0000 1.0000 1.0000 1 + PREP+PART 1.0000 0.0000 0.0000 1 + CONJ+NOUN 0.8333 0.8333 0.8333 6 + ADJ+NSUFF+PRON 1.0000 0.0000 0.0000 1 + PROG_PART+V+NEG_PART 0.0000 1.0000 0.0000 0 + NOUN+NSUFF+NSUFF 1.0000 0.0000 0.0000 1 + ADV+PRON 1.0000 0.0000 0.0000 1 + PROG_PART+V+PRON+PRON 0.0000 1.0000 0.0000 0 + PART+FUT_PART 1.0000 0.0000 0.0000 1 + URL 1.0000 1.0000 1.0000 1 + CONJ+PART+V+PRON 1.0000 0.0000 0.0000 1 + CONJ+PART+PROG_PART+V 1.0000 0.0000 0.0000 1 + PART+NOUN+PRON+NEG_PART 1.0000 0.0000 0.0000 1 + PART+PREP+PRON+NEG_PART 0.0000 1.0000 0.0000 0 + NUM+NSUFF 1.0000 0.0000 0.0000 1 + CONJ+NOUN+NSUFF+PRON 0.0000 1.0000 0.0000 0 + ADV+NSUFF 1.0000 0.6667 0.8000 3 + PART+V+PRON+NEG_PART 0.4286 0.7500 0.5455 4 + NOUN+CASE 1.0000 0.7500 0.8571 4 + DET+ADJ 0.8750 1.0000 0.9333 7 + ADJ+CASE 1.0000 1.0000 1.0000 3 +PART+V+PREP+PRON+NEG_PART 1.0000 0.0000 0.0000 1 +PART+PROG_PART+V+NEG_PART 0.5000 1.0000 0.6667 1 + PART+NOUN+NEG_PART 1.0000 0.0000 0.0000 2 + PART+PREP+NEG_PART 1.0000 1.0000 1.0000 4 + DET+NUM 1.0000 0.0000 0.0000 1 + PART+NOUN 1.0000 1.0000 1.0000 1 + PART+PART 1.0000 0.0000 0.0000 1 + CONJ+PROG_PART+V+PRON 1.0000 1.0000 1.0000 1 + FUT_PART+V+PRON+PRON 1.0000 0.0000 0.0000 1 + CONJ+ADV 1.0000 1.0000 1.0000 1 + V+NEG_PART 1.0000 0.0000 0.0000 1 + + micro avg 0.9058 0.9058 0.9058 2623 + macro avg 0.8270 0.6506 0.5975 2623 + weighted avg 0.9135 0.9058 0.9019 2623 + +2021-03-26 05:19:15,031 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:19:15,031 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:19:21,164 Reading data from ../../Datasets_adhoc/CSCS_corpus-GUC +2021-03-26 05:19:21,165 Train: ../../Datasets_adhoc/CSCS_corpus-GUC/all_participants.conllu +2021-03-26 05:19:21,165 Dev: None +2021-03-26 05:19:21,165 Test: None +2021-03-26 05:19:21,457 Reading data from ../../Datasets_adhoc/UD_MADAR +2021-03-26 05:19:21,458 Train: ../../Datasets_adhoc/UD_MADAR/ajp_madar-ud-test-edit.conllu +2021-03-26 05:19:21,458 Dev: None +2021-03-26 05:19:21,458 Test: None +2021-03-26 05:19:22,836 Reading data from ../../Datasets_adhoc/dialectal_arabic_resources-master +2021-03-26 05:19:22,836 Train: ../../Datasets_adhoc/dialectal_arabic_resources-master/seg_plus_pos_lev.txt +2021-03-26 05:19:22,837 Dev: None +2021-03-26 05:19:22,837 Test: None +2021-03-26 05:19:23,009 Reading data from ../../Datasets_adhoc/dialectal_arabic_resources-master +2021-03-26 05:19:23,009 Train: ../../Datasets_adhoc/dialectal_arabic_resources-master/seg_plus_pos_egy.txt +2021-03-26 05:19:23,009 Dev: None +2021-03-26 05:19:23,010 Test: None +2021-03-26 05:19:23,174 Reading data from ../../Datasets_adhoc/dialectal_arabic_resources-master +2021-03-26 05:19:23,175 Train: ../../Datasets_adhoc/dialectal_arabic_resources-master/seg_plus_pos_glf.txt +2021-03-26 05:19:23,175 Dev: None +2021-03-26 05:19:23,175 Test: None +2021-03-26 05:19:23,333 Reading data from ../../Datasets_adhoc/dialectal_arabic_resources-master +2021-03-26 05:19:23,333 Train: ../../Datasets_adhoc/dialectal_arabic_resources-master/seg_plus_pos_mgr.txt +2021-03-26 05:19:23,333 Dev: None +2021-03-26 05:19:23,333 Test: None +2021-03-26 05:19:23,489 Filtering long sentences +2021-03-26 05:19:23,531 MultiCorpus: 1573 train + 177 dev + 194 test sentences + - ColumnCorpus Corpus: 934 train + 104 dev + 115 test sentences + - ColumnCorpus Corpus: 81 train + 9 dev + 10 test sentences + - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences + - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences +2021-03-26 05:19:23,936 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:19:23,937 Model: "SequenceTagger( + (embeddings): StackedEmbeddings( + (list_embedding_0): WordEmbeddings('ar') + (list_embedding_1): FlairEmbeddings( + (lm): LanguageModel( + (drop): Dropout(p=0.1, inplace=False) + (encoder): Embedding(7125, 100) + (rnn): LSTM(100, 2048) + (decoder): Linear(in_features=2048, out_features=7125, bias=True) + ) + ) + (list_embedding_2): FlairEmbeddings( + (lm): LanguageModel( + (drop): Dropout(p=0.1, inplace=False) + (encoder): Embedding(7125, 100) + (rnn): LSTM(100, 2048) + (decoder): Linear(in_features=2048, out_features=7125, bias=True) + ) + ) + ) + (word_dropout): WordDropout(p=0.05) + (locked_dropout): LockedDropout(p=0.5) + (embedding2nn): Linear(in_features=4396, out_features=4396, bias=True) + (rnn): LSTM(4396, 256, batch_first=True, bidirectional=True) + (linear): Linear(in_features=512, out_features=206, bias=True) + (beta): 1.0 + (weights): None + (weight_tensor) None +)" +2021-03-26 05:19:23,937 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:19:23,937 Corpus: "MultiCorpus: 1573 train + 177 dev + 194 test sentences + - ColumnCorpus Corpus: 934 train + 104 dev + 115 test sentences + - ColumnCorpus Corpus: 81 train + 9 dev + 10 test sentences + - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences + - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences" +2021-03-26 05:19:23,938 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:19:23,938 Parameters: +2021-03-26 05:19:23,938 - learning_rate: "0.3" +2021-03-26 05:19:23,938 - mini_batch_size: "64" +2021-03-26 05:19:23,938 - patience: "3" +2021-03-26 05:19:23,939 - anneal_factor: "0.5" +2021-03-26 05:19:23,939 - max_epochs: "150" +2021-03-26 05:19:23,939 - shuffle: "True" +2021-03-26 05:19:23,939 - train_with_dev: "False" +2021-03-26 05:19:23,940 - batch_growth_annealing: "False" +2021-03-26 05:19:23,940 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:19:23,940 Model training base path: "/home/tmp/megahedm/models/multipos/multipos_UDMADAR_4Diale-LEV_EGY_GLF_MGR__fasttext_flairbwfw__64__0.3_202103260519" +2021-03-26 05:19:23,940 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:19:23,941 Device: cuda:0 +2021-03-26 05:19:23,941 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:19:23,941 Embeddings storage mode: cpu +2021-03-26 05:19:23,943 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:19:25,612 epoch 1 - iter 2/25 - loss 81.95887375 - samples/sec: 76.75 - lr: 0.300000 +2021-03-26 05:19:26,880 epoch 1 - iter 4/25 - loss 74.30733109 - samples/sec: 101.12 - lr: 0.300000 +2021-03-26 05:19:28,119 epoch 1 - iter 6/25 - loss 70.52834956 - samples/sec: 103.39 - lr: 0.300000 +2021-03-26 05:19:29,401 epoch 1 - iter 8/25 - loss 68.74974346 - samples/sec: 99.98 - lr: 0.300000 +2021-03-26 05:19:30,716 epoch 1 - iter 10/25 - loss 66.45365906 - samples/sec: 97.44 - lr: 0.300000 +2021-03-26 05:19:31,966 epoch 1 - iter 12/25 - loss 65.50715129 - samples/sec: 102.53 - lr: 0.300000 +2021-03-26 05:19:33,366 epoch 1 - iter 14/25 - loss 63.79074342 - samples/sec: 91.50 - lr: 0.300000 +2021-03-26 05:19:34,641 epoch 1 - iter 16/25 - loss 62.57994103 - samples/sec: 100.49 - lr: 0.300000 +2021-03-26 05:19:36,005 epoch 1 - iter 18/25 - loss 61.44075076 - samples/sec: 93.93 - lr: 0.300000 +2021-03-26 05:19:37,361 epoch 1 - iter 20/25 - loss 60.14129944 - samples/sec: 94.47 - lr: 0.300000 +2021-03-26 05:19:38,797 epoch 1 - iter 22/25 - loss 59.64039404 - samples/sec: 89.22 - lr: 0.300000 +2021-03-26 05:19:40,124 epoch 1 - iter 24/25 - loss 58.65103499 - samples/sec: 96.54 - lr: 0.300000 +2021-03-26 05:19:40,645 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:19:40,645 EPOCH 1 done: loss 57.9464 - lr 0.3000000 +2021-03-26 05:19:41,921 DEV : loss 43.50901412963867 - score 0.3135 +2021-03-26 05:19:41,947 BAD EPOCHS (no improvement): 0 +2021-03-26 05:19:51,382 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:19:52,367 epoch 2 - iter 2/25 - loss 44.36688232 - samples/sec: 130.47 - lr: 0.300000 +2021-03-26 05:19:53,356 epoch 2 - iter 4/25 - loss 42.73228741 - samples/sec: 129.72 - lr: 0.300000 +2021-03-26 05:19:54,393 epoch 2 - iter 6/25 - loss 40.84166209 - samples/sec: 123.59 - lr: 0.300000 +2021-03-26 05:19:55,413 epoch 2 - iter 8/25 - loss 40.15511179 - samples/sec: 125.69 - lr: 0.300000 +2021-03-26 05:19:56,447 epoch 2 - iter 10/25 - loss 40.64278603 - samples/sec: 124.04 - lr: 0.300000 +2021-03-26 05:19:57,458 epoch 2 - iter 12/25 - loss 40.30080064 - samples/sec: 126.89 - lr: 0.300000 +2021-03-26 05:19:58,498 epoch 2 - iter 14/25 - loss 40.01132883 - samples/sec: 123.24 - lr: 0.300000 +2021-03-26 05:19:59,430 epoch 2 - iter 16/25 - loss 39.21363187 - samples/sec: 137.60 - lr: 0.300000 +2021-03-26 05:20:00,477 epoch 2 - iter 18/25 - loss 38.71213330 - samples/sec: 122.37 - lr: 0.300000 +2021-03-26 05:20:01,407 epoch 2 - iter 20/25 - loss 37.77274303 - samples/sec: 137.85 - lr: 0.300000 +2021-03-26 05:20:02,361 epoch 2 - iter 22/25 - loss 37.60560018 - samples/sec: 134.37 - lr: 0.300000 +2021-03-26 05:20:03,321 epoch 2 - iter 24/25 - loss 37.28476222 - samples/sec: 133.44 - lr: 0.300000 +2021-03-26 05:20:03,656 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:20:03,656 EPOCH 2 done: loss 36.7479 - lr 0.3000000 +2021-03-26 05:20:04,447 DEV : loss 33.068904876708984 - score 0.4714 +2021-03-26 05:20:04,465 BAD EPOCHS (no improvement): 0 +2021-03-26 05:20:14,204 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:20:15,255 epoch 3 - iter 2/25 - loss 31.90020370 - samples/sec: 122.13 - lr: 0.300000 +2021-03-26 05:20:16,285 epoch 3 - iter 4/25 - loss 31.23775816 - samples/sec: 124.44 - lr: 0.300000 +2021-03-26 05:20:17,265 epoch 3 - iter 6/25 - loss 30.26333078 - samples/sec: 131.63 - lr: 0.300000 +2021-03-26 05:20:18,213 epoch 3 - iter 8/25 - loss 29.47907281 - samples/sec: 135.10 - lr: 0.300000 +2021-03-26 05:20:19,165 epoch 3 - iter 10/25 - loss 29.23647823 - samples/sec: 134.77 - lr: 0.300000 +2021-03-26 05:20:20,207 epoch 3 - iter 12/25 - loss 29.02191750 - samples/sec: 123.00 - lr: 0.300000 +2021-03-26 05:20:21,277 epoch 3 - iter 14/25 - loss 29.22410951 - samples/sec: 119.75 - lr: 0.300000 +2021-03-26 05:20:22,232 epoch 3 - iter 16/25 - loss 29.01444256 - samples/sec: 134.22 - lr: 0.300000 +2021-03-26 05:20:23,303 epoch 3 - iter 18/25 - loss 28.52765550 - samples/sec: 119.69 - lr: 0.300000 +2021-03-26 05:20:24,415 epoch 3 - iter 20/25 - loss 28.52102280 - samples/sec: 115.28 - lr: 0.300000 +2021-03-26 05:20:25,377 epoch 3 - iter 22/25 - loss 28.27422359 - samples/sec: 133.24 - lr: 0.300000 +2021-03-26 05:20:26,336 epoch 3 - iter 24/25 - loss 27.83582926 - samples/sec: 133.70 - lr: 0.300000 +2021-03-26 05:20:26,824 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:20:26,825 EPOCH 3 done: loss 27.8058 - lr 0.3000000 +2021-03-26 05:20:27,615 DEV : loss 22.344778060913086 - score 0.6352 +2021-03-26 05:20:27,642 BAD EPOCHS (no improvement): 0 +2021-03-26 05:20:37,530 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:20:38,578 epoch 4 - iter 2/25 - loss 23.20929146 - samples/sec: 122.51 - lr: 0.300000 +2021-03-26 05:20:39,519 epoch 4 - iter 4/25 - loss 23.56956482 - samples/sec: 136.19 - lr: 0.300000 +2021-03-26 05:20:40,446 epoch 4 - iter 6/25 - loss 22.66243076 - samples/sec: 138.36 - lr: 0.300000 +2021-03-26 05:20:41,449 epoch 4 - iter 8/25 - loss 23.12808657 - samples/sec: 127.94 - lr: 0.300000 +2021-03-26 05:20:42,463 epoch 4 - iter 10/25 - loss 23.03308296 - samples/sec: 126.45 - lr: 0.300000 +2021-03-26 05:20:43,457 epoch 4 - iter 12/25 - loss 23.30867656 - samples/sec: 128.93 - lr: 0.300000 +2021-03-26 05:20:44,382 epoch 4 - iter 14/25 - loss 22.81305817 - samples/sec: 138.56 - lr: 0.300000 +2021-03-26 05:20:45,381 epoch 4 - iter 16/25 - loss 22.66594744 - samples/sec: 128.33 - lr: 0.300000 +2021-03-26 05:20:46,427 epoch 4 - iter 18/25 - loss 22.44728237 - samples/sec: 122.62 - lr: 0.300000 +2021-03-26 05:20:47,545 epoch 4 - iter 20/25 - loss 22.31258392 - samples/sec: 115.19 - lr: 0.300000 +2021-03-26 05:20:48,552 epoch 4 - iter 22/25 - loss 22.11111285 - samples/sec: 127.39 - lr: 0.300000 +2021-03-26 05:20:49,610 epoch 4 - iter 24/25 - loss 22.03010790 - samples/sec: 121.30 - lr: 0.300000 +2021-03-26 05:20:50,058 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:20:50,058 EPOCH 4 done: loss 22.0868 - lr 0.3000000 +2021-03-26 05:20:50,854 DEV : loss 17.412494659423828 - score 0.7033 +2021-03-26 05:20:50,873 BAD EPOCHS (no improvement): 0 +2021-03-26 05:21:00,674 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:21:01,811 epoch 5 - iter 2/25 - loss 22.17486286 - samples/sec: 112.84 - lr: 0.300000 +2021-03-26 05:21:02,788 epoch 5 - iter 4/25 - loss 21.01916075 - samples/sec: 131.36 - lr: 0.300000 +2021-03-26 05:21:03,715 epoch 5 - iter 6/25 - loss 20.32295863 - samples/sec: 138.47 - lr: 0.300000 +2021-03-26 05:21:04,677 epoch 5 - iter 8/25 - loss 19.95706081 - samples/sec: 133.45 - lr: 0.300000 +2021-03-26 05:21:05,654 epoch 5 - iter 10/25 - loss 19.42259941 - samples/sec: 131.30 - lr: 0.300000 +2021-03-26 05:21:06,592 epoch 5 - iter 12/25 - loss 19.28429031 - samples/sec: 136.74 - lr: 0.300000 +2021-03-26 05:21:07,655 epoch 5 - iter 14/25 - loss 19.41569465 - samples/sec: 120.63 - lr: 0.300000 +2021-03-26 05:21:08,657 epoch 5 - iter 16/25 - loss 19.41432917 - samples/sec: 127.88 - lr: 0.300000 +2021-03-26 05:21:09,640 epoch 5 - iter 18/25 - loss 19.03300916 - samples/sec: 130.35 - lr: 0.300000 +2021-03-26 05:21:10,581 epoch 5 - iter 20/25 - loss 18.95893478 - samples/sec: 136.20 - lr: 0.300000 +2021-03-26 05:21:11,511 epoch 5 - iter 22/25 - loss 18.87176644 - samples/sec: 137.83 - lr: 0.300000 +2021-03-26 05:21:12,484 epoch 5 - iter 24/25 - loss 18.72764615 - samples/sec: 131.73 - lr: 0.300000 +2021-03-26 05:21:12,918 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:21:12,918 EPOCH 5 done: loss 18.6854 - lr 0.3000000 +2021-03-26 05:21:13,709 DEV : loss 14.548635482788086 - score 0.7653 +2021-03-26 05:21:13,736 BAD EPOCHS (no improvement): 0 +2021-03-26 05:21:23,422 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:21:24,410 epoch 6 - iter 2/25 - loss 15.13766766 - samples/sec: 129.88 - lr: 0.300000 +2021-03-26 05:21:25,573 epoch 6 - iter 4/25 - loss 16.41607046 - samples/sec: 110.14 - lr: 0.300000 +2021-03-26 05:21:26,539 epoch 6 - iter 6/25 - loss 16.33872000 - samples/sec: 132.86 - lr: 0.300000 +2021-03-26 05:21:27,528 epoch 6 - iter 8/25 - loss 16.65881395 - samples/sec: 129.65 - lr: 0.300000 +2021-03-26 05:21:28,566 epoch 6 - iter 10/25 - loss 16.38934278 - samples/sec: 123.48 - lr: 0.300000 +2021-03-26 05:21:29,472 epoch 6 - iter 12/25 - loss 15.87123386 - samples/sec: 141.46 - lr: 0.300000 +2021-03-26 05:21:30,478 epoch 6 - iter 14/25 - loss 15.92985732 - samples/sec: 127.35 - lr: 0.300000 +2021-03-26 05:21:31,514 epoch 6 - iter 16/25 - loss 15.99678320 - samples/sec: 123.90 - lr: 0.300000 +2021-03-26 05:21:32,521 epoch 6 - iter 18/25 - loss 15.88874594 - samples/sec: 127.46 - lr: 0.300000 +2021-03-26 05:21:33,613 epoch 6 - iter 20/25 - loss 16.16925859 - samples/sec: 117.39 - lr: 0.300000 +2021-03-26 05:21:34,611 epoch 6 - iter 22/25 - loss 16.03849519 - samples/sec: 128.51 - lr: 0.300000 +2021-03-26 05:21:35,683 epoch 6 - iter 24/25 - loss 16.23500764 - samples/sec: 119.66 - lr: 0.300000 +2021-03-26 05:21:36,060 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:21:36,060 EPOCH 6 done: loss 16.4187 - lr 0.3000000 +2021-03-26 05:21:36,879 DEV : loss 13.509325981140137 - score 0.7897 +2021-03-26 05:21:36,909 BAD EPOCHS (no improvement): 0 +2021-03-26 05:21:46,893 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:21:47,945 epoch 7 - iter 2/25 - loss 17.45014191 - samples/sec: 122.04 - lr: 0.300000 +2021-03-26 05:21:49,160 epoch 7 - iter 4/25 - loss 16.16570139 - samples/sec: 105.47 - lr: 0.300000 +2021-03-26 05:21:50,573 epoch 7 - iter 6/25 - loss 15.32639710 - samples/sec: 90.65 - lr: 0.300000 +2021-03-26 05:21:51,939 epoch 7 - iter 8/25 - loss 14.89120066 - samples/sec: 93.85 - lr: 0.300000 +2021-03-26 05:21:53,445 epoch 7 - iter 10/25 - loss 14.58858881 - samples/sec: 85.07 - lr: 0.300000 +2021-03-26 05:21:54,772 epoch 7 - iter 12/25 - loss 14.68719117 - samples/sec: 96.69 - lr: 0.300000 +2021-03-26 05:21:55,866 epoch 7 - iter 14/25 - loss 14.78044850 - samples/sec: 117.17 - lr: 0.300000 +2021-03-26 05:21:57,147 epoch 7 - iter 16/25 - loss 14.58556020 - samples/sec: 99.99 - lr: 0.300000 +2021-03-26 05:21:58,149 epoch 7 - iter 18/25 - loss 14.70696529 - samples/sec: 127.97 - lr: 0.300000 +2021-03-26 05:21:59,192 epoch 7 - iter 20/25 - loss 14.69041018 - samples/sec: 122.89 - lr: 0.300000 +2021-03-26 05:22:00,259 epoch 7 - iter 22/25 - loss 14.80504721 - samples/sec: 120.07 - lr: 0.300000 +2021-03-26 05:22:01,233 epoch 7 - iter 24/25 - loss 14.69321915 - samples/sec: 131.79 - lr: 0.300000 +2021-03-26 05:22:01,714 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:22:01,714 EPOCH 7 done: loss 14.6265 - lr 0.3000000 +2021-03-26 05:22:02,536 DEV : loss 10.955870628356934 - score 0.8189 +2021-03-26 05:22:02,559 BAD EPOCHS (no improvement): 0 +2021-03-26 05:22:12,290 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:22:13,430 epoch 8 - iter 2/25 - loss 12.19628286 - samples/sec: 112.50 - lr: 0.300000 +2021-03-26 05:22:14,425 epoch 8 - iter 4/25 - loss 13.13171005 - samples/sec: 128.97 - lr: 0.300000 +2021-03-26 05:22:15,458 epoch 8 - iter 6/25 - loss 13.02878348 - samples/sec: 124.17 - lr: 0.300000 +2021-03-26 05:22:16,442 epoch 8 - iter 8/25 - loss 12.53637004 - samples/sec: 130.31 - lr: 0.300000 +2021-03-26 05:22:17,454 epoch 8 - iter 10/25 - loss 12.59346609 - samples/sec: 126.79 - lr: 0.300000 +2021-03-26 05:22:18,601 epoch 8 - iter 12/25 - loss 12.86952901 - samples/sec: 111.75 - lr: 0.300000 +2021-03-26 05:22:19,582 epoch 8 - iter 14/25 - loss 13.26604230 - samples/sec: 130.67 - lr: 0.300000 +2021-03-26 05:22:20,571 epoch 8 - iter 16/25 - loss 13.08011448 - samples/sec: 129.67 - lr: 0.300000 +2021-03-26 05:22:21,576 epoch 8 - iter 18/25 - loss 12.98361148 - samples/sec: 127.56 - lr: 0.300000 +2021-03-26 05:22:22,700 epoch 8 - iter 20/25 - loss 12.99734645 - samples/sec: 114.08 - lr: 0.300000 +2021-03-26 05:22:23,728 epoch 8 - iter 22/25 - loss 13.00599228 - samples/sec: 124.65 - lr: 0.300000 +2021-03-26 05:22:24,735 epoch 8 - iter 24/25 - loss 12.97749150 - samples/sec: 127.30 - lr: 0.300000 +2021-03-26 05:22:25,180 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:22:25,181 EPOCH 8 done: loss 12.9387 - lr 0.3000000 +2021-03-26 05:22:26,010 DEV : loss 9.875782012939453 - score 0.836 +2021-03-26 05:22:26,038 BAD EPOCHS (no improvement): 0 +2021-03-26 05:22:35,930 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:22:37,002 epoch 9 - iter 2/25 - loss 13.14775276 - samples/sec: 119.88 - lr: 0.300000 +2021-03-26 05:22:38,044 epoch 9 - iter 4/25 - loss 12.90201044 - samples/sec: 123.00 - lr: 0.300000 +2021-03-26 05:22:39,092 epoch 9 - iter 6/25 - loss 12.12569332 - samples/sec: 122.33 - lr: 0.300000 +2021-03-26 05:22:40,101 epoch 9 - iter 8/25 - loss 11.99610913 - samples/sec: 127.05 - lr: 0.300000 +2021-03-26 05:22:41,177 epoch 9 - iter 10/25 - loss 12.22638769 - samples/sec: 119.09 - lr: 0.300000 +2021-03-26 05:22:42,153 epoch 9 - iter 12/25 - loss 12.00737731 - samples/sec: 131.48 - lr: 0.300000 +2021-03-26 05:22:43,172 epoch 9 - iter 14/25 - loss 12.14194461 - samples/sec: 125.80 - lr: 0.300000 +2021-03-26 05:22:44,022 epoch 9 - iter 16/25 - loss 11.95955336 - samples/sec: 150.96 - lr: 0.300000 +2021-03-26 05:22:45,047 epoch 9 - iter 18/25 - loss 12.03510804 - samples/sec: 125.01 - lr: 0.300000 +2021-03-26 05:22:46,007 epoch 9 - iter 20/25 - loss 12.04640160 - samples/sec: 133.67 - lr: 0.300000 +2021-03-26 05:22:46,961 epoch 9 - iter 22/25 - loss 12.06278606 - samples/sec: 134.33 - lr: 0.300000 +2021-03-26 05:22:47,959 epoch 9 - iter 24/25 - loss 12.02169251 - samples/sec: 128.46 - lr: 0.300000 +2021-03-26 05:22:48,357 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:22:48,358 EPOCH 9 done: loss 11.9918 - lr 0.3000000 +2021-03-26 05:22:49,153 DEV : loss 9.951400756835938 - score 0.8373 +2021-03-26 05:22:49,173 BAD EPOCHS (no improvement): 0 +2021-03-26 05:22:59,122 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:23:00,224 epoch 10 - iter 2/25 - loss 12.44782639 - samples/sec: 116.48 - lr: 0.300000 +2021-03-26 05:23:01,238 epoch 10 - iter 4/25 - loss 11.81677580 - samples/sec: 126.41 - lr: 0.300000 +2021-03-26 05:23:02,329 epoch 10 - iter 6/25 - loss 11.34556882 - samples/sec: 117.52 - lr: 0.300000 +2021-03-26 05:23:03,619 epoch 10 - iter 8/25 - loss 11.62646890 - samples/sec: 99.33 - lr: 0.300000 +2021-03-26 05:23:04,555 epoch 10 - iter 10/25 - loss 11.20175314 - samples/sec: 137.26 - lr: 0.300000 +2021-03-26 05:23:05,546 epoch 10 - iter 12/25 - loss 11.30701804 - samples/sec: 129.31 - lr: 0.300000 +2021-03-26 05:23:06,587 epoch 10 - iter 14/25 - loss 11.24839946 - samples/sec: 123.13 - lr: 0.300000 +2021-03-26 05:23:07,580 epoch 10 - iter 16/25 - loss 11.15712929 - samples/sec: 129.04 - lr: 0.300000 +2021-03-26 05:23:08,536 epoch 10 - iter 18/25 - loss 10.91551023 - samples/sec: 134.13 - lr: 0.300000 +2021-03-26 05:23:09,584 epoch 10 - iter 20/25 - loss 11.12621126 - samples/sec: 122.41 - lr: 0.300000 +2021-03-26 05:23:10,656 epoch 10 - iter 22/25 - loss 11.30736299 - samples/sec: 119.70 - lr: 0.300000 +2021-03-26 05:23:11,635 epoch 10 - iter 24/25 - loss 11.32537091 - samples/sec: 130.96 - lr: 0.300000 +2021-03-26 05:23:12,084 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:23:12,089 EPOCH 10 done: loss 11.1850 - lr 0.3000000 +2021-03-26 05:23:12,892 DEV : loss 9.316814422607422 - score 0.85 +2021-03-26 05:23:12,918 BAD EPOCHS (no improvement): 0 +2021-03-26 05:23:22,610 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:23:23,675 epoch 11 - iter 2/25 - loss 11.53073597 - samples/sec: 120.46 - lr: 0.300000 +2021-03-26 05:23:24,974 epoch 11 - iter 4/25 - loss 11.28462696 - samples/sec: 98.57 - lr: 0.300000 +2021-03-26 05:23:25,985 epoch 11 - iter 6/25 - loss 10.85450268 - samples/sec: 126.86 - lr: 0.300000 +2021-03-26 05:23:26,991 epoch 11 - iter 8/25 - loss 11.06903195 - samples/sec: 127.47 - lr: 0.300000 +2021-03-26 05:23:27,961 epoch 11 - iter 10/25 - loss 10.65050640 - samples/sec: 132.10 - lr: 0.300000 +2021-03-26 05:23:28,893 epoch 11 - iter 12/25 - loss 10.32414166 - samples/sec: 137.71 - lr: 0.300000 +2021-03-26 05:23:29,793 epoch 11 - iter 14/25 - loss 10.17382424 - samples/sec: 142.40 - lr: 0.300000 +2021-03-26 05:23:30,854 epoch 11 - iter 16/25 - loss 10.42914373 - samples/sec: 120.84 - lr: 0.300000 +2021-03-26 05:23:31,796 epoch 11 - iter 18/25 - loss 10.50087696 - samples/sec: 136.03 - lr: 0.300000 +2021-03-26 05:23:32,865 epoch 11 - iter 20/25 - loss 10.54544296 - samples/sec: 119.85 - lr: 0.300000 +2021-03-26 05:23:33,806 epoch 11 - iter 22/25 - loss 10.47581499 - samples/sec: 136.31 - lr: 0.300000 +2021-03-26 05:23:34,916 epoch 11 - iter 24/25 - loss 10.45924973 - samples/sec: 115.39 - lr: 0.300000 +2021-03-26 05:23:35,288 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:23:35,289 EPOCH 11 done: loss 10.5797 - lr 0.3000000 +2021-03-26 05:23:36,124 DEV : loss 8.551046371459961 - score 0.8569 +2021-03-26 05:23:36,151 BAD EPOCHS (no improvement): 0 +2021-03-26 05:23:45,942 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:23:46,915 epoch 12 - iter 2/25 - loss 10.74679136 - samples/sec: 131.89 - lr: 0.300000 +2021-03-26 05:23:47,927 epoch 12 - iter 4/25 - loss 10.23639774 - samples/sec: 126.62 - lr: 0.300000 +2021-03-26 05:23:48,890 epoch 12 - iter 6/25 - loss 10.10948531 - samples/sec: 133.25 - lr: 0.300000 +2021-03-26 05:23:49,902 epoch 12 - iter 8/25 - loss 10.19528675 - samples/sec: 126.61 - lr: 0.300000 +2021-03-26 05:23:50,901 epoch 12 - iter 10/25 - loss 9.82142744 - samples/sec: 128.32 - lr: 0.300000 +2021-03-26 05:23:51,943 epoch 12 - iter 12/25 - loss 9.58679080 - samples/sec: 123.03 - lr: 0.300000 +2021-03-26 05:23:52,946 epoch 12 - iter 14/25 - loss 9.66052955 - samples/sec: 127.91 - lr: 0.300000 +2021-03-26 05:23:54,045 epoch 12 - iter 16/25 - loss 9.79544091 - samples/sec: 116.59 - lr: 0.300000 +2021-03-26 05:23:55,070 epoch 12 - iter 18/25 - loss 9.79722193 - samples/sec: 125.09 - lr: 0.300000 +2021-03-26 05:23:56,102 epoch 12 - iter 20/25 - loss 9.64639330 - samples/sec: 124.27 - lr: 0.300000 +2021-03-26 05:23:57,259 epoch 12 - iter 22/25 - loss 9.72506645 - samples/sec: 110.77 - lr: 0.300000 +2021-03-26 05:23:58,241 epoch 12 - iter 24/25 - loss 9.83624764 - samples/sec: 130.41 - lr: 0.300000 +2021-03-26 05:23:58,657 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:23:58,657 EPOCH 12 done: loss 9.9765 - lr 0.3000000 +2021-03-26 05:23:59,449 DEV : loss 8.387750625610352 - score 0.8586 +2021-03-26 05:23:59,467 BAD EPOCHS (no improvement): 0 +2021-03-26 05:24:09,166 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:24:10,298 epoch 13 - iter 2/25 - loss 8.35482121 - samples/sec: 113.22 - lr: 0.300000 +2021-03-26 05:24:11,330 epoch 13 - iter 4/25 - loss 8.62327766 - samples/sec: 124.28 - lr: 0.300000 +2021-03-26 05:24:12,379 epoch 13 - iter 6/25 - loss 8.79010773 - samples/sec: 122.13 - lr: 0.300000 +2021-03-26 05:24:13,375 epoch 13 - iter 8/25 - loss 9.15598595 - samples/sec: 128.79 - lr: 0.300000 +2021-03-26 05:24:14,375 epoch 13 - iter 10/25 - loss 9.02062783 - samples/sec: 128.09 - lr: 0.300000 +2021-03-26 05:24:15,440 epoch 13 - iter 12/25 - loss 9.00885872 - samples/sec: 120.33 - lr: 0.300000 +2021-03-26 05:24:16,420 epoch 13 - iter 14/25 - loss 9.12177137 - samples/sec: 130.81 - lr: 0.300000 +2021-03-26 05:24:17,316 epoch 13 - iter 16/25 - loss 9.05517802 - samples/sec: 143.08 - lr: 0.300000 +2021-03-26 05:24:18,340 epoch 13 - iter 18/25 - loss 9.30420470 - samples/sec: 125.33 - lr: 0.300000 +2021-03-26 05:24:19,351 epoch 13 - iter 20/25 - loss 9.40496576 - samples/sec: 126.84 - lr: 0.300000 +2021-03-26 05:24:20,275 epoch 13 - iter 22/25 - loss 9.29032285 - samples/sec: 138.84 - lr: 0.300000 +2021-03-26 05:24:21,216 epoch 13 - iter 24/25 - loss 9.41522477 - samples/sec: 136.34 - lr: 0.300000 +2021-03-26 05:24:21,581 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:24:21,583 EPOCH 13 done: loss 9.3761 - lr 0.3000000 +2021-03-26 05:24:22,387 DEV : loss 8.103069305419922 - score 0.8616 +2021-03-26 05:24:22,417 BAD EPOCHS (no improvement): 0 +2021-03-26 05:24:32,178 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:24:33,201 epoch 14 - iter 2/25 - loss 8.71276283 - samples/sec: 125.42 - lr: 0.300000 +2021-03-26 05:24:34,223 epoch 14 - iter 4/25 - loss 8.67824745 - samples/sec: 125.44 - lr: 0.300000 +2021-03-26 05:24:35,347 epoch 14 - iter 6/25 - loss 9.10136700 - samples/sec: 114.10 - lr: 0.300000 +2021-03-26 05:24:36,303 epoch 14 - iter 8/25 - loss 9.12157381 - samples/sec: 134.01 - lr: 0.300000 +2021-03-26 05:24:37,296 epoch 14 - iter 10/25 - loss 8.77482038 - samples/sec: 129.18 - lr: 0.300000 +2021-03-26 05:24:38,248 epoch 14 - iter 12/25 - loss 8.90701234 - samples/sec: 134.67 - lr: 0.300000 +2021-03-26 05:24:39,248 epoch 14 - iter 14/25 - loss 8.99114633 - samples/sec: 128.19 - lr: 0.300000 +2021-03-26 05:24:40,225 epoch 14 - iter 16/25 - loss 9.13066396 - samples/sec: 131.33 - lr: 0.300000 +2021-03-26 05:24:41,134 epoch 14 - iter 18/25 - loss 9.10450154 - samples/sec: 140.92 - lr: 0.300000 +2021-03-26 05:24:42,083 epoch 14 - iter 20/25 - loss 9.14051502 - samples/sec: 135.24 - lr: 0.300000 +2021-03-26 05:24:43,230 epoch 14 - iter 22/25 - loss 9.12343365 - samples/sec: 111.78 - lr: 0.300000 +2021-03-26 05:24:44,225 epoch 14 - iter 24/25 - loss 9.09766744 - samples/sec: 128.73 - lr: 0.300000 +2021-03-26 05:24:44,585 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:24:44,586 EPOCH 14 done: loss 9.0100 - lr 0.3000000 +2021-03-26 05:24:45,383 DEV : loss 7.8762006759643555 - score 0.8742 +2021-03-26 05:24:45,408 BAD EPOCHS (no improvement): 0 +2021-03-26 05:24:55,253 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:24:56,534 epoch 15 - iter 2/25 - loss 7.36967874 - samples/sec: 100.07 - lr: 0.300000 +2021-03-26 05:24:57,584 epoch 15 - iter 4/25 - loss 8.21823275 - samples/sec: 122.07 - lr: 0.300000 +2021-03-26 05:24:58,702 epoch 15 - iter 6/25 - loss 8.26005371 - samples/sec: 114.61 - lr: 0.300000 +2021-03-26 05:24:59,681 epoch 15 - iter 8/25 - loss 8.40027934 - samples/sec: 131.16 - lr: 0.300000 +2021-03-26 05:25:00,589 epoch 15 - iter 10/25 - loss 8.38377128 - samples/sec: 141.26 - lr: 0.300000 +2021-03-26 05:25:01,554 epoch 15 - iter 12/25 - loss 8.43352679 - samples/sec: 132.81 - lr: 0.300000 +2021-03-26 05:25:02,543 epoch 15 - iter 14/25 - loss 8.60027528 - samples/sec: 129.54 - lr: 0.300000 +2021-03-26 05:25:03,456 epoch 15 - iter 16/25 - loss 8.50458527 - samples/sec: 140.39 - lr: 0.300000 +2021-03-26 05:25:04,441 epoch 15 - iter 18/25 - loss 8.59310924 - samples/sec: 130.13 - lr: 0.300000 +2021-03-26 05:25:05,374 epoch 15 - iter 20/25 - loss 8.45520668 - samples/sec: 137.44 - lr: 0.300000 +2021-03-26 05:25:06,528 epoch 15 - iter 22/25 - loss 8.46085405 - samples/sec: 111.06 - lr: 0.300000 +2021-03-26 05:25:07,471 epoch 15 - iter 24/25 - loss 8.52216919 - samples/sec: 135.86 - lr: 0.300000 +2021-03-26 05:25:07,959 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:25:07,960 EPOCH 15 done: loss 8.5789 - lr 0.3000000 +2021-03-26 05:25:08,749 DEV : loss 7.669604301452637 - score 0.8778 +2021-03-26 05:25:08,767 BAD EPOCHS (no improvement): 0 +2021-03-26 05:25:18,338 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:25:19,395 epoch 16 - iter 2/25 - loss 8.13550901 - samples/sec: 121.28 - lr: 0.300000 +2021-03-26 05:25:20,334 epoch 16 - iter 4/25 - loss 7.03732085 - samples/sec: 136.58 - lr: 0.300000 +2021-03-26 05:25:21,549 epoch 16 - iter 6/25 - loss 7.44733779 - samples/sec: 105.45 - lr: 0.300000 +2021-03-26 05:25:22,598 epoch 16 - iter 8/25 - loss 7.37813056 - samples/sec: 122.42 - lr: 0.300000 +2021-03-26 05:25:23,678 epoch 16 - iter 10/25 - loss 7.22389278 - samples/sec: 118.64 - lr: 0.300000 +2021-03-26 05:25:24,947 epoch 16 - iter 12/25 - loss 7.47639052 - samples/sec: 100.94 - lr: 0.300000 +2021-03-26 05:25:26,035 epoch 16 - iter 14/25 - loss 7.56412601 - samples/sec: 117.79 - lr: 0.300000 +2021-03-26 05:25:27,062 epoch 16 - iter 16/25 - loss 7.49645185 - samples/sec: 125.03 - lr: 0.300000 +2021-03-26 05:25:27,988 epoch 16 - iter 18/25 - loss 7.50306392 - samples/sec: 138.30 - lr: 0.300000 +2021-03-26 05:25:28,967 epoch 16 - iter 20/25 - loss 7.53772652 - samples/sec: 131.12 - lr: 0.300000 +2021-03-26 05:25:29,986 epoch 16 - iter 22/25 - loss 7.72518433 - samples/sec: 125.90 - lr: 0.300000 +2021-03-26 05:25:31,108 epoch 16 - iter 24/25 - loss 7.87033107 - samples/sec: 114.28 - lr: 0.300000 +2021-03-26 05:25:31,507 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:25:31,509 EPOCH 16 done: loss 7.9388 - lr 0.3000000 +2021-03-26 05:25:32,308 DEV : loss 7.1785101890563965 - score 0.8821 +2021-03-26 05:25:32,334 BAD EPOCHS (no improvement): 0 +2021-03-26 05:25:41,929 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:25:42,993 epoch 17 - iter 2/25 - loss 7.94741535 - samples/sec: 120.59 - lr: 0.300000 +2021-03-26 05:25:44,103 epoch 17 - iter 4/25 - loss 7.50709152 - samples/sec: 115.58 - lr: 0.300000 +2021-03-26 05:25:45,093 epoch 17 - iter 6/25 - loss 7.20844897 - samples/sec: 129.49 - lr: 0.300000 +2021-03-26 05:25:45,997 epoch 17 - iter 8/25 - loss 7.34367311 - samples/sec: 141.68 - lr: 0.300000 +2021-03-26 05:25:46,963 epoch 17 - iter 10/25 - loss 7.38157673 - samples/sec: 132.71 - lr: 0.300000 +2021-03-26 05:25:48,126 epoch 17 - iter 12/25 - loss 7.26675844 - samples/sec: 110.15 - lr: 0.300000 +2021-03-26 05:25:49,145 epoch 17 - iter 14/25 - loss 7.44016184 - samples/sec: 125.87 - lr: 0.300000 +2021-03-26 05:25:50,131 epoch 17 - iter 16/25 - loss 7.60327804 - samples/sec: 130.01 - lr: 0.300000 +2021-03-26 05:25:51,126 epoch 17 - iter 18/25 - loss 7.59039590 - samples/sec: 128.86 - lr: 0.300000 +2021-03-26 05:25:52,082 epoch 17 - iter 20/25 - loss 7.60469155 - samples/sec: 134.06 - lr: 0.300000 +2021-03-26 05:25:53,058 epoch 17 - iter 22/25 - loss 7.59717865 - samples/sec: 131.42 - lr: 0.300000 +2021-03-26 05:25:54,143 epoch 17 - iter 24/25 - loss 7.81563658 - samples/sec: 118.16 - lr: 0.300000 +2021-03-26 05:25:54,547 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:25:54,548 EPOCH 17 done: loss 7.8554 - lr 0.3000000 +2021-03-26 05:25:55,364 DEV : loss 7.210777282714844 - score 0.8833 +2021-03-26 05:25:55,401 BAD EPOCHS (no improvement): 0 +2021-03-26 05:26:05,034 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:26:06,147 epoch 18 - iter 2/25 - loss 6.63195515 - samples/sec: 115.19 - lr: 0.300000 +2021-03-26 05:26:07,231 epoch 18 - iter 4/25 - loss 7.14458799 - samples/sec: 118.29 - lr: 0.300000 +2021-03-26 05:26:08,413 epoch 18 - iter 6/25 - loss 7.36422459 - samples/sec: 108.38 - lr: 0.300000 +2021-03-26 05:26:09,472 epoch 18 - iter 8/25 - loss 7.26226878 - samples/sec: 121.10 - lr: 0.300000 +2021-03-26 05:26:10,477 epoch 18 - iter 10/25 - loss 7.17983127 - samples/sec: 127.57 - lr: 0.300000 +2021-03-26 05:26:11,527 epoch 18 - iter 12/25 - loss 7.28797615 - samples/sec: 121.98 - lr: 0.300000 +2021-03-26 05:26:12,576 epoch 18 - iter 14/25 - loss 7.35998808 - samples/sec: 122.43 - lr: 0.300000 +2021-03-26 05:26:13,580 epoch 18 - iter 16/25 - loss 7.37090909 - samples/sec: 127.61 - lr: 0.300000 +2021-03-26 05:26:14,574 epoch 18 - iter 18/25 - loss 7.48112631 - samples/sec: 128.90 - lr: 0.300000 +2021-03-26 05:26:15,576 epoch 18 - iter 20/25 - loss 7.40971677 - samples/sec: 127.94 - lr: 0.300000 +2021-03-26 05:26:16,544 epoch 18 - iter 22/25 - loss 7.27288305 - samples/sec: 132.49 - lr: 0.300000 +2021-03-26 05:26:17,487 epoch 18 - iter 24/25 - loss 7.28376927 - samples/sec: 135.93 - lr: 0.300000 +2021-03-26 05:26:17,878 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:26:17,878 EPOCH 18 done: loss 7.3653 - lr 0.3000000 +2021-03-26 05:26:18,660 DEV : loss 7.1419501304626465 - score 0.8825 +2021-03-26 05:26:18,681 BAD EPOCHS (no improvement): 1 +2021-03-26 05:26:18,682 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:26:19,665 epoch 19 - iter 2/25 - loss 6.25295830 - samples/sec: 130.48 - lr: 0.300000 +2021-03-26 05:26:20,672 epoch 19 - iter 4/25 - loss 6.57761550 - samples/sec: 127.28 - lr: 0.300000 +2021-03-26 05:26:21,720 epoch 19 - iter 6/25 - loss 6.53581572 - samples/sec: 122.31 - lr: 0.300000 +2021-03-26 05:26:22,894 epoch 19 - iter 8/25 - loss 6.69568771 - samples/sec: 109.14 - lr: 0.300000 +2021-03-26 05:26:24,062 epoch 19 - iter 10/25 - loss 6.88163967 - samples/sec: 109.81 - lr: 0.300000 +2021-03-26 05:26:25,069 epoch 19 - iter 12/25 - loss 7.00405916 - samples/sec: 127.20 - lr: 0.300000 +2021-03-26 05:26:26,000 epoch 19 - iter 14/25 - loss 7.03675880 - samples/sec: 137.68 - lr: 0.300000 +2021-03-26 05:26:26,956 epoch 19 - iter 16/25 - loss 7.00491729 - samples/sec: 134.10 - lr: 0.300000 +2021-03-26 05:26:27,982 epoch 19 - iter 18/25 - loss 6.89046571 - samples/sec: 124.92 - lr: 0.300000 +2021-03-26 05:26:28,945 epoch 19 - iter 20/25 - loss 6.84953601 - samples/sec: 133.14 - lr: 0.300000 +2021-03-26 05:26:29,932 epoch 19 - iter 22/25 - loss 6.87707803 - samples/sec: 130.02 - lr: 0.300000 +2021-03-26 05:26:30,914 epoch 19 - iter 24/25 - loss 6.94888598 - samples/sec: 130.60 - lr: 0.300000 +2021-03-26 05:26:31,419 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:26:31,420 EPOCH 19 done: loss 6.9400 - lr 0.3000000 +2021-03-26 05:26:32,208 DEV : loss 7.064170837402344 - score 0.8805 +2021-03-26 05:26:32,234 BAD EPOCHS (no improvement): 2 +2021-03-26 05:26:32,235 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:26:33,210 epoch 20 - iter 2/25 - loss 6.28246975 - samples/sec: 131.61 - lr: 0.300000 +2021-03-26 05:26:34,385 epoch 20 - iter 4/25 - loss 6.50505972 - samples/sec: 109.18 - lr: 0.300000 +2021-03-26 05:26:35,386 epoch 20 - iter 6/25 - loss 6.39607708 - samples/sec: 128.61 - lr: 0.300000 +2021-03-26 05:26:36,443 epoch 20 - iter 8/25 - loss 6.44121069 - samples/sec: 121.30 - lr: 0.300000 +2021-03-26 05:26:37,390 epoch 20 - iter 10/25 - loss 6.51437483 - samples/sec: 135.54 - lr: 0.300000 +2021-03-26 05:26:38,382 epoch 20 - iter 12/25 - loss 6.48822912 - samples/sec: 129.24 - lr: 0.300000 +2021-03-26 05:26:39,321 epoch 20 - iter 14/25 - loss 6.52995566 - samples/sec: 136.39 - lr: 0.300000 +2021-03-26 05:26:40,374 epoch 20 - iter 16/25 - loss 6.48401153 - samples/sec: 121.82 - lr: 0.300000 +2021-03-26 05:26:41,452 epoch 20 - iter 18/25 - loss 6.68182256 - samples/sec: 118.89 - lr: 0.300000 +2021-03-26 05:26:42,403 epoch 20 - iter 20/25 - loss 6.63910098 - samples/sec: 134.85 - lr: 0.300000 +2021-03-26 05:26:43,378 epoch 20 - iter 22/25 - loss 6.61011570 - samples/sec: 131.52 - lr: 0.300000 +2021-03-26 05:26:44,399 epoch 20 - iter 24/25 - loss 6.74762255 - samples/sec: 125.65 - lr: 0.300000 +2021-03-26 05:26:44,788 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:26:44,788 EPOCH 20 done: loss 6.7392 - lr 0.3000000 +2021-03-26 05:26:45,587 DEV : loss 7.2709784507751465 - score 0.878 +2021-03-26 05:26:45,613 BAD EPOCHS (no improvement): 3 +2021-03-26 05:26:45,613 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:26:46,601 epoch 21 - iter 2/25 - loss 6.23770928 - samples/sec: 129.85 - lr: 0.300000 +2021-03-26 05:26:47,602 epoch 21 - iter 4/25 - loss 6.37881196 - samples/sec: 128.15 - lr: 0.300000 +2021-03-26 05:26:48,649 epoch 21 - iter 6/25 - loss 6.54572686 - samples/sec: 122.42 - lr: 0.300000 +2021-03-26 05:26:49,646 epoch 21 - iter 8/25 - loss 6.36524707 - samples/sec: 128.57 - lr: 0.300000 +2021-03-26 05:26:50,621 epoch 21 - iter 10/25 - loss 6.47547174 - samples/sec: 131.58 - lr: 0.300000 +2021-03-26 05:26:51,622 epoch 21 - iter 12/25 - loss 6.50450758 - samples/sec: 128.00 - lr: 0.300000 +2021-03-26 05:26:52,728 epoch 21 - iter 14/25 - loss 6.59004596 - samples/sec: 115.91 - lr: 0.300000 +2021-03-26 05:26:53,747 epoch 21 - iter 16/25 - loss 6.56259722 - samples/sec: 125.80 - lr: 0.300000 +2021-03-26 05:26:54,718 epoch 21 - iter 18/25 - loss 6.59617737 - samples/sec: 131.99 - lr: 0.300000 +2021-03-26 05:26:55,717 epoch 21 - iter 20/25 - loss 6.62009044 - samples/sec: 128.23 - lr: 0.300000 +2021-03-26 05:26:56,831 epoch 21 - iter 22/25 - loss 6.60993984 - samples/sec: 115.11 - lr: 0.300000 +2021-03-26 05:26:57,793 epoch 21 - iter 24/25 - loss 6.62797530 - samples/sec: 133.20 - lr: 0.300000 +2021-03-26 05:26:58,192 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:26:58,193 EPOCH 21 done: loss 6.5851 - lr 0.3000000 +2021-03-26 05:26:59,025 DEV : loss 6.789755821228027 - score 0.8945 +2021-03-26 05:26:59,051 BAD EPOCHS (no improvement): 0 +2021-03-26 05:27:08,721 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:27:09,779 epoch 22 - iter 2/25 - loss 5.53058386 - samples/sec: 121.32 - lr: 0.300000 +2021-03-26 05:27:10,809 epoch 22 - iter 4/25 - loss 5.39357162 - samples/sec: 124.36 - lr: 0.300000 +2021-03-26 05:27:11,833 epoch 22 - iter 6/25 - loss 5.63118982 - samples/sec: 125.28 - lr: 0.300000 +2021-03-26 05:27:12,780 epoch 22 - iter 8/25 - loss 6.02222788 - samples/sec: 135.52 - lr: 0.300000 +2021-03-26 05:27:13,839 epoch 22 - iter 10/25 - loss 6.07552347 - samples/sec: 121.04 - lr: 0.300000 +2021-03-26 05:27:14,765 epoch 22 - iter 12/25 - loss 6.22429490 - samples/sec: 138.41 - lr: 0.300000 +2021-03-26 05:27:15,759 epoch 22 - iter 14/25 - loss 6.22745047 - samples/sec: 129.08 - lr: 0.300000 +2021-03-26 05:27:16,689 epoch 22 - iter 16/25 - loss 6.30747709 - samples/sec: 137.81 - lr: 0.300000 +2021-03-26 05:27:17,708 epoch 22 - iter 18/25 - loss 6.26303742 - samples/sec: 125.81 - lr: 0.300000 +2021-03-26 05:27:18,708 epoch 22 - iter 20/25 - loss 6.22910388 - samples/sec: 128.20 - lr: 0.300000 +2021-03-26 05:27:19,780 epoch 22 - iter 22/25 - loss 6.24298755 - samples/sec: 119.67 - lr: 0.300000 +2021-03-26 05:27:20,741 epoch 22 - iter 24/25 - loss 6.24154566 - samples/sec: 133.33 - lr: 0.300000 +2021-03-26 05:27:21,120 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:27:21,121 EPOCH 22 done: loss 6.2946 - lr 0.3000000 +2021-03-26 05:27:21,895 DEV : loss 6.656912803649902 - score 0.8945 +2021-03-26 05:27:21,921 BAD EPOCHS (no improvement): 0 +2021-03-26 05:27:31,685 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:27:32,736 epoch 23 - iter 2/25 - loss 4.46555138 - samples/sec: 122.08 - lr: 0.300000 +2021-03-26 05:27:33,969 epoch 23 - iter 4/25 - loss 4.78493762 - samples/sec: 103.93 - lr: 0.300000 +2021-03-26 05:27:35,147 epoch 23 - iter 6/25 - loss 5.48626884 - samples/sec: 108.79 - lr: 0.300000 +2021-03-26 05:27:36,228 epoch 23 - iter 8/25 - loss 5.68426472 - samples/sec: 118.54 - lr: 0.300000 +2021-03-26 05:27:37,297 epoch 23 - iter 10/25 - loss 5.91127291 - samples/sec: 119.97 - lr: 0.300000 +2021-03-26 05:27:38,310 epoch 23 - iter 12/25 - loss 6.00124419 - samples/sec: 126.64 - lr: 0.300000 +2021-03-26 05:27:39,237 epoch 23 - iter 14/25 - loss 5.97189999 - samples/sec: 138.59 - lr: 0.300000 +2021-03-26 05:27:40,249 epoch 23 - iter 16/25 - loss 6.07553238 - samples/sec: 126.71 - lr: 0.300000 +2021-03-26 05:27:41,284 epoch 23 - iter 18/25 - loss 6.01765153 - samples/sec: 123.81 - lr: 0.300000 +2021-03-26 05:27:42,351 epoch 23 - iter 20/25 - loss 6.02717052 - samples/sec: 120.24 - lr: 0.300000 +2021-03-26 05:27:43,362 epoch 23 - iter 22/25 - loss 5.97675254 - samples/sec: 126.76 - lr: 0.300000 +2021-03-26 05:27:44,345 epoch 23 - iter 24/25 - loss 6.03377473 - samples/sec: 130.46 - lr: 0.300000 +2021-03-26 05:27:44,802 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:27:44,804 EPOCH 23 done: loss 6.0530 - lr 0.3000000 +2021-03-26 05:27:45,614 DEV : loss 6.680357933044434 - score 0.8937 +2021-03-26 05:27:45,644 BAD EPOCHS (no improvement): 1 +2021-03-26 05:27:45,645 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:27:47,974 epoch 24 - iter 2/25 - loss 6.29046893 - samples/sec: 55.00 - lr: 0.300000 +2021-03-26 05:27:48,979 epoch 24 - iter 4/25 - loss 5.58529961 - samples/sec: 127.72 - lr: 0.300000 +2021-03-26 05:27:49,976 epoch 24 - iter 6/25 - loss 5.39165346 - samples/sec: 128.73 - lr: 0.300000 +2021-03-26 05:27:50,908 epoch 24 - iter 8/25 - loss 5.54855865 - samples/sec: 137.57 - lr: 0.300000 +2021-03-26 05:27:51,986 epoch 24 - iter 10/25 - loss 5.66706057 - samples/sec: 118.97 - lr: 0.300000 +2021-03-26 05:27:52,966 epoch 24 - iter 12/25 - loss 5.52589273 - samples/sec: 130.86 - lr: 0.300000 +2021-03-26 05:27:53,939 epoch 24 - iter 14/25 - loss 5.49458109 - samples/sec: 131.71 - lr: 0.300000 +2021-03-26 05:27:54,993 epoch 24 - iter 16/25 - loss 5.55637434 - samples/sec: 121.63 - lr: 0.300000 +2021-03-26 05:27:56,061 epoch 24 - iter 18/25 - loss 5.59738755 - samples/sec: 120.01 - lr: 0.300000 +2021-03-26 05:27:57,045 epoch 24 - iter 20/25 - loss 5.65361750 - samples/sec: 130.29 - lr: 0.300000 +2021-03-26 05:27:58,039 epoch 24 - iter 22/25 - loss 5.59108576 - samples/sec: 128.97 - lr: 0.300000 +2021-03-26 05:27:59,097 epoch 24 - iter 24/25 - loss 5.64387566 - samples/sec: 121.36 - lr: 0.300000 +2021-03-26 05:27:59,523 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:27:59,525 EPOCH 24 done: loss 5.6586 - lr 0.3000000 +2021-03-26 05:28:00,334 DEV : loss 6.7325544357299805 - score 0.8902 +2021-03-26 05:28:00,359 BAD EPOCHS (no improvement): 2 +2021-03-26 05:28:00,360 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:28:01,283 epoch 25 - iter 2/25 - loss 5.46722913 - samples/sec: 138.87 - lr: 0.300000 +2021-03-26 05:28:02,392 epoch 25 - iter 4/25 - loss 5.36274683 - samples/sec: 115.62 - lr: 0.300000 +2021-03-26 05:28:03,433 epoch 25 - iter 6/25 - loss 5.46451076 - samples/sec: 123.11 - lr: 0.300000 +2021-03-26 05:28:04,419 epoch 25 - iter 8/25 - loss 5.23664898 - samples/sec: 129.97 - lr: 0.300000 +2021-03-26 05:28:05,482 epoch 25 - iter 10/25 - loss 5.23238530 - samples/sec: 120.68 - lr: 0.300000 +2021-03-26 05:28:06,497 epoch 25 - iter 12/25 - loss 5.37240080 - samples/sec: 126.34 - lr: 0.300000 +2021-03-26 05:28:07,519 epoch 25 - iter 14/25 - loss 5.52948417 - samples/sec: 125.40 - lr: 0.300000 +2021-03-26 05:28:08,558 epoch 25 - iter 16/25 - loss 5.39657971 - samples/sec: 123.42 - lr: 0.300000 +2021-03-26 05:28:09,553 epoch 25 - iter 18/25 - loss 5.50258454 - samples/sec: 128.79 - lr: 0.300000 +2021-03-26 05:28:10,536 epoch 25 - iter 20/25 - loss 5.53387783 - samples/sec: 130.55 - lr: 0.300000 +2021-03-26 05:28:11,567 epoch 25 - iter 22/25 - loss 5.66683754 - samples/sec: 124.40 - lr: 0.300000 +2021-03-26 05:28:12,579 epoch 25 - iter 24/25 - loss 5.67239670 - samples/sec: 126.73 - lr: 0.300000 +2021-03-26 05:28:13,003 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:28:13,004 EPOCH 25 done: loss 5.6747 - lr 0.3000000 +2021-03-26 05:28:13,806 DEV : loss 6.763164043426514 - score 0.8958 +2021-03-26 05:28:13,832 BAD EPOCHS (no improvement): 0 +2021-03-26 05:28:23,462 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:28:24,555 epoch 26 - iter 2/25 - loss 6.63353133 - samples/sec: 117.27 - lr: 0.300000 +2021-03-26 05:28:25,475 epoch 26 - iter 4/25 - loss 5.68939912 - samples/sec: 139.37 - lr: 0.300000 +2021-03-26 05:28:26,410 epoch 26 - iter 6/25 - loss 5.65168484 - samples/sec: 137.19 - lr: 0.300000 +2021-03-26 05:28:27,395 epoch 26 - iter 8/25 - loss 5.62042195 - samples/sec: 130.19 - lr: 0.300000 +2021-03-26 05:28:28,351 epoch 26 - iter 10/25 - loss 5.50891285 - samples/sec: 134.16 - lr: 0.300000 +2021-03-26 05:28:29,378 epoch 26 - iter 12/25 - loss 5.33578368 - samples/sec: 124.93 - lr: 0.300000 +2021-03-26 05:28:30,697 epoch 26 - iter 14/25 - loss 5.28669374 - samples/sec: 97.14 - lr: 0.300000 +2021-03-26 05:28:31,976 epoch 26 - iter 16/25 - loss 5.39646566 - samples/sec: 100.15 - lr: 0.300000 +2021-03-26 05:28:33,171 epoch 26 - iter 18/25 - loss 5.44764969 - samples/sec: 107.27 - lr: 0.300000 +2021-03-26 05:28:34,436 epoch 26 - iter 20/25 - loss 5.51364100 - samples/sec: 101.31 - lr: 0.300000 +2021-03-26 05:28:35,782 epoch 26 - iter 22/25 - loss 5.52997396 - samples/sec: 95.16 - lr: 0.300000 +2021-03-26 05:28:36,856 epoch 26 - iter 24/25 - loss 5.52822081 - samples/sec: 119.42 - lr: 0.300000 +2021-03-26 05:28:37,296 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:28:37,297 EPOCH 26 done: loss 5.5572 - lr 0.3000000 +2021-03-26 05:28:38,110 DEV : loss 6.5847883224487305 - score 0.9026 +2021-03-26 05:28:38,136 BAD EPOCHS (no improvement): 0 +2021-03-26 05:28:47,954 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:28:48,908 epoch 27 - iter 2/25 - loss 4.54578447 - samples/sec: 134.48 - lr: 0.300000 +2021-03-26 05:28:49,852 epoch 27 - iter 4/25 - loss 5.14534843 - samples/sec: 135.88 - lr: 0.300000 +2021-03-26 05:28:50,817 epoch 27 - iter 6/25 - loss 5.24506561 - samples/sec: 132.91 - lr: 0.300000 +2021-03-26 05:28:51,775 epoch 27 - iter 8/25 - loss 5.30547184 - samples/sec: 133.91 - lr: 0.300000 +2021-03-26 05:28:52,740 epoch 27 - iter 10/25 - loss 5.18562655 - samples/sec: 132.79 - lr: 0.300000 +2021-03-26 05:28:53,790 epoch 27 - iter 12/25 - loss 5.12651292 - samples/sec: 122.02 - lr: 0.300000 +2021-03-26 05:28:54,822 epoch 27 - iter 14/25 - loss 5.12403495 - samples/sec: 124.34 - lr: 0.300000 +2021-03-26 05:28:55,773 epoch 27 - iter 16/25 - loss 5.09699774 - samples/sec: 134.84 - lr: 0.300000 +2021-03-26 05:28:56,885 epoch 27 - iter 18/25 - loss 5.10937105 - samples/sec: 115.29 - lr: 0.300000 +2021-03-26 05:28:57,943 epoch 27 - iter 20/25 - loss 5.11198812 - samples/sec: 121.82 - lr: 0.300000 +2021-03-26 05:28:58,995 epoch 27 - iter 22/25 - loss 5.11410750 - samples/sec: 121.95 - lr: 0.300000 +2021-03-26 05:29:00,035 epoch 27 - iter 24/25 - loss 5.11160312 - samples/sec: 123.37 - lr: 0.300000 +2021-03-26 05:29:00,435 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:29:00,436 EPOCH 27 done: loss 5.1355 - lr 0.3000000 +2021-03-26 05:29:01,239 DEV : loss 6.560229301452637 - score 0.8943 +2021-03-26 05:29:01,264 BAD EPOCHS (no improvement): 1 +2021-03-26 05:29:01,265 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:29:02,186 epoch 28 - iter 2/25 - loss 4.66952753 - samples/sec: 139.12 - lr: 0.300000 +2021-03-26 05:29:03,296 epoch 28 - iter 4/25 - loss 4.94452524 - samples/sec: 115.61 - lr: 0.300000 +2021-03-26 05:29:04,307 epoch 28 - iter 6/25 - loss 5.28915143 - samples/sec: 126.82 - lr: 0.300000 +2021-03-26 05:29:05,387 epoch 28 - iter 8/25 - loss 5.23088485 - samples/sec: 118.70 - lr: 0.300000 +2021-03-26 05:29:06,361 epoch 28 - iter 10/25 - loss 5.18750887 - samples/sec: 132.29 - lr: 0.300000 +2021-03-26 05:29:07,301 epoch 28 - iter 12/25 - loss 5.27375034 - samples/sec: 136.41 - lr: 0.300000 +2021-03-26 05:29:08,326 epoch 28 - iter 14/25 - loss 5.17865321 - samples/sec: 125.17 - lr: 0.300000 +2021-03-26 05:29:09,257 epoch 28 - iter 16/25 - loss 5.02847534 - samples/sec: 137.74 - lr: 0.300000 +2021-03-26 05:29:10,228 epoch 28 - iter 18/25 - loss 4.98988385 - samples/sec: 132.00 - lr: 0.300000 +2021-03-26 05:29:11,255 epoch 28 - iter 20/25 - loss 5.05867081 - samples/sec: 124.96 - lr: 0.300000 +2021-03-26 05:29:12,241 epoch 28 - iter 22/25 - loss 5.07502324 - samples/sec: 129.99 - lr: 0.300000 +2021-03-26 05:29:13,203 epoch 28 - iter 24/25 - loss 5.07763763 - samples/sec: 133.29 - lr: 0.300000 +2021-03-26 05:29:13,619 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:29:13,620 EPOCH 28 done: loss 5.0847 - lr 0.3000000 +2021-03-26 05:29:14,425 DEV : loss 6.717233180999756 - score 0.8993 +2021-03-26 05:29:14,451 BAD EPOCHS (no improvement): 2 +2021-03-26 05:29:14,452 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:29:15,416 epoch 29 - iter 2/25 - loss 4.26012993 - samples/sec: 132.99 - lr: 0.300000 +2021-03-26 05:29:16,349 epoch 29 - iter 4/25 - loss 4.56135571 - samples/sec: 137.37 - lr: 0.300000 +2021-03-26 05:29:17,389 epoch 29 - iter 6/25 - loss 4.93187809 - samples/sec: 123.23 - lr: 0.300000 +2021-03-26 05:29:18,366 epoch 29 - iter 8/25 - loss 4.83429003 - samples/sec: 131.28 - lr: 0.300000 +2021-03-26 05:29:19,296 epoch 29 - iter 10/25 - loss 4.67303157 - samples/sec: 137.83 - lr: 0.300000 +2021-03-26 05:29:20,392 epoch 29 - iter 12/25 - loss 4.82254970 - samples/sec: 116.98 - lr: 0.300000 +2021-03-26 05:29:21,387 epoch 29 - iter 14/25 - loss 4.81970096 - samples/sec: 128.74 - lr: 0.300000 +2021-03-26 05:29:22,338 epoch 29 - iter 16/25 - loss 4.91178760 - samples/sec: 134.92 - lr: 0.300000 +2021-03-26 05:29:23,320 epoch 29 - iter 18/25 - loss 4.96595081 - samples/sec: 130.51 - lr: 0.300000 +2021-03-26 05:29:24,274 epoch 29 - iter 20/25 - loss 4.99328244 - samples/sec: 134.36 - lr: 0.300000 +2021-03-26 05:29:25,226 epoch 29 - iter 22/25 - loss 4.95126521 - samples/sec: 134.75 - lr: 0.300000 +2021-03-26 05:29:26,292 epoch 29 - iter 24/25 - loss 5.03944759 - samples/sec: 120.22 - lr: 0.300000 +2021-03-26 05:29:26,696 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:29:26,697 EPOCH 29 done: loss 5.0159 - lr 0.3000000 +2021-03-26 05:29:27,485 DEV : loss 6.611323356628418 - score 0.8941 +2021-03-26 05:29:27,510 BAD EPOCHS (no improvement): 3 +2021-03-26 05:29:27,511 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:29:28,445 epoch 30 - iter 2/25 - loss 4.52283883 - samples/sec: 137.36 - lr: 0.300000 +2021-03-26 05:29:29,467 epoch 30 - iter 4/25 - loss 4.83641267 - samples/sec: 125.41 - lr: 0.300000 +2021-03-26 05:29:30,417 epoch 30 - iter 6/25 - loss 4.50342270 - samples/sec: 135.10 - lr: 0.300000 +2021-03-26 05:29:31,465 epoch 30 - iter 8/25 - loss 4.70275000 - samples/sec: 122.34 - lr: 0.300000 +2021-03-26 05:29:32,435 epoch 30 - iter 10/25 - loss 4.65241845 - samples/sec: 132.29 - lr: 0.300000 +2021-03-26 05:29:33,378 epoch 30 - iter 12/25 - loss 4.55542413 - samples/sec: 135.97 - lr: 0.300000 +2021-03-26 05:29:34,269 epoch 30 - iter 14/25 - loss 4.53860392 - samples/sec: 143.94 - lr: 0.300000 +2021-03-26 05:29:35,282 epoch 30 - iter 16/25 - loss 4.59294629 - samples/sec: 126.47 - lr: 0.300000 +2021-03-26 05:29:36,315 epoch 30 - iter 18/25 - loss 4.61952384 - samples/sec: 124.11 - lr: 0.300000 +2021-03-26 05:29:37,454 epoch 30 - iter 20/25 - loss 4.71150489 - samples/sec: 112.55 - lr: 0.300000 +2021-03-26 05:29:38,557 epoch 30 - iter 22/25 - loss 4.75680065 - samples/sec: 116.20 - lr: 0.300000 +2021-03-26 05:29:39,645 epoch 30 - iter 24/25 - loss 4.75982225 - samples/sec: 117.87 - lr: 0.300000 +2021-03-26 05:29:40,043 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:29:40,045 EPOCH 30 done: loss 4.7312 - lr 0.3000000 +2021-03-26 05:29:40,842 DEV : loss 6.537320613861084 - score 0.8995 +2021-03-26 05:29:40,863 BAD EPOCHS (no improvement): 4 +2021-03-26 05:29:40,864 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:29:41,845 epoch 31 - iter 2/25 - loss 4.45332122 - samples/sec: 130.55 - lr: 0.150000 +2021-03-26 05:29:42,845 epoch 31 - iter 4/25 - loss 4.77456248 - samples/sec: 128.23 - lr: 0.150000 +2021-03-26 05:29:43,911 epoch 31 - iter 6/25 - loss 4.98453426 - samples/sec: 120.27 - lr: 0.150000 +2021-03-26 05:29:44,926 epoch 31 - iter 8/25 - loss 4.82937908 - samples/sec: 126.26 - lr: 0.150000 +2021-03-26 05:29:46,072 epoch 31 - iter 10/25 - loss 4.81652617 - samples/sec: 111.93 - lr: 0.150000 +2021-03-26 05:29:47,143 epoch 31 - iter 12/25 - loss 4.75675062 - samples/sec: 119.70 - lr: 0.150000 +2021-03-26 05:29:48,116 epoch 31 - iter 14/25 - loss 4.63152540 - samples/sec: 131.80 - lr: 0.150000 +2021-03-26 05:29:49,085 epoch 31 - iter 16/25 - loss 4.68138213 - samples/sec: 132.39 - lr: 0.150000 +2021-03-26 05:29:50,143 epoch 31 - iter 18/25 - loss 4.68593766 - samples/sec: 121.12 - lr: 0.150000 +2021-03-26 05:29:51,145 epoch 31 - iter 20/25 - loss 4.57285674 - samples/sec: 127.93 - lr: 0.150000 +2021-03-26 05:29:52,061 epoch 31 - iter 22/25 - loss 4.52167766 - samples/sec: 140.15 - lr: 0.150000 +2021-03-26 05:29:53,091 epoch 31 - iter 24/25 - loss 4.50825019 - samples/sec: 124.33 - lr: 0.150000 +2021-03-26 05:29:53,517 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:29:53,518 EPOCH 31 done: loss 4.5345 - lr 0.1500000 +2021-03-26 05:29:54,316 DEV : loss 6.355429649353027 - score 0.9032 +2021-03-26 05:29:54,342 BAD EPOCHS (no improvement): 0 +2021-03-26 05:30:04,001 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:30:04,983 epoch 32 - iter 2/25 - loss 3.71819592 - samples/sec: 130.68 - lr: 0.150000 +2021-03-26 05:30:06,056 epoch 32 - iter 4/25 - loss 3.12705719 - samples/sec: 119.66 - lr: 0.150000 +2021-03-26 05:30:07,109 epoch 32 - iter 6/25 - loss 3.75584714 - samples/sec: 121.68 - lr: 0.150000 +2021-03-26 05:30:08,066 epoch 32 - iter 8/25 - loss 3.80353063 - samples/sec: 134.08 - lr: 0.150000 +2021-03-26 05:30:09,040 epoch 32 - iter 10/25 - loss 3.88620362 - samples/sec: 131.58 - lr: 0.150000 +2021-03-26 05:30:10,029 epoch 32 - iter 12/25 - loss 3.96990573 - samples/sec: 129.68 - lr: 0.150000 +2021-03-26 05:30:11,052 epoch 32 - iter 14/25 - loss 3.94366251 - samples/sec: 125.40 - lr: 0.150000 +2021-03-26 05:30:12,113 epoch 32 - iter 16/25 - loss 3.95921826 - samples/sec: 120.83 - lr: 0.150000 +2021-03-26 05:30:13,076 epoch 32 - iter 18/25 - loss 4.00487158 - samples/sec: 133.16 - lr: 0.150000 +2021-03-26 05:30:14,024 epoch 32 - iter 20/25 - loss 3.98131732 - samples/sec: 135.31 - lr: 0.150000 +2021-03-26 05:30:15,062 epoch 32 - iter 22/25 - loss 3.95869911 - samples/sec: 123.46 - lr: 0.150000 +2021-03-26 05:30:16,072 epoch 32 - iter 24/25 - loss 3.95320159 - samples/sec: 126.86 - lr: 0.150000 +2021-03-26 05:30:16,475 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:30:16,476 EPOCH 32 done: loss 4.0076 - lr 0.1500000 +2021-03-26 05:30:17,282 DEV : loss 6.234824180603027 - score 0.9098 +2021-03-26 05:30:17,308 BAD EPOCHS (no improvement): 0 +2021-03-26 05:30:27,096 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:30:28,289 epoch 33 - iter 2/25 - loss 3.86456370 - samples/sec: 107.51 - lr: 0.150000 +2021-03-26 05:30:29,477 epoch 33 - iter 4/25 - loss 3.84213394 - samples/sec: 107.94 - lr: 0.150000 +2021-03-26 05:30:30,473 epoch 33 - iter 6/25 - loss 3.88618358 - samples/sec: 128.64 - lr: 0.150000 +2021-03-26 05:30:31,506 epoch 33 - iter 8/25 - loss 3.77276155 - samples/sec: 124.09 - lr: 0.150000 +2021-03-26 05:30:32,470 epoch 33 - iter 10/25 - loss 3.84732611 - samples/sec: 132.98 - lr: 0.150000 +2021-03-26 05:30:33,478 epoch 33 - iter 12/25 - loss 3.89968775 - samples/sec: 127.17 - lr: 0.150000 +2021-03-26 05:30:34,514 epoch 33 - iter 14/25 - loss 3.83863291 - samples/sec: 123.71 - lr: 0.150000 +2021-03-26 05:30:35,475 epoch 33 - iter 16/25 - loss 3.93329898 - samples/sec: 133.42 - lr: 0.150000 +2021-03-26 05:30:36,449 epoch 33 - iter 18/25 - loss 3.88469691 - samples/sec: 131.60 - lr: 0.150000 +2021-03-26 05:30:37,630 epoch 33 - iter 20/25 - loss 3.87576485 - samples/sec: 108.50 - lr: 0.150000 +2021-03-26 05:30:39,099 epoch 33 - iter 22/25 - loss 3.88772985 - samples/sec: 87.29 - lr: 0.150000 +2021-03-26 05:30:40,419 epoch 33 - iter 24/25 - loss 3.87317472 - samples/sec: 97.19 - lr: 0.150000 +2021-03-26 05:30:40,976 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:30:40,977 EPOCH 33 done: loss 3.9395 - lr 0.1500000 +2021-03-26 05:30:41,789 DEV : loss 6.255607604980469 - score 0.9077 +2021-03-26 05:30:41,816 BAD EPOCHS (no improvement): 1 +2021-03-26 05:30:41,817 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:30:42,792 epoch 34 - iter 2/25 - loss 4.43583751 - samples/sec: 131.50 - lr: 0.150000 +2021-03-26 05:30:43,852 epoch 34 - iter 4/25 - loss 4.17272305 - samples/sec: 120.94 - lr: 0.150000 +2021-03-26 05:30:44,826 epoch 34 - iter 6/25 - loss 4.04421759 - samples/sec: 131.67 - lr: 0.150000 +2021-03-26 05:30:45,798 epoch 34 - iter 8/25 - loss 3.94364154 - samples/sec: 131.89 - lr: 0.150000 +2021-03-26 05:30:46,941 epoch 34 - iter 10/25 - loss 4.09087896 - samples/sec: 112.17 - lr: 0.150000 +2021-03-26 05:30:48,254 epoch 34 - iter 12/25 - loss 4.05837268 - samples/sec: 97.58 - lr: 0.150000 +2021-03-26 05:30:49,198 epoch 34 - iter 14/25 - loss 4.10885179 - samples/sec: 135.76 - lr: 0.150000 +2021-03-26 05:30:50,242 epoch 34 - iter 16/25 - loss 4.08000012 - samples/sec: 122.93 - lr: 0.150000 +2021-03-26 05:30:51,204 epoch 34 - iter 18/25 - loss 4.03305531 - samples/sec: 133.30 - lr: 0.150000 +2021-03-26 05:30:52,261 epoch 34 - iter 20/25 - loss 4.00350456 - samples/sec: 121.28 - lr: 0.150000 +2021-03-26 05:30:53,384 epoch 34 - iter 22/25 - loss 4.03260352 - samples/sec: 114.20 - lr: 0.150000 +2021-03-26 05:30:54,485 epoch 34 - iter 24/25 - loss 4.04323124 - samples/sec: 116.79 - lr: 0.150000 +2021-03-26 05:30:54,899 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:30:54,900 EPOCH 34 done: loss 4.0360 - lr 0.1500000 +2021-03-26 05:30:55,696 DEV : loss 6.293361186981201 - score 0.9056 +2021-03-26 05:30:55,723 BAD EPOCHS (no improvement): 2 +2021-03-26 05:30:55,723 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:30:56,785 epoch 35 - iter 2/25 - loss 4.15070677 - samples/sec: 120.74 - lr: 0.150000 +2021-03-26 05:30:57,731 epoch 35 - iter 4/25 - loss 3.72079283 - samples/sec: 135.67 - lr: 0.150000 +2021-03-26 05:30:58,683 epoch 35 - iter 6/25 - loss 3.54775671 - samples/sec: 134.58 - lr: 0.150000 +2021-03-26 05:30:59,699 epoch 35 - iter 8/25 - loss 3.53297698 - samples/sec: 126.23 - lr: 0.150000 +2021-03-26 05:31:00,601 epoch 35 - iter 10/25 - loss 3.63503337 - samples/sec: 142.13 - lr: 0.150000 +2021-03-26 05:31:01,469 epoch 35 - iter 12/25 - loss 3.80509043 - samples/sec: 147.60 - lr: 0.150000 +2021-03-26 05:31:02,457 epoch 35 - iter 14/25 - loss 3.84039981 - samples/sec: 129.73 - lr: 0.150000 +2021-03-26 05:31:03,372 epoch 35 - iter 16/25 - loss 3.75528948 - samples/sec: 140.06 - lr: 0.150000 +2021-03-26 05:31:04,357 epoch 35 - iter 18/25 - loss 3.78421675 - samples/sec: 130.23 - lr: 0.150000 +2021-03-26 05:31:05,294 epoch 35 - iter 20/25 - loss 3.84083500 - samples/sec: 136.68 - lr: 0.150000 +2021-03-26 05:31:06,327 epoch 35 - iter 22/25 - loss 3.80932782 - samples/sec: 124.13 - lr: 0.150000 +2021-03-26 05:31:07,391 epoch 35 - iter 24/25 - loss 3.83606430 - samples/sec: 120.46 - lr: 0.150000 +2021-03-26 05:31:07,829 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:31:07,830 EPOCH 35 done: loss 3.8055 - lr 0.1500000 +2021-03-26 05:31:08,645 DEV : loss 6.475340843200684 - score 0.9042 +2021-03-26 05:31:08,664 BAD EPOCHS (no improvement): 3 +2021-03-26 05:31:08,664 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:31:09,654 epoch 36 - iter 2/25 - loss 3.07113969 - samples/sec: 129.55 - lr: 0.150000 +2021-03-26 05:31:10,606 epoch 36 - iter 4/25 - loss 3.30959159 - samples/sec: 134.75 - lr: 0.150000 +2021-03-26 05:31:11,589 epoch 36 - iter 6/25 - loss 3.37420321 - samples/sec: 130.38 - lr: 0.150000 +2021-03-26 05:31:12,610 epoch 36 - iter 8/25 - loss 3.33646402 - samples/sec: 125.47 - lr: 0.150000 +2021-03-26 05:31:13,666 epoch 36 - iter 10/25 - loss 3.33366504 - samples/sec: 121.47 - lr: 0.150000 +2021-03-26 05:31:14,639 epoch 36 - iter 12/25 - loss 3.28806067 - samples/sec: 131.85 - lr: 0.150000 +2021-03-26 05:31:15,681 epoch 36 - iter 14/25 - loss 3.42399103 - samples/sec: 123.02 - lr: 0.150000 +2021-03-26 05:31:16,717 epoch 36 - iter 16/25 - loss 3.52277085 - samples/sec: 123.67 - lr: 0.150000 +2021-03-26 05:31:17,703 epoch 36 - iter 18/25 - loss 3.48878141 - samples/sec: 130.17 - lr: 0.150000 +2021-03-26 05:31:18,687 epoch 36 - iter 20/25 - loss 3.50925676 - samples/sec: 130.27 - lr: 0.150000 +2021-03-26 05:31:19,707 epoch 36 - iter 22/25 - loss 3.53277724 - samples/sec: 125.79 - lr: 0.150000 +2021-03-26 05:31:20,915 epoch 36 - iter 24/25 - loss 3.55849650 - samples/sec: 106.08 - lr: 0.150000 +2021-03-26 05:31:21,477 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:31:21,479 EPOCH 36 done: loss 3.6749 - lr 0.1500000 +2021-03-26 05:31:22,280 DEV : loss 6.371415615081787 - score 0.904 +2021-03-26 05:31:22,311 BAD EPOCHS (no improvement): 4 +2021-03-26 05:31:22,312 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:31:23,281 epoch 37 - iter 2/25 - loss 3.39781427 - samples/sec: 132.23 - lr: 0.075000 +2021-03-26 05:31:24,245 epoch 37 - iter 4/25 - loss 3.25910103 - samples/sec: 132.95 - lr: 0.075000 +2021-03-26 05:31:25,398 epoch 37 - iter 6/25 - loss 3.40158709 - samples/sec: 111.26 - lr: 0.075000 +2021-03-26 05:31:26,477 epoch 37 - iter 8/25 - loss 3.44064540 - samples/sec: 118.84 - lr: 0.075000 +2021-03-26 05:31:27,459 epoch 37 - iter 10/25 - loss 3.39384527 - samples/sec: 130.54 - lr: 0.075000 +2021-03-26 05:31:28,466 epoch 37 - iter 12/25 - loss 3.36764000 - samples/sec: 127.31 - lr: 0.075000 +2021-03-26 05:31:29,547 epoch 37 - iter 14/25 - loss 3.42956323 - samples/sec: 118.53 - lr: 0.075000 +2021-03-26 05:31:30,658 epoch 37 - iter 16/25 - loss 3.52977823 - samples/sec: 115.41 - lr: 0.075000 +2021-03-26 05:31:31,864 epoch 37 - iter 18/25 - loss 3.54157498 - samples/sec: 106.20 - lr: 0.075000 +2021-03-26 05:31:32,845 epoch 37 - iter 20/25 - loss 3.55722978 - samples/sec: 130.65 - lr: 0.075000 +2021-03-26 05:31:33,849 epoch 37 - iter 22/25 - loss 3.58427523 - samples/sec: 127.62 - lr: 0.075000 +2021-03-26 05:31:34,859 epoch 37 - iter 24/25 - loss 3.56133931 - samples/sec: 126.97 - lr: 0.075000 +2021-03-26 05:31:35,250 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:31:35,251 EPOCH 37 done: loss 3.5676 - lr 0.0750000 +2021-03-26 05:31:36,082 DEV : loss 6.338122367858887 - score 0.9034 +2021-03-26 05:31:36,112 BAD EPOCHS (no improvement): 1 +2021-03-26 05:31:36,113 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:31:37,043 epoch 38 - iter 2/25 - loss 3.31011808 - samples/sec: 137.87 - lr: 0.075000 +2021-03-26 05:31:38,021 epoch 38 - iter 4/25 - loss 3.29740214 - samples/sec: 131.14 - lr: 0.075000 +2021-03-26 05:31:39,028 epoch 38 - iter 6/25 - loss 3.31071031 - samples/sec: 127.24 - lr: 0.075000 +2021-03-26 05:31:40,026 epoch 38 - iter 8/25 - loss 3.45570368 - samples/sec: 128.49 - lr: 0.075000 +2021-03-26 05:31:41,035 epoch 38 - iter 10/25 - loss 3.50104599 - samples/sec: 126.94 - lr: 0.075000 +2021-03-26 05:31:41,963 epoch 38 - iter 12/25 - loss 3.52571265 - samples/sec: 138.43 - lr: 0.075000 +2021-03-26 05:31:42,920 epoch 38 - iter 14/25 - loss 3.42115290 - samples/sec: 133.98 - lr: 0.075000 +2021-03-26 05:31:43,892 epoch 38 - iter 16/25 - loss 3.41037132 - samples/sec: 131.84 - lr: 0.075000 +2021-03-26 05:31:44,868 epoch 38 - iter 18/25 - loss 3.32819162 - samples/sec: 131.31 - lr: 0.075000 +2021-03-26 05:31:45,977 epoch 38 - iter 20/25 - loss 3.37640109 - samples/sec: 115.61 - lr: 0.075000 +2021-03-26 05:31:46,938 epoch 38 - iter 22/25 - loss 3.35057510 - samples/sec: 133.40 - lr: 0.075000 +2021-03-26 05:31:47,950 epoch 38 - iter 24/25 - loss 3.31927971 - samples/sec: 126.65 - lr: 0.075000 +2021-03-26 05:31:48,426 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:31:48,428 EPOCH 38 done: loss 3.3323 - lr 0.0750000 +2021-03-26 05:31:49,240 DEV : loss 6.319576263427734 - score 0.9028 +2021-03-26 05:31:49,262 BAD EPOCHS (no improvement): 2 +2021-03-26 05:31:49,262 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:31:50,330 epoch 39 - iter 2/25 - loss 3.29510450 - samples/sec: 120.01 - lr: 0.075000 +2021-03-26 05:31:51,365 epoch 39 - iter 4/25 - loss 3.25340539 - samples/sec: 123.87 - lr: 0.075000 +2021-03-26 05:31:52,482 epoch 39 - iter 6/25 - loss 3.37839731 - samples/sec: 114.79 - lr: 0.075000 +2021-03-26 05:31:53,461 epoch 39 - iter 8/25 - loss 3.27371565 - samples/sec: 131.10 - lr: 0.075000 +2021-03-26 05:31:54,453 epoch 39 - iter 10/25 - loss 3.29483247 - samples/sec: 129.14 - lr: 0.075000 +2021-03-26 05:31:55,474 epoch 39 - iter 12/25 - loss 3.41713019 - samples/sec: 125.51 - lr: 0.075000 +2021-03-26 05:31:56,485 epoch 39 - iter 14/25 - loss 3.50181120 - samples/sec: 126.82 - lr: 0.075000 +2021-03-26 05:31:57,551 epoch 39 - iter 16/25 - loss 3.52467915 - samples/sec: 120.23 - lr: 0.075000 +2021-03-26 05:31:58,575 epoch 39 - iter 18/25 - loss 3.49890432 - samples/sec: 125.11 - lr: 0.075000 +2021-03-26 05:31:59,549 epoch 39 - iter 20/25 - loss 3.44117440 - samples/sec: 131.83 - lr: 0.075000 +2021-03-26 05:32:00,602 epoch 39 - iter 22/25 - loss 3.44448699 - samples/sec: 121.74 - lr: 0.075000 +2021-03-26 05:32:01,693 epoch 39 - iter 24/25 - loss 3.42074833 - samples/sec: 117.51 - lr: 0.075000 +2021-03-26 05:32:02,109 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:32:02,109 EPOCH 39 done: loss 3.4132 - lr 0.0750000 +2021-03-26 05:32:03,022 DEV : loss 6.410008907318115 - score 0.9026 +2021-03-26 05:32:03,059 BAD EPOCHS (no improvement): 3 +2021-03-26 05:32:03,060 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:32:04,419 epoch 40 - iter 2/25 - loss 3.88798189 - samples/sec: 94.24 - lr: 0.075000 +2021-03-26 05:32:05,455 epoch 40 - iter 4/25 - loss 3.43252563 - samples/sec: 123.75 - lr: 0.075000 +2021-03-26 05:32:06,402 epoch 40 - iter 6/25 - loss 3.40401618 - samples/sec: 135.34 - lr: 0.075000 +2021-03-26 05:32:07,373 epoch 40 - iter 8/25 - loss 3.47805119 - samples/sec: 132.02 - lr: 0.075000 +2021-03-26 05:32:08,406 epoch 40 - iter 10/25 - loss 3.35810285 - samples/sec: 124.20 - lr: 0.075000 +2021-03-26 05:32:09,540 epoch 40 - iter 12/25 - loss 3.38514515 - samples/sec: 113.06 - lr: 0.075000 +2021-03-26 05:32:10,601 epoch 40 - iter 14/25 - loss 3.45678117 - samples/sec: 120.87 - lr: 0.075000 +2021-03-26 05:32:11,651 epoch 40 - iter 16/25 - loss 3.44114582 - samples/sec: 122.06 - lr: 0.075000 +2021-03-26 05:32:12,652 epoch 40 - iter 18/25 - loss 3.41769059 - samples/sec: 128.04 - lr: 0.075000 +2021-03-26 05:32:13,656 epoch 40 - iter 20/25 - loss 3.35850424 - samples/sec: 127.65 - lr: 0.075000 +2021-03-26 05:32:14,629 epoch 40 - iter 22/25 - loss 3.37367962 - samples/sec: 131.95 - lr: 0.075000 +2021-03-26 05:32:15,607 epoch 40 - iter 24/25 - loss 3.38378793 - samples/sec: 131.13 - lr: 0.075000 +2021-03-26 05:32:16,021 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:32:16,022 EPOCH 40 done: loss 3.4051 - lr 0.0750000 +2021-03-26 05:32:16,848 DEV : loss 6.3716535568237305 - score 0.905 +2021-03-26 05:32:16,869 BAD EPOCHS (no improvement): 4 +2021-03-26 05:32:16,870 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:32:17,957 epoch 41 - iter 2/25 - loss 3.34791684 - samples/sec: 117.92 - lr: 0.037500 +2021-03-26 05:32:18,933 epoch 41 - iter 4/25 - loss 3.20045227 - samples/sec: 131.34 - lr: 0.037500 +2021-03-26 05:32:19,933 epoch 41 - iter 6/25 - loss 3.11513686 - samples/sec: 128.19 - lr: 0.037500 +2021-03-26 05:32:20,901 epoch 41 - iter 8/25 - loss 3.14670616 - samples/sec: 132.54 - lr: 0.037500 +2021-03-26 05:32:21,826 epoch 41 - iter 10/25 - loss 3.06280284 - samples/sec: 138.61 - lr: 0.037500 +2021-03-26 05:32:22,758 epoch 41 - iter 12/25 - loss 2.96115875 - samples/sec: 137.70 - lr: 0.037500 +2021-03-26 05:32:23,710 epoch 41 - iter 14/25 - loss 3.05098459 - samples/sec: 134.59 - lr: 0.037500 +2021-03-26 05:32:24,724 epoch 41 - iter 16/25 - loss 3.09193727 - samples/sec: 126.46 - lr: 0.037500 +2021-03-26 05:32:25,707 epoch 41 - iter 18/25 - loss 3.13746330 - samples/sec: 130.49 - lr: 0.037500 +2021-03-26 05:32:26,736 epoch 41 - iter 20/25 - loss 3.17778745 - samples/sec: 124.68 - lr: 0.037500 +2021-03-26 05:32:27,756 epoch 41 - iter 22/25 - loss 3.17188667 - samples/sec: 125.86 - lr: 0.037500 +2021-03-26 05:32:28,751 epoch 41 - iter 24/25 - loss 3.18510689 - samples/sec: 128.90 - lr: 0.037500 +2021-03-26 05:32:29,223 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:32:29,223 EPOCH 41 done: loss 3.1504 - lr 0.0375000 +2021-03-26 05:32:30,005 DEV : loss 6.398674011230469 - score 0.9053 +2021-03-26 05:32:30,030 BAD EPOCHS (no improvement): 1 +2021-03-26 05:32:30,031 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:32:30,985 epoch 42 - iter 2/25 - loss 3.15925717 - samples/sec: 134.36 - lr: 0.037500 +2021-03-26 05:32:32,147 epoch 42 - iter 4/25 - loss 3.45493281 - samples/sec: 110.30 - lr: 0.037500 +2021-03-26 05:32:33,177 epoch 42 - iter 6/25 - loss 3.47178149 - samples/sec: 124.61 - lr: 0.037500 +2021-03-26 05:32:34,163 epoch 42 - iter 8/25 - loss 3.59958142 - samples/sec: 129.93 - lr: 0.037500 +2021-03-26 05:32:35,197 epoch 42 - iter 10/25 - loss 3.63480940 - samples/sec: 123.95 - lr: 0.037500 +2021-03-26 05:32:36,247 epoch 42 - iter 12/25 - loss 3.63019852 - samples/sec: 122.12 - lr: 0.037500 +2021-03-26 05:32:37,260 epoch 42 - iter 14/25 - loss 3.54821042 - samples/sec: 126.46 - lr: 0.037500 +2021-03-26 05:32:38,251 epoch 42 - iter 16/25 - loss 3.53470874 - samples/sec: 129.42 - lr: 0.037500 +2021-03-26 05:32:39,251 epoch 42 - iter 18/25 - loss 3.45202567 - samples/sec: 128.12 - lr: 0.037500 +2021-03-26 05:32:40,278 epoch 42 - iter 20/25 - loss 3.44232910 - samples/sec: 124.83 - lr: 0.037500 +2021-03-26 05:32:41,627 epoch 42 - iter 22/25 - loss 3.45118979 - samples/sec: 95.01 - lr: 0.037500 +2021-03-26 05:32:42,888 epoch 42 - iter 24/25 - loss 3.40880919 - samples/sec: 101.62 - lr: 0.037500 +2021-03-26 05:32:43,434 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:32:43,435 EPOCH 42 done: loss 3.3765 - lr 0.0375000 +2021-03-26 05:32:44,352 DEV : loss 6.4176788330078125 - score 0.9057 +2021-03-26 05:32:44,377 BAD EPOCHS (no improvement): 2 +2021-03-26 05:32:44,378 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:32:45,335 epoch 43 - iter 2/25 - loss 2.82083404 - samples/sec: 133.97 - lr: 0.037500 +2021-03-26 05:32:46,395 epoch 43 - iter 4/25 - loss 3.20271707 - samples/sec: 120.95 - lr: 0.037500 +2021-03-26 05:32:47,336 epoch 43 - iter 6/25 - loss 3.25204869 - samples/sec: 136.25 - lr: 0.037500 +2021-03-26 05:32:48,299 epoch 43 - iter 8/25 - loss 3.03663874 - samples/sec: 133.12 - lr: 0.037500 +2021-03-26 05:32:49,351 epoch 43 - iter 10/25 - loss 3.14305215 - samples/sec: 121.88 - lr: 0.037500 +2021-03-26 05:32:50,326 epoch 43 - iter 12/25 - loss 3.22221923 - samples/sec: 131.41 - lr: 0.037500 +2021-03-26 05:32:51,400 epoch 43 - iter 14/25 - loss 3.29125517 - samples/sec: 119.39 - lr: 0.037500 +2021-03-26 05:32:52,432 epoch 43 - iter 16/25 - loss 3.22110757 - samples/sec: 124.14 - lr: 0.037500 +2021-03-26 05:32:53,486 epoch 43 - iter 18/25 - loss 3.21015777 - samples/sec: 121.63 - lr: 0.037500 +2021-03-26 05:32:54,487 epoch 43 - iter 20/25 - loss 3.20252607 - samples/sec: 128.80 - lr: 0.037500 +2021-03-26 05:32:55,439 epoch 43 - iter 22/25 - loss 3.22109760 - samples/sec: 134.82 - lr: 0.037500 +2021-03-26 05:32:56,690 epoch 43 - iter 24/25 - loss 3.17892468 - samples/sec: 102.49 - lr: 0.037500 +2021-03-26 05:32:57,175 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:32:57,177 EPOCH 43 done: loss 3.1621 - lr 0.0375000 +2021-03-26 05:32:57,997 DEV : loss 6.35455322265625 - score 0.9098 +2021-03-26 05:32:58,023 BAD EPOCHS (no improvement): 3 +2021-03-26 05:32:58,024 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:32:59,031 epoch 44 - iter 2/25 - loss 3.62210631 - samples/sec: 127.41 - lr: 0.037500 +2021-03-26 05:33:00,050 epoch 44 - iter 4/25 - loss 3.45865715 - samples/sec: 125.70 - lr: 0.037500 +2021-03-26 05:33:01,058 epoch 44 - iter 6/25 - loss 3.61776721 - samples/sec: 127.25 - lr: 0.037500 +2021-03-26 05:33:02,125 epoch 44 - iter 8/25 - loss 3.50572792 - samples/sec: 120.06 - lr: 0.037500 +2021-03-26 05:33:03,192 epoch 44 - iter 10/25 - loss 3.52592924 - samples/sec: 120.09 - lr: 0.037500 +2021-03-26 05:33:04,135 epoch 44 - iter 12/25 - loss 3.39799915 - samples/sec: 136.00 - lr: 0.037500 +2021-03-26 05:33:05,091 epoch 44 - iter 14/25 - loss 3.30765348 - samples/sec: 134.25 - lr: 0.037500 +2021-03-26 05:33:06,141 epoch 44 - iter 16/25 - loss 3.21087818 - samples/sec: 122.22 - lr: 0.037500 +2021-03-26 05:33:07,140 epoch 44 - iter 18/25 - loss 3.22574906 - samples/sec: 128.36 - lr: 0.037500 +2021-03-26 05:33:08,100 epoch 44 - iter 20/25 - loss 3.22437223 - samples/sec: 133.61 - lr: 0.037500 +2021-03-26 05:33:09,127 epoch 44 - iter 22/25 - loss 3.21107636 - samples/sec: 124.97 - lr: 0.037500 +2021-03-26 05:33:10,079 epoch 44 - iter 24/25 - loss 3.21195141 - samples/sec: 134.66 - lr: 0.037500 +2021-03-26 05:33:10,502 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:33:10,503 EPOCH 44 done: loss 3.2208 - lr 0.0375000 +2021-03-26 05:33:11,338 DEV : loss 6.370487213134766 - score 0.9091 +2021-03-26 05:33:11,361 BAD EPOCHS (no improvement): 4 +2021-03-26 05:33:11,361 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:33:12,317 epoch 45 - iter 2/25 - loss 3.09825575 - samples/sec: 134.09 - lr: 0.018750 +2021-03-26 05:33:13,357 epoch 45 - iter 4/25 - loss 3.01253098 - samples/sec: 123.29 - lr: 0.018750 +2021-03-26 05:33:14,358 epoch 45 - iter 6/25 - loss 3.01518444 - samples/sec: 128.01 - lr: 0.018750 +2021-03-26 05:33:15,372 epoch 45 - iter 8/25 - loss 2.96483403 - samples/sec: 126.48 - lr: 0.018750 +2021-03-26 05:33:16,288 epoch 45 - iter 10/25 - loss 3.02756095 - samples/sec: 139.85 - lr: 0.018750 +2021-03-26 05:33:17,306 epoch 45 - iter 12/25 - loss 3.05424559 - samples/sec: 125.89 - lr: 0.018750 +2021-03-26 05:33:18,188 epoch 45 - iter 14/25 - loss 2.99477315 - samples/sec: 145.41 - lr: 0.018750 +2021-03-26 05:33:19,206 epoch 45 - iter 16/25 - loss 3.03542289 - samples/sec: 125.96 - lr: 0.018750 +2021-03-26 05:33:20,261 epoch 45 - iter 18/25 - loss 2.99328383 - samples/sec: 121.49 - lr: 0.018750 +2021-03-26 05:33:21,242 epoch 45 - iter 20/25 - loss 2.96314116 - samples/sec: 130.65 - lr: 0.018750 +2021-03-26 05:33:22,238 epoch 45 - iter 22/25 - loss 2.94665932 - samples/sec: 128.64 - lr: 0.018750 +2021-03-26 05:33:23,322 epoch 45 - iter 24/25 - loss 2.94883043 - samples/sec: 118.29 - lr: 0.018750 +2021-03-26 05:33:23,816 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:33:23,817 EPOCH 45 done: loss 3.0238 - lr 0.0187500 +2021-03-26 05:33:24,632 DEV : loss 6.327853679656982 - score 0.9091 +2021-03-26 05:33:24,657 BAD EPOCHS (no improvement): 1 +2021-03-26 05:33:24,658 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:33:25,639 epoch 46 - iter 2/25 - loss 3.14161575 - samples/sec: 130.62 - lr: 0.018750 +2021-03-26 05:33:26,581 epoch 46 - iter 4/25 - loss 3.40499622 - samples/sec: 136.11 - lr: 0.018750 +2021-03-26 05:33:27,609 epoch 46 - iter 6/25 - loss 3.28151731 - samples/sec: 124.88 - lr: 0.018750 +2021-03-26 05:33:28,675 epoch 46 - iter 8/25 - loss 3.20727524 - samples/sec: 120.27 - lr: 0.018750 +2021-03-26 05:33:29,690 epoch 46 - iter 10/25 - loss 3.12398851 - samples/sec: 126.18 - lr: 0.018750 +2021-03-26 05:33:30,707 epoch 46 - iter 12/25 - loss 3.18230096 - samples/sec: 126.08 - lr: 0.018750 +2021-03-26 05:33:31,716 epoch 46 - iter 14/25 - loss 3.17806830 - samples/sec: 127.03 - lr: 0.018750 +2021-03-26 05:33:32,697 epoch 46 - iter 16/25 - loss 3.15068266 - samples/sec: 130.74 - lr: 0.018750 +2021-03-26 05:33:33,684 epoch 46 - iter 18/25 - loss 3.09301374 - samples/sec: 129.92 - lr: 0.018750 +2021-03-26 05:33:34,735 epoch 46 - iter 20/25 - loss 3.11505752 - samples/sec: 122.02 - lr: 0.018750 +2021-03-26 05:33:35,761 epoch 46 - iter 22/25 - loss 3.13960059 - samples/sec: 124.91 - lr: 0.018750 +2021-03-26 05:33:36,756 epoch 46 - iter 24/25 - loss 3.14284589 - samples/sec: 128.82 - lr: 0.018750 +2021-03-26 05:33:37,172 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:33:37,173 EPOCH 46 done: loss 3.1034 - lr 0.0187500 +2021-03-26 05:33:37,957 DEV : loss 6.337489128112793 - score 0.9087 +2021-03-26 05:33:37,976 BAD EPOCHS (no improvement): 2 +2021-03-26 05:33:37,977 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:33:38,964 epoch 47 - iter 2/25 - loss 2.61264515 - samples/sec: 129.86 - lr: 0.018750 +2021-03-26 05:33:40,083 epoch 47 - iter 4/25 - loss 3.06716627 - samples/sec: 114.66 - lr: 0.018750 +2021-03-26 05:33:41,147 epoch 47 - iter 6/25 - loss 3.07863943 - samples/sec: 120.99 - lr: 0.018750 +2021-03-26 05:33:42,200 epoch 47 - iter 8/25 - loss 3.02197772 - samples/sec: 121.70 - lr: 0.018750 +2021-03-26 05:33:43,263 epoch 47 - iter 10/25 - loss 2.94771438 - samples/sec: 120.58 - lr: 0.018750 +2021-03-26 05:33:44,381 epoch 47 - iter 12/25 - loss 3.08820546 - samples/sec: 114.68 - lr: 0.018750 +2021-03-26 05:33:45,348 epoch 47 - iter 14/25 - loss 3.06957422 - samples/sec: 132.64 - lr: 0.018750 +2021-03-26 05:33:46,439 epoch 47 - iter 16/25 - loss 3.05880760 - samples/sec: 117.44 - lr: 0.018750 +2021-03-26 05:33:47,395 epoch 47 - iter 18/25 - loss 3.07288737 - samples/sec: 134.16 - lr: 0.018750 +2021-03-26 05:33:48,386 epoch 47 - iter 20/25 - loss 3.03714904 - samples/sec: 129.25 - lr: 0.018750 +2021-03-26 05:33:49,410 epoch 47 - iter 22/25 - loss 3.10431696 - samples/sec: 125.20 - lr: 0.018750 +2021-03-26 05:33:50,409 epoch 47 - iter 24/25 - loss 3.18330712 - samples/sec: 128.37 - lr: 0.018750 +2021-03-26 05:33:50,815 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:33:50,815 EPOCH 47 done: loss 3.2111 - lr 0.0187500 +2021-03-26 05:33:51,603 DEV : loss 6.363578796386719 - score 0.9063 +2021-03-26 05:33:51,628 BAD EPOCHS (no improvement): 3 +2021-03-26 05:33:51,629 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:33:52,737 epoch 48 - iter 2/25 - loss 3.24833369 - samples/sec: 115.76 - lr: 0.018750 +2021-03-26 05:33:53,781 epoch 48 - iter 4/25 - loss 3.13858742 - samples/sec: 122.69 - lr: 0.018750 +2021-03-26 05:33:54,843 epoch 48 - iter 6/25 - loss 3.05964986 - samples/sec: 120.77 - lr: 0.018750 +2021-03-26 05:33:55,835 epoch 48 - iter 8/25 - loss 3.11314756 - samples/sec: 129.26 - lr: 0.018750 +2021-03-26 05:33:56,761 epoch 48 - iter 10/25 - loss 3.24263086 - samples/sec: 138.52 - lr: 0.018750 +2021-03-26 05:33:57,660 epoch 48 - iter 12/25 - loss 3.24108704 - samples/sec: 142.56 - lr: 0.018750 +2021-03-26 05:33:58,584 epoch 48 - iter 14/25 - loss 3.15899248 - samples/sec: 138.89 - lr: 0.018750 +2021-03-26 05:33:59,661 epoch 48 - iter 16/25 - loss 3.08050616 - samples/sec: 119.03 - lr: 0.018750 +2021-03-26 05:34:00,609 epoch 48 - iter 18/25 - loss 3.03743966 - samples/sec: 135.31 - lr: 0.018750 +2021-03-26 05:34:01,608 epoch 48 - iter 20/25 - loss 3.04794432 - samples/sec: 128.33 - lr: 0.018750 +2021-03-26 05:34:02,693 epoch 48 - iter 22/25 - loss 3.06857406 - samples/sec: 118.19 - lr: 0.018750 +2021-03-26 05:34:03,674 epoch 48 - iter 24/25 - loss 3.06631840 - samples/sec: 130.70 - lr: 0.018750 +2021-03-26 05:34:04,129 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:34:04,130 EPOCH 48 done: loss 3.0756 - lr 0.0187500 +2021-03-26 05:34:04,920 DEV : loss 6.344338893890381 - score 0.9079 +2021-03-26 05:34:04,946 BAD EPOCHS (no improvement): 4 +2021-03-26 05:34:04,947 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:34:05,956 epoch 49 - iter 2/25 - loss 2.90295172 - samples/sec: 127.06 - lr: 0.009375 +2021-03-26 05:34:07,057 epoch 49 - iter 4/25 - loss 2.80841428 - samples/sec: 116.39 - lr: 0.009375 +2021-03-26 05:34:08,051 epoch 49 - iter 6/25 - loss 3.03871755 - samples/sec: 128.94 - lr: 0.009375 +2021-03-26 05:34:09,033 epoch 49 - iter 8/25 - loss 2.99644694 - samples/sec: 130.55 - lr: 0.009375 +2021-03-26 05:34:10,126 epoch 49 - iter 10/25 - loss 3.09761269 - samples/sec: 117.32 - lr: 0.009375 +2021-03-26 05:34:11,289 epoch 49 - iter 12/25 - loss 3.13177150 - samples/sec: 110.28 - lr: 0.009375 +2021-03-26 05:34:12,285 epoch 49 - iter 14/25 - loss 3.14276028 - samples/sec: 128.80 - lr: 0.009375 +2021-03-26 05:34:13,293 epoch 49 - iter 16/25 - loss 3.15640897 - samples/sec: 127.17 - lr: 0.009375 +2021-03-26 05:34:14,359 epoch 49 - iter 18/25 - loss 3.10915577 - samples/sec: 120.22 - lr: 0.009375 +2021-03-26 05:34:15,401 epoch 49 - iter 20/25 - loss 3.11137698 - samples/sec: 122.95 - lr: 0.009375 +2021-03-26 05:34:16,557 epoch 49 - iter 22/25 - loss 3.11102456 - samples/sec: 110.91 - lr: 0.009375 +2021-03-26 05:34:17,577 epoch 49 - iter 24/25 - loss 3.09136966 - samples/sec: 125.73 - lr: 0.009375 +2021-03-26 05:34:17,991 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:34:17,992 EPOCH 49 done: loss 3.0979 - lr 0.0093750 +2021-03-26 05:34:18,835 DEV : loss 6.343489646911621 - score 0.9083 +2021-03-26 05:34:18,863 BAD EPOCHS (no improvement): 1 +2021-03-26 05:34:18,863 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:34:20,004 epoch 50 - iter 2/25 - loss 3.47275829 - samples/sec: 112.39 - lr: 0.009375 +2021-03-26 05:34:20,952 epoch 50 - iter 4/25 - loss 3.08802807 - samples/sec: 135.18 - lr: 0.009375 +2021-03-26 05:34:21,939 epoch 50 - iter 6/25 - loss 3.08415246 - samples/sec: 129.81 - lr: 0.009375 +2021-03-26 05:34:22,827 epoch 50 - iter 8/25 - loss 2.93821096 - samples/sec: 144.47 - lr: 0.009375 +2021-03-26 05:34:23,795 epoch 50 - iter 10/25 - loss 2.97539239 - samples/sec: 132.39 - lr: 0.009375 +2021-03-26 05:34:24,733 epoch 50 - iter 12/25 - loss 3.06500099 - samples/sec: 136.72 - lr: 0.009375 +2021-03-26 05:34:25,685 epoch 50 - iter 14/25 - loss 3.03774973 - samples/sec: 134.60 - lr: 0.009375 +2021-03-26 05:34:26,743 epoch 50 - iter 16/25 - loss 3.02531400 - samples/sec: 121.21 - lr: 0.009375 +2021-03-26 05:34:27,751 epoch 50 - iter 18/25 - loss 3.06442942 - samples/sec: 127.10 - lr: 0.009375 +2021-03-26 05:34:28,821 epoch 50 - iter 20/25 - loss 3.06219168 - samples/sec: 119.87 - lr: 0.009375 +2021-03-26 05:34:29,823 epoch 50 - iter 22/25 - loss 3.11562943 - samples/sec: 127.91 - lr: 0.009375 +2021-03-26 05:34:30,851 epoch 50 - iter 24/25 - loss 3.09196840 - samples/sec: 124.64 - lr: 0.009375 +2021-03-26 05:34:31,287 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:34:31,288 EPOCH 50 done: loss 3.0812 - lr 0.0093750 +2021-03-26 05:34:32,108 DEV : loss 6.350255489349365 - score 0.9083 +2021-03-26 05:34:32,134 BAD EPOCHS (no improvement): 2 +2021-03-26 05:34:32,135 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:34:33,177 epoch 51 - iter 2/25 - loss 2.86416495 - samples/sec: 123.12 - lr: 0.009375 +2021-03-26 05:34:34,203 epoch 51 - iter 4/25 - loss 2.97483855 - samples/sec: 125.12 - lr: 0.009375 +2021-03-26 05:34:35,163 epoch 51 - iter 6/25 - loss 3.07030086 - samples/sec: 133.64 - lr: 0.009375 +2021-03-26 05:34:36,245 epoch 51 - iter 8/25 - loss 3.02221566 - samples/sec: 118.49 - lr: 0.009375 +2021-03-26 05:34:37,260 epoch 51 - iter 10/25 - loss 3.09667757 - samples/sec: 126.23 - lr: 0.009375 +2021-03-26 05:34:38,207 epoch 51 - iter 12/25 - loss 2.98070341 - samples/sec: 135.28 - lr: 0.009375 +2021-03-26 05:34:39,182 epoch 51 - iter 14/25 - loss 3.00901944 - samples/sec: 131.61 - lr: 0.009375 +2021-03-26 05:34:40,156 epoch 51 - iter 16/25 - loss 3.00842324 - samples/sec: 131.62 - lr: 0.009375 +2021-03-26 05:34:41,214 epoch 51 - iter 18/25 - loss 3.00116184 - samples/sec: 121.14 - lr: 0.009375 +2021-03-26 05:34:42,194 epoch 51 - iter 20/25 - loss 2.96676878 - samples/sec: 130.76 - lr: 0.009375 +2021-03-26 05:34:43,226 epoch 51 - iter 22/25 - loss 2.99044778 - samples/sec: 124.21 - lr: 0.009375 +2021-03-26 05:34:44,183 epoch 51 - iter 24/25 - loss 2.96513258 - samples/sec: 133.94 - lr: 0.009375 +2021-03-26 05:34:44,625 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:34:44,626 EPOCH 51 done: loss 2.9734 - lr 0.0093750 +2021-03-26 05:34:45,445 DEV : loss 6.363671779632568 - score 0.9087 +2021-03-26 05:34:45,464 BAD EPOCHS (no improvement): 3 +2021-03-26 05:34:45,464 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:34:46,437 epoch 52 - iter 2/25 - loss 2.26139700 - samples/sec: 131.88 - lr: 0.009375 +2021-03-26 05:34:47,526 epoch 52 - iter 4/25 - loss 3.09052759 - samples/sec: 117.63 - lr: 0.009375 +2021-03-26 05:34:48,565 epoch 52 - iter 6/25 - loss 3.00616634 - samples/sec: 123.55 - lr: 0.009375 +2021-03-26 05:34:49,646 epoch 52 - iter 8/25 - loss 3.15239090 - samples/sec: 118.57 - lr: 0.009375 +2021-03-26 05:34:50,583 epoch 52 - iter 10/25 - loss 3.08683448 - samples/sec: 136.88 - lr: 0.009375 +2021-03-26 05:34:51,666 epoch 52 - iter 12/25 - loss 3.13589352 - samples/sec: 118.42 - lr: 0.009375 +2021-03-26 05:34:52,599 epoch 52 - iter 14/25 - loss 3.12352077 - samples/sec: 137.38 - lr: 0.009375 +2021-03-26 05:34:53,593 epoch 52 - iter 16/25 - loss 3.15578875 - samples/sec: 128.99 - lr: 0.009375 +2021-03-26 05:34:54,508 epoch 52 - iter 18/25 - loss 3.09990822 - samples/sec: 139.98 - lr: 0.009375 +2021-03-26 05:34:55,486 epoch 52 - iter 20/25 - loss 3.11360563 - samples/sec: 131.33 - lr: 0.009375 +2021-03-26 05:34:56,500 epoch 52 - iter 22/25 - loss 3.08807242 - samples/sec: 126.45 - lr: 0.009375 +2021-03-26 05:34:57,670 epoch 52 - iter 24/25 - loss 3.12041364 - samples/sec: 109.59 - lr: 0.009375 +2021-03-26 05:34:58,163 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:34:58,164 EPOCH 52 done: loss 3.1056 - lr 0.0093750 +2021-03-26 05:34:58,969 DEV : loss 6.35579252243042 - score 0.9071 +2021-03-26 05:34:58,995 BAD EPOCHS (no improvement): 4 +2021-03-26 05:34:58,996 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:34:59,929 epoch 53 - iter 2/25 - loss 3.55681157 - samples/sec: 137.52 - lr: 0.004687 +2021-03-26 05:35:01,034 epoch 53 - iter 4/25 - loss 3.48824114 - samples/sec: 116.18 - lr: 0.004687 +2021-03-26 05:35:01,980 epoch 53 - iter 6/25 - loss 3.35435299 - samples/sec: 135.43 - lr: 0.004687 +2021-03-26 05:35:02,911 epoch 53 - iter 8/25 - loss 3.39199522 - samples/sec: 137.81 - lr: 0.004687 +2021-03-26 05:35:03,926 epoch 53 - iter 10/25 - loss 3.30036414 - samples/sec: 126.29 - lr: 0.004687 +2021-03-26 05:35:04,928 epoch 53 - iter 12/25 - loss 3.37839397 - samples/sec: 127.93 - lr: 0.004687 +2021-03-26 05:35:05,957 epoch 53 - iter 14/25 - loss 3.29179309 - samples/sec: 124.68 - lr: 0.004687 +2021-03-26 05:35:06,908 epoch 53 - iter 16/25 - loss 3.22510788 - samples/sec: 134.90 - lr: 0.004687 +2021-03-26 05:35:07,859 epoch 53 - iter 18/25 - loss 3.10731896 - samples/sec: 134.79 - lr: 0.004687 +2021-03-26 05:35:08,904 epoch 53 - iter 20/25 - loss 3.15653430 - samples/sec: 122.68 - lr: 0.004687 +2021-03-26 05:35:09,953 epoch 53 - iter 22/25 - loss 3.10727158 - samples/sec: 122.14 - lr: 0.004687 +2021-03-26 05:35:10,992 epoch 53 - iter 24/25 - loss 3.12946189 - samples/sec: 123.56 - lr: 0.004687 +2021-03-26 05:35:11,388 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:35:11,389 EPOCH 53 done: loss 3.1814 - lr 0.0046875 +2021-03-26 05:35:12,227 DEV : loss 6.365192413330078 - score 0.9063 +2021-03-26 05:35:12,253 BAD EPOCHS (no improvement): 1 +2021-03-26 05:35:12,254 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:35:13,283 epoch 54 - iter 2/25 - loss 3.76396322 - samples/sec: 124.63 - lr: 0.004687 +2021-03-26 05:35:14,337 epoch 54 - iter 4/25 - loss 3.37461126 - samples/sec: 121.51 - lr: 0.004687 +2021-03-26 05:35:15,308 epoch 54 - iter 6/25 - loss 3.34941999 - samples/sec: 132.10 - lr: 0.004687 +2021-03-26 05:35:16,349 epoch 54 - iter 8/25 - loss 3.31672096 - samples/sec: 123.21 - lr: 0.004687 +2021-03-26 05:35:17,361 epoch 54 - iter 10/25 - loss 3.20109630 - samples/sec: 126.83 - lr: 0.004687 +2021-03-26 05:35:18,394 epoch 54 - iter 12/25 - loss 3.12487588 - samples/sec: 124.05 - lr: 0.004687 +2021-03-26 05:35:19,355 epoch 54 - iter 14/25 - loss 3.03142987 - samples/sec: 133.38 - lr: 0.004687 +2021-03-26 05:35:20,262 epoch 54 - iter 16/25 - loss 3.00063097 - samples/sec: 141.34 - lr: 0.004687 +2021-03-26 05:35:21,245 epoch 54 - iter 18/25 - loss 2.98818886 - samples/sec: 130.38 - lr: 0.004687 +2021-03-26 05:35:22,153 epoch 54 - iter 20/25 - loss 2.98626262 - samples/sec: 141.15 - lr: 0.004687 +2021-03-26 05:35:23,284 epoch 54 - iter 22/25 - loss 3.06153673 - samples/sec: 113.38 - lr: 0.004687 +2021-03-26 05:35:24,313 epoch 54 - iter 24/25 - loss 3.05973284 - samples/sec: 124.95 - lr: 0.004687 +2021-03-26 05:35:24,650 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:35:24,651 EPOCH 54 done: loss 3.0402 - lr 0.0046875 +2021-03-26 05:35:25,433 DEV : loss 6.367043972015381 - score 0.9083 +2021-03-26 05:35:25,458 BAD EPOCHS (no improvement): 2 +2021-03-26 05:35:25,458 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:35:26,443 epoch 55 - iter 2/25 - loss 3.18132973 - samples/sec: 130.12 - lr: 0.004687 +2021-03-26 05:35:27,402 epoch 55 - iter 4/25 - loss 3.01191485 - samples/sec: 133.68 - lr: 0.004687 +2021-03-26 05:35:28,349 epoch 55 - iter 6/25 - loss 3.20665197 - samples/sec: 135.35 - lr: 0.004687 +2021-03-26 05:35:29,357 epoch 55 - iter 8/25 - loss 3.06920093 - samples/sec: 127.17 - lr: 0.004687 +2021-03-26 05:35:30,308 epoch 55 - iter 10/25 - loss 3.01357651 - samples/sec: 134.84 - lr: 0.004687 +2021-03-26 05:35:31,295 epoch 55 - iter 12/25 - loss 2.96591622 - samples/sec: 130.00 - lr: 0.004687 +2021-03-26 05:35:32,279 epoch 55 - iter 14/25 - loss 3.02729602 - samples/sec: 130.45 - lr: 0.004687 +2021-03-26 05:35:33,331 epoch 55 - iter 16/25 - loss 3.05965899 - samples/sec: 121.77 - lr: 0.004687 +2021-03-26 05:35:34,362 epoch 55 - iter 18/25 - loss 3.08807895 - samples/sec: 124.51 - lr: 0.004687 +2021-03-26 05:35:35,329 epoch 55 - iter 20/25 - loss 3.05238564 - samples/sec: 132.51 - lr: 0.004687 +2021-03-26 05:35:36,240 epoch 55 - iter 22/25 - loss 3.09957942 - samples/sec: 140.64 - lr: 0.004687 +2021-03-26 05:35:37,262 epoch 55 - iter 24/25 - loss 3.08548250 - samples/sec: 125.64 - lr: 0.004687 +2021-03-26 05:35:37,675 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:35:37,676 EPOCH 55 done: loss 3.0630 - lr 0.0046875 +2021-03-26 05:35:38,494 DEV : loss 6.367519378662109 - score 0.9083 +2021-03-26 05:35:38,519 BAD EPOCHS (no improvement): 3 +2021-03-26 05:35:38,520 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:35:40,903 epoch 56 - iter 2/25 - loss 3.04370713 - samples/sec: 53.74 - lr: 0.004687 +2021-03-26 05:35:41,945 epoch 56 - iter 4/25 - loss 2.89844847 - samples/sec: 123.12 - lr: 0.004687 +2021-03-26 05:35:42,927 epoch 56 - iter 6/25 - loss 3.06276659 - samples/sec: 130.57 - lr: 0.004687 +2021-03-26 05:35:43,962 epoch 56 - iter 8/25 - loss 2.99346858 - samples/sec: 123.83 - lr: 0.004687 +2021-03-26 05:35:45,041 epoch 56 - iter 10/25 - loss 3.06103053 - samples/sec: 118.82 - lr: 0.004687 +2021-03-26 05:35:46,109 epoch 56 - iter 12/25 - loss 3.04417761 - samples/sec: 119.97 - lr: 0.004687 +2021-03-26 05:35:47,080 epoch 56 - iter 14/25 - loss 2.95337142 - samples/sec: 132.06 - lr: 0.004687 +2021-03-26 05:35:48,062 epoch 56 - iter 16/25 - loss 2.97934601 - samples/sec: 130.64 - lr: 0.004687 +2021-03-26 05:35:49,057 epoch 56 - iter 18/25 - loss 3.03061761 - samples/sec: 128.70 - lr: 0.004687 +2021-03-26 05:35:50,112 epoch 56 - iter 20/25 - loss 2.99428663 - samples/sec: 121.59 - lr: 0.004687 +2021-03-26 05:35:51,098 epoch 56 - iter 22/25 - loss 2.97807759 - samples/sec: 129.87 - lr: 0.004687 +2021-03-26 05:35:52,007 epoch 56 - iter 24/25 - loss 3.01112747 - samples/sec: 141.03 - lr: 0.004687 +2021-03-26 05:35:52,426 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:35:52,426 EPOCH 56 done: loss 3.0223 - lr 0.0046875 +2021-03-26 05:35:53,229 DEV : loss 6.379022121429443 - score 0.9079 +2021-03-26 05:35:53,255 BAD EPOCHS (no improvement): 4 +2021-03-26 05:35:53,256 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:35:54,324 epoch 57 - iter 2/25 - loss 2.88998890 - samples/sec: 119.97 - lr: 0.002344 +2021-03-26 05:35:55,245 epoch 57 - iter 4/25 - loss 2.84257925 - samples/sec: 139.19 - lr: 0.002344 +2021-03-26 05:35:56,317 epoch 57 - iter 6/25 - loss 2.82792898 - samples/sec: 119.56 - lr: 0.002344 +2021-03-26 05:35:57,254 epoch 57 - iter 8/25 - loss 2.98162171 - samples/sec: 136.75 - lr: 0.002344 +2021-03-26 05:35:58,260 epoch 57 - iter 10/25 - loss 2.88686635 - samples/sec: 127.45 - lr: 0.002344 +2021-03-26 05:35:59,248 epoch 57 - iter 12/25 - loss 2.93057438 - samples/sec: 129.76 - lr: 0.002344 +2021-03-26 05:36:00,294 epoch 57 - iter 14/25 - loss 3.00041650 - samples/sec: 122.51 - lr: 0.002344 +2021-03-26 05:36:01,305 epoch 57 - iter 16/25 - loss 2.98985383 - samples/sec: 126.77 - lr: 0.002344 +2021-03-26 05:36:02,389 epoch 57 - iter 18/25 - loss 3.00885790 - samples/sec: 118.16 - lr: 0.002344 +2021-03-26 05:36:03,359 epoch 57 - iter 20/25 - loss 3.08822603 - samples/sec: 132.10 - lr: 0.002344 +2021-03-26 05:36:04,328 epoch 57 - iter 22/25 - loss 3.06954510 - samples/sec: 132.25 - lr: 0.002344 +2021-03-26 05:36:05,306 epoch 57 - iter 24/25 - loss 3.01040073 - samples/sec: 131.05 - lr: 0.002344 +2021-03-26 05:36:05,745 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:36:05,746 EPOCH 57 done: loss 3.0233 - lr 0.0023437 +2021-03-26 05:36:06,549 DEV : loss 6.381903648376465 - score 0.9079 +2021-03-26 05:36:06,568 BAD EPOCHS (no improvement): 1 +2021-03-26 05:36:06,569 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:36:07,556 epoch 58 - iter 2/25 - loss 3.42313194 - samples/sec: 129.79 - lr: 0.002344 +2021-03-26 05:36:08,494 epoch 58 - iter 4/25 - loss 2.94785899 - samples/sec: 136.95 - lr: 0.002344 +2021-03-26 05:36:09,451 epoch 58 - iter 6/25 - loss 3.20075154 - samples/sec: 133.97 - lr: 0.002344 +2021-03-26 05:36:10,430 epoch 58 - iter 8/25 - loss 3.01168898 - samples/sec: 130.85 - lr: 0.002344 +2021-03-26 05:36:11,344 epoch 58 - iter 10/25 - loss 3.00945287 - samples/sec: 140.48 - lr: 0.002344 +2021-03-26 05:36:12,317 epoch 58 - iter 12/25 - loss 2.96118955 - samples/sec: 131.81 - lr: 0.002344 +2021-03-26 05:36:13,387 epoch 58 - iter 14/25 - loss 3.06854313 - samples/sec: 119.75 - lr: 0.002344 +2021-03-26 05:36:14,458 epoch 58 - iter 16/25 - loss 3.09105952 - samples/sec: 119.70 - lr: 0.002344 +2021-03-26 05:36:15,444 epoch 58 - iter 18/25 - loss 3.07459023 - samples/sec: 129.99 - lr: 0.002344 +2021-03-26 05:36:16,498 epoch 58 - iter 20/25 - loss 3.08885949 - samples/sec: 121.59 - lr: 0.002344 +2021-03-26 05:36:17,478 epoch 58 - iter 22/25 - loss 3.01594584 - samples/sec: 130.82 - lr: 0.002344 +2021-03-26 05:36:18,463 epoch 58 - iter 24/25 - loss 3.01048505 - samples/sec: 130.16 - lr: 0.002344 +2021-03-26 05:36:18,853 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:36:18,854 EPOCH 58 done: loss 3.0107 - lr 0.0023437 +2021-03-26 05:36:19,644 DEV : loss 6.3849005699157715 - score 0.9075 +2021-03-26 05:36:19,665 BAD EPOCHS (no improvement): 2 +2021-03-26 05:36:19,665 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:36:20,693 epoch 59 - iter 2/25 - loss 3.14258122 - samples/sec: 124.68 - lr: 0.002344 +2021-03-26 05:36:21,687 epoch 59 - iter 4/25 - loss 3.02675045 - samples/sec: 128.98 - lr: 0.002344 +2021-03-26 05:36:22,686 epoch 59 - iter 6/25 - loss 3.07165686 - samples/sec: 128.31 - lr: 0.002344 +2021-03-26 05:36:23,795 epoch 59 - iter 8/25 - loss 3.22081539 - samples/sec: 115.57 - lr: 0.002344 +2021-03-26 05:36:24,772 epoch 59 - iter 10/25 - loss 3.13599381 - samples/sec: 131.19 - lr: 0.002344 +2021-03-26 05:36:25,738 epoch 59 - iter 12/25 - loss 3.14673634 - samples/sec: 132.74 - lr: 0.002344 +2021-03-26 05:36:26,705 epoch 59 - iter 14/25 - loss 3.11754748 - samples/sec: 132.56 - lr: 0.002344 +2021-03-26 05:36:27,768 epoch 59 - iter 16/25 - loss 3.15714070 - samples/sec: 120.55 - lr: 0.002344 +2021-03-26 05:36:28,789 epoch 59 - iter 18/25 - loss 3.15854804 - samples/sec: 125.54 - lr: 0.002344 +2021-03-26 05:36:29,748 epoch 59 - iter 20/25 - loss 3.11217291 - samples/sec: 133.71 - lr: 0.002344 +2021-03-26 05:36:30,762 epoch 59 - iter 22/25 - loss 3.09361850 - samples/sec: 126.35 - lr: 0.002344 +2021-03-26 05:36:31,769 epoch 59 - iter 24/25 - loss 3.06539866 - samples/sec: 127.36 - lr: 0.002344 +2021-03-26 05:36:32,160 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:36:32,161 EPOCH 59 done: loss 3.0495 - lr 0.0023437 +2021-03-26 05:36:32,961 DEV : loss 6.381946563720703 - score 0.9071 +2021-03-26 05:36:32,981 BAD EPOCHS (no improvement): 3 +2021-03-26 05:36:32,982 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:36:33,952 epoch 60 - iter 2/25 - loss 3.10077047 - samples/sec: 132.13 - lr: 0.002344 +2021-03-26 05:36:34,962 epoch 60 - iter 4/25 - loss 2.88989782 - samples/sec: 126.93 - lr: 0.002344 +2021-03-26 05:36:35,873 epoch 60 - iter 6/25 - loss 2.90555080 - samples/sec: 140.67 - lr: 0.002344 +2021-03-26 05:36:36,838 epoch 60 - iter 8/25 - loss 2.95553589 - samples/sec: 132.88 - lr: 0.002344 +2021-03-26 05:36:37,849 epoch 60 - iter 10/25 - loss 2.89822590 - samples/sec: 126.71 - lr: 0.002344 +2021-03-26 05:36:38,844 epoch 60 - iter 12/25 - loss 2.84939009 - samples/sec: 128.89 - lr: 0.002344 +2021-03-26 05:36:39,908 epoch 60 - iter 14/25 - loss 2.96647346 - samples/sec: 120.48 - lr: 0.002344 +2021-03-26 05:36:40,890 epoch 60 - iter 16/25 - loss 2.92440678 - samples/sec: 130.50 - lr: 0.002344 +2021-03-26 05:36:41,859 epoch 60 - iter 18/25 - loss 2.91018026 - samples/sec: 132.35 - lr: 0.002344 +2021-03-26 05:36:42,913 epoch 60 - iter 20/25 - loss 2.87115874 - samples/sec: 121.70 - lr: 0.002344 +2021-03-26 05:36:43,925 epoch 60 - iter 22/25 - loss 2.86456972 - samples/sec: 126.67 - lr: 0.002344 +2021-03-26 05:36:44,911 epoch 60 - iter 24/25 - loss 2.85872015 - samples/sec: 130.03 - lr: 0.002344 +2021-03-26 05:36:45,329 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:36:45,331 EPOCH 60 done: loss 2.8299 - lr 0.0023437 +2021-03-26 05:36:46,121 DEV : loss 6.377230644226074 - score 0.9075 +2021-03-26 05:36:46,148 BAD EPOCHS (no improvement): 4 +2021-03-26 05:36:46,149 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:36:47,118 epoch 61 - iter 2/25 - loss 2.99023068 - samples/sec: 132.24 - lr: 0.001172 +2021-03-26 05:36:48,077 epoch 61 - iter 4/25 - loss 3.16693401 - samples/sec: 133.73 - lr: 0.001172 +2021-03-26 05:36:49,123 epoch 61 - iter 6/25 - loss 3.01899978 - samples/sec: 122.59 - lr: 0.001172 +2021-03-26 05:36:50,130 epoch 61 - iter 8/25 - loss 2.93510966 - samples/sec: 127.22 - lr: 0.001172 +2021-03-26 05:36:51,117 epoch 61 - iter 10/25 - loss 2.98047098 - samples/sec: 129.84 - lr: 0.001172 +2021-03-26 05:36:52,018 epoch 61 - iter 12/25 - loss 2.97246026 - samples/sec: 142.56 - lr: 0.001172 +2021-03-26 05:36:52,953 epoch 61 - iter 14/25 - loss 2.95344056 - samples/sec: 137.22 - lr: 0.001172 +2021-03-26 05:36:53,988 epoch 61 - iter 16/25 - loss 2.95693909 - samples/sec: 123.85 - lr: 0.001172 +2021-03-26 05:36:54,997 epoch 61 - iter 18/25 - loss 2.91543525 - samples/sec: 127.00 - lr: 0.001172 +2021-03-26 05:36:55,949 epoch 61 - iter 20/25 - loss 2.89163441 - samples/sec: 134.74 - lr: 0.001172 +2021-03-26 05:36:56,862 epoch 61 - iter 22/25 - loss 2.89356511 - samples/sec: 140.45 - lr: 0.001172 +2021-03-26 05:36:57,976 epoch 61 - iter 24/25 - loss 2.88996286 - samples/sec: 115.07 - lr: 0.001172 +2021-03-26 05:36:58,341 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:36:58,343 EPOCH 61 done: loss 2.8845 - lr 0.0011719 +2021-03-26 05:36:59,160 DEV : loss 6.3796706199646 - score 0.9059 +2021-03-26 05:36:59,178 BAD EPOCHS (no improvement): 1 +2021-03-26 05:36:59,179 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:37:00,132 epoch 62 - iter 2/25 - loss 3.14777243 - samples/sec: 134.59 - lr: 0.001172 +2021-03-26 05:37:01,114 epoch 62 - iter 4/25 - loss 3.04239559 - samples/sec: 130.54 - lr: 0.001172 +2021-03-26 05:37:02,109 epoch 62 - iter 6/25 - loss 2.89608514 - samples/sec: 128.83 - lr: 0.001172 +2021-03-26 05:37:03,144 epoch 62 - iter 8/25 - loss 3.01286557 - samples/sec: 123.77 - lr: 0.001172 +2021-03-26 05:37:04,403 epoch 62 - iter 10/25 - loss 3.09971406 - samples/sec: 101.85 - lr: 0.001172 +2021-03-26 05:37:05,541 epoch 62 - iter 12/25 - loss 3.12393322 - samples/sec: 112.61 - lr: 0.001172 +2021-03-26 05:37:06,497 epoch 62 - iter 14/25 - loss 3.08095258 - samples/sec: 134.34 - lr: 0.001172 +2021-03-26 05:37:07,512 epoch 62 - iter 16/25 - loss 3.00411677 - samples/sec: 126.24 - lr: 0.001172 +2021-03-26 05:37:08,586 epoch 62 - iter 18/25 - loss 2.98847085 - samples/sec: 119.31 - lr: 0.001172 +2021-03-26 05:37:09,476 epoch 62 - iter 20/25 - loss 2.92107501 - samples/sec: 144.82 - lr: 0.001172 +2021-03-26 05:37:10,445 epoch 62 - iter 22/25 - loss 2.91261944 - samples/sec: 132.26 - lr: 0.001172 +2021-03-26 05:37:11,455 epoch 62 - iter 24/25 - loss 2.91488413 - samples/sec: 126.93 - lr: 0.001172 +2021-03-26 05:37:11,925 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:37:11,925 EPOCH 62 done: loss 2.9002 - lr 0.0011719 +2021-03-26 05:37:12,718 DEV : loss 6.378934383392334 - score 0.9071 +2021-03-26 05:37:12,744 BAD EPOCHS (no improvement): 2 +2021-03-26 05:37:12,745 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:37:13,690 epoch 63 - iter 2/25 - loss 3.09829128 - samples/sec: 135.69 - lr: 0.001172 +2021-03-26 05:37:14,650 epoch 63 - iter 4/25 - loss 3.14043283 - samples/sec: 133.44 - lr: 0.001172 +2021-03-26 05:37:15,746 epoch 63 - iter 6/25 - loss 3.22749770 - samples/sec: 117.13 - lr: 0.001172 +2021-03-26 05:37:16,890 epoch 63 - iter 8/25 - loss 3.31378135 - samples/sec: 111.97 - lr: 0.001172 +2021-03-26 05:37:17,947 epoch 63 - iter 10/25 - loss 3.31359937 - samples/sec: 121.74 - lr: 0.001172 +2021-03-26 05:37:19,012 epoch 63 - iter 12/25 - loss 3.16441872 - samples/sec: 120.37 - lr: 0.001172 +2021-03-26 05:37:20,024 epoch 63 - iter 14/25 - loss 3.16780548 - samples/sec: 126.63 - lr: 0.001172 +2021-03-26 05:37:20,964 epoch 63 - iter 16/25 - loss 3.11357681 - samples/sec: 136.39 - lr: 0.001172 +2021-03-26 05:37:21,908 epoch 63 - iter 18/25 - loss 3.13789663 - samples/sec: 135.82 - lr: 0.001172 +2021-03-26 05:37:22,882 epoch 63 - iter 20/25 - loss 3.10503579 - samples/sec: 131.60 - lr: 0.001172 +2021-03-26 05:37:23,869 epoch 63 - iter 22/25 - loss 3.10525139 - samples/sec: 129.83 - lr: 0.001172 +2021-03-26 05:37:24,891 epoch 63 - iter 24/25 - loss 3.08538382 - samples/sec: 125.54 - lr: 0.001172 +2021-03-26 05:37:25,262 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:37:25,263 EPOCH 63 done: loss 3.0456 - lr 0.0011719 +2021-03-26 05:37:26,081 DEV : loss 6.38121223449707 - score 0.9067 +2021-03-26 05:37:26,111 BAD EPOCHS (no improvement): 3 +2021-03-26 05:37:26,112 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:37:27,011 epoch 64 - iter 2/25 - loss 2.78713417 - samples/sec: 142.63 - lr: 0.001172 +2021-03-26 05:37:28,020 epoch 64 - iter 4/25 - loss 3.04669195 - samples/sec: 126.99 - lr: 0.001172 +2021-03-26 05:37:29,108 epoch 64 - iter 6/25 - loss 3.15520930 - samples/sec: 117.81 - lr: 0.001172 +2021-03-26 05:37:30,040 epoch 64 - iter 8/25 - loss 3.03692526 - samples/sec: 137.48 - lr: 0.001172 +2021-03-26 05:37:31,237 epoch 64 - iter 10/25 - loss 3.20279713 - samples/sec: 107.03 - lr: 0.001172 +2021-03-26 05:37:32,351 epoch 64 - iter 12/25 - loss 3.16470673 - samples/sec: 115.08 - lr: 0.001172 +2021-03-26 05:37:33,328 epoch 64 - iter 14/25 - loss 3.18147813 - samples/sec: 131.12 - lr: 0.001172 +2021-03-26 05:37:34,319 epoch 64 - iter 16/25 - loss 3.21489021 - samples/sec: 129.42 - lr: 0.001172 +2021-03-26 05:37:35,387 epoch 64 - iter 18/25 - loss 3.15895859 - samples/sec: 120.14 - lr: 0.001172 +2021-03-26 05:37:36,331 epoch 64 - iter 20/25 - loss 3.12339705 - samples/sec: 135.77 - lr: 0.001172 +2021-03-26 05:37:37,332 epoch 64 - iter 22/25 - loss 3.08132965 - samples/sec: 127.99 - lr: 0.001172 +2021-03-26 05:37:38,330 epoch 64 - iter 24/25 - loss 3.05778714 - samples/sec: 128.62 - lr: 0.001172 +2021-03-26 05:37:38,785 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:37:38,785 EPOCH 64 done: loss 3.0483 - lr 0.0011719 +2021-03-26 05:37:39,569 DEV : loss 6.384737968444824 - score 0.9071 +2021-03-26 05:37:39,594 BAD EPOCHS (no improvement): 4 +2021-03-26 05:37:39,595 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:37:40,592 epoch 65 - iter 2/25 - loss 2.51159430 - samples/sec: 128.67 - lr: 0.000586 +2021-03-26 05:37:41,546 epoch 65 - iter 4/25 - loss 2.72224629 - samples/sec: 134.31 - lr: 0.000586 +2021-03-26 05:37:42,488 epoch 65 - iter 6/25 - loss 2.85944200 - samples/sec: 136.11 - lr: 0.000586 +2021-03-26 05:37:43,417 epoch 65 - iter 8/25 - loss 2.79438365 - samples/sec: 138.13 - lr: 0.000586 +2021-03-26 05:37:44,388 epoch 65 - iter 10/25 - loss 2.92464945 - samples/sec: 131.98 - lr: 0.000586 +2021-03-26 05:37:45,512 epoch 65 - iter 12/25 - loss 2.93784885 - samples/sec: 114.03 - lr: 0.000586 +2021-03-26 05:37:46,508 epoch 65 - iter 14/25 - loss 2.94183019 - samples/sec: 128.66 - lr: 0.000586 +2021-03-26 05:37:47,508 epoch 65 - iter 16/25 - loss 2.93601930 - samples/sec: 128.24 - lr: 0.000586 +2021-03-26 05:37:48,508 epoch 65 - iter 18/25 - loss 2.96305468 - samples/sec: 128.09 - lr: 0.000586 +2021-03-26 05:37:49,589 epoch 65 - iter 20/25 - loss 2.99386072 - samples/sec: 118.71 - lr: 0.000586 +2021-03-26 05:37:50,851 epoch 65 - iter 22/25 - loss 2.97353937 - samples/sec: 101.97 - lr: 0.000586 +2021-03-26 05:37:51,905 epoch 65 - iter 24/25 - loss 2.96730961 - samples/sec: 121.63 - lr: 0.000586 +2021-03-26 05:37:52,346 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:37:52,346 EPOCH 65 done: loss 2.9653 - lr 0.0005859 +2021-03-26 05:37:53,121 DEV : loss 6.384479522705078 - score 0.9071 +2021-03-26 05:37:53,147 BAD EPOCHS (no improvement): 1 +2021-03-26 05:37:53,148 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:37:54,108 epoch 66 - iter 2/25 - loss 2.86535895 - samples/sec: 133.54 - lr: 0.000586 +2021-03-26 05:37:55,153 epoch 66 - iter 4/25 - loss 3.12844360 - samples/sec: 122.85 - lr: 0.000586 +2021-03-26 05:37:56,028 epoch 66 - iter 6/25 - loss 3.19749145 - samples/sec: 146.66 - lr: 0.000586 +2021-03-26 05:37:57,019 epoch 66 - iter 8/25 - loss 3.20166433 - samples/sec: 129.42 - lr: 0.000586 +2021-03-26 05:37:58,045 epoch 66 - iter 10/25 - loss 3.20313184 - samples/sec: 124.89 - lr: 0.000586 +2021-03-26 05:37:59,123 epoch 66 - iter 12/25 - loss 3.24322170 - samples/sec: 118.85 - lr: 0.000586 +2021-03-26 05:38:00,118 epoch 66 - iter 14/25 - loss 3.24902667 - samples/sec: 128.97 - lr: 0.000586 +2021-03-26 05:38:01,092 epoch 66 - iter 16/25 - loss 3.24626854 - samples/sec: 131.65 - lr: 0.000586 +2021-03-26 05:38:02,049 epoch 66 - iter 18/25 - loss 3.15644164 - samples/sec: 134.00 - lr: 0.000586 +2021-03-26 05:38:02,984 epoch 66 - iter 20/25 - loss 3.15955758 - samples/sec: 137.25 - lr: 0.000586 +2021-03-26 05:38:03,937 epoch 66 - iter 22/25 - loss 3.14094888 - samples/sec: 134.56 - lr: 0.000586 +2021-03-26 05:38:04,959 epoch 66 - iter 24/25 - loss 3.12368099 - samples/sec: 125.60 - lr: 0.000586 +2021-03-26 05:38:05,393 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:38:05,394 EPOCH 66 done: loss 3.1324 - lr 0.0005859 +2021-03-26 05:38:06,193 DEV : loss 6.383347988128662 - score 0.9071 +2021-03-26 05:38:06,219 BAD EPOCHS (no improvement): 2 +2021-03-26 05:38:06,220 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:38:07,222 epoch 67 - iter 2/25 - loss 2.55367327 - samples/sec: 127.93 - lr: 0.000586 +2021-03-26 05:38:08,192 epoch 67 - iter 4/25 - loss 2.99387980 - samples/sec: 132.23 - lr: 0.000586 +2021-03-26 05:38:09,163 epoch 67 - iter 6/25 - loss 2.84334783 - samples/sec: 132.02 - lr: 0.000586 +2021-03-26 05:38:10,224 epoch 67 - iter 8/25 - loss 2.86086762 - samples/sec: 120.83 - lr: 0.000586 +2021-03-26 05:38:11,237 epoch 67 - iter 10/25 - loss 2.88043385 - samples/sec: 126.68 - lr: 0.000586 +2021-03-26 05:38:12,189 epoch 67 - iter 12/25 - loss 2.88910687 - samples/sec: 134.72 - lr: 0.000586 +2021-03-26 05:38:13,192 epoch 67 - iter 14/25 - loss 2.86610917 - samples/sec: 127.81 - lr: 0.000586 +2021-03-26 05:38:14,202 epoch 67 - iter 16/25 - loss 2.89737588 - samples/sec: 126.87 - lr: 0.000586 +2021-03-26 05:38:15,159 epoch 67 - iter 18/25 - loss 2.94335104 - samples/sec: 133.91 - lr: 0.000586 +2021-03-26 05:38:16,344 epoch 67 - iter 20/25 - loss 2.94453681 - samples/sec: 108.16 - lr: 0.000586 +2021-03-26 05:38:17,347 epoch 67 - iter 22/25 - loss 2.95571147 - samples/sec: 127.73 - lr: 0.000586 +2021-03-26 05:38:18,284 epoch 67 - iter 24/25 - loss 2.99705405 - samples/sec: 136.80 - lr: 0.000586 +2021-03-26 05:38:18,689 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:38:18,690 EPOCH 67 done: loss 3.0371 - lr 0.0005859 +2021-03-26 05:38:19,480 DEV : loss 6.382637977600098 - score 0.9071 +2021-03-26 05:38:19,506 BAD EPOCHS (no improvement): 3 +2021-03-26 05:38:19,507 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:38:20,492 epoch 68 - iter 2/25 - loss 3.10730994 - samples/sec: 130.21 - lr: 0.000586 +2021-03-26 05:38:21,431 epoch 68 - iter 4/25 - loss 2.95590889 - samples/sec: 136.60 - lr: 0.000586 +2021-03-26 05:38:22,477 epoch 68 - iter 6/25 - loss 3.17937148 - samples/sec: 122.65 - lr: 0.000586 +2021-03-26 05:38:23,406 epoch 68 - iter 8/25 - loss 3.15134969 - samples/sec: 137.97 - lr: 0.000586 +2021-03-26 05:38:24,325 epoch 68 - iter 10/25 - loss 3.12095699 - samples/sec: 139.40 - lr: 0.000586 +2021-03-26 05:38:25,360 epoch 68 - iter 12/25 - loss 3.15177401 - samples/sec: 123.89 - lr: 0.000586 +2021-03-26 05:38:26,362 epoch 68 - iter 14/25 - loss 3.06146678 - samples/sec: 127.90 - lr: 0.000586 +2021-03-26 05:38:27,372 epoch 68 - iter 16/25 - loss 3.05155736 - samples/sec: 126.82 - lr: 0.000586 +2021-03-26 05:38:28,280 epoch 68 - iter 18/25 - loss 3.01436926 - samples/sec: 141.24 - lr: 0.000586 +2021-03-26 05:38:29,272 epoch 68 - iter 20/25 - loss 2.95574481 - samples/sec: 129.20 - lr: 0.000586 +2021-03-26 05:38:30,317 epoch 68 - iter 22/25 - loss 2.91949245 - samples/sec: 122.69 - lr: 0.000586 +2021-03-26 05:38:31,374 epoch 68 - iter 24/25 - loss 2.93033016 - samples/sec: 121.20 - lr: 0.000586 +2021-03-26 05:38:31,740 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:38:31,740 EPOCH 68 done: loss 2.9355 - lr 0.0005859 +2021-03-26 05:38:32,512 DEV : loss 6.382974147796631 - score 0.9075 +2021-03-26 05:38:32,538 BAD EPOCHS (no improvement): 4 +2021-03-26 05:38:32,538 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:38:33,505 epoch 69 - iter 2/25 - loss 3.36692905 - samples/sec: 132.58 - lr: 0.000293 +2021-03-26 05:38:34,582 epoch 69 - iter 4/25 - loss 3.05667657 - samples/sec: 119.03 - lr: 0.000293 +2021-03-26 05:38:35,605 epoch 69 - iter 6/25 - loss 2.85133298 - samples/sec: 125.44 - lr: 0.000293 +2021-03-26 05:38:36,648 epoch 69 - iter 8/25 - loss 3.03246859 - samples/sec: 122.90 - lr: 0.000293 +2021-03-26 05:38:37,639 epoch 69 - iter 10/25 - loss 3.05118198 - samples/sec: 129.33 - lr: 0.000293 +2021-03-26 05:38:38,621 epoch 69 - iter 12/25 - loss 3.07936551 - samples/sec: 130.59 - lr: 0.000293 +2021-03-26 05:38:39,694 epoch 69 - iter 14/25 - loss 3.08012353 - samples/sec: 119.44 - lr: 0.000293 +2021-03-26 05:38:40,723 epoch 69 - iter 16/25 - loss 3.05436873 - samples/sec: 124.73 - lr: 0.000293 +2021-03-26 05:38:41,691 epoch 69 - iter 18/25 - loss 3.02919669 - samples/sec: 132.74 - lr: 0.000293 +2021-03-26 05:38:42,782 epoch 69 - iter 20/25 - loss 3.06230100 - samples/sec: 117.41 - lr: 0.000293 +2021-03-26 05:38:43,804 epoch 69 - iter 22/25 - loss 3.05611721 - samples/sec: 125.67 - lr: 0.000293 +2021-03-26 05:38:44,805 epoch 69 - iter 24/25 - loss 3.02325180 - samples/sec: 128.11 - lr: 0.000293 +2021-03-26 05:38:45,197 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:38:45,198 EPOCH 69 done: loss 2.9970 - lr 0.0002930 +2021-03-26 05:38:46,013 DEV : loss 6.38275671005249 - score 0.9079 +2021-03-26 05:38:46,039 BAD EPOCHS (no improvement): 1 +2021-03-26 05:38:46,040 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:38:47,094 epoch 70 - iter 2/25 - loss 3.02799535 - samples/sec: 121.76 - lr: 0.000293 +2021-03-26 05:38:48,094 epoch 70 - iter 4/25 - loss 2.87969232 - samples/sec: 128.27 - lr: 0.000293 +2021-03-26 05:38:49,006 epoch 70 - iter 6/25 - loss 2.79067183 - samples/sec: 140.62 - lr: 0.000293 +2021-03-26 05:38:49,987 epoch 70 - iter 8/25 - loss 2.86862090 - samples/sec: 130.89 - lr: 0.000293 +2021-03-26 05:38:50,945 epoch 70 - iter 10/25 - loss 2.88230951 - samples/sec: 133.74 - lr: 0.000293 +2021-03-26 05:38:51,994 epoch 70 - iter 12/25 - loss 2.93184088 - samples/sec: 122.29 - lr: 0.000293 +2021-03-26 05:38:52,990 epoch 70 - iter 14/25 - loss 2.92722591 - samples/sec: 128.83 - lr: 0.000293 +2021-03-26 05:38:53,917 epoch 70 - iter 16/25 - loss 2.90687205 - samples/sec: 138.33 - lr: 0.000293 +2021-03-26 05:38:54,874 epoch 70 - iter 18/25 - loss 2.88737437 - samples/sec: 133.88 - lr: 0.000293 +2021-03-26 05:38:55,812 epoch 70 - iter 20/25 - loss 2.90604259 - samples/sec: 136.69 - lr: 0.000293 +2021-03-26 05:38:56,785 epoch 70 - iter 22/25 - loss 2.91495455 - samples/sec: 131.71 - lr: 0.000293 +2021-03-26 05:38:57,872 epoch 70 - iter 24/25 - loss 2.93928660 - samples/sec: 117.95 - lr: 0.000293 +2021-03-26 05:38:58,335 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:38:58,336 EPOCH 70 done: loss 2.9512 - lr 0.0002930 +2021-03-26 05:38:59,159 DEV : loss 6.38282585144043 - score 0.9075 +2021-03-26 05:38:59,179 BAD EPOCHS (no improvement): 2 +2021-03-26 05:38:59,179 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:39:00,199 epoch 71 - iter 2/25 - loss 3.33338666 - samples/sec: 125.78 - lr: 0.000293 +2021-03-26 05:39:01,226 epoch 71 - iter 4/25 - loss 3.29438347 - samples/sec: 125.03 - lr: 0.000293 +2021-03-26 05:39:02,194 epoch 71 - iter 6/25 - loss 3.51837246 - samples/sec: 132.36 - lr: 0.000293 +2021-03-26 05:39:03,224 epoch 71 - iter 8/25 - loss 3.15652186 - samples/sec: 124.46 - lr: 0.000293 +2021-03-26 05:39:04,315 epoch 71 - iter 10/25 - loss 3.04163837 - samples/sec: 117.53 - lr: 0.000293 +2021-03-26 05:39:05,358 epoch 71 - iter 12/25 - loss 3.07537399 - samples/sec: 123.09 - lr: 0.000293 +2021-03-26 05:39:06,447 epoch 71 - iter 14/25 - loss 3.09981923 - samples/sec: 117.62 - lr: 0.000293 +2021-03-26 05:39:07,478 epoch 71 - iter 16/25 - loss 3.06394473 - samples/sec: 124.39 - lr: 0.000293 +2021-03-26 05:39:08,442 epoch 71 - iter 18/25 - loss 2.99298355 - samples/sec: 132.98 - lr: 0.000293 +2021-03-26 05:39:09,463 epoch 71 - iter 20/25 - loss 3.02383423 - samples/sec: 125.50 - lr: 0.000293 +2021-03-26 05:39:10,448 epoch 71 - iter 22/25 - loss 2.97648482 - samples/sec: 130.27 - lr: 0.000293 +2021-03-26 05:39:11,363 epoch 71 - iter 24/25 - loss 2.90388579 - samples/sec: 140.01 - lr: 0.000293 +2021-03-26 05:39:11,716 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:39:11,718 EPOCH 71 done: loss 2.8741 - lr 0.0002930 +2021-03-26 05:39:12,534 DEV : loss 6.382222652435303 - score 0.9079 +2021-03-26 05:39:12,560 BAD EPOCHS (no improvement): 3 +2021-03-26 05:39:12,560 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:39:13,492 epoch 72 - iter 2/25 - loss 2.51642132 - samples/sec: 137.58 - lr: 0.000293 +2021-03-26 05:39:14,583 epoch 72 - iter 4/25 - loss 2.58781946 - samples/sec: 117.48 - lr: 0.000293 +2021-03-26 05:39:15,685 epoch 72 - iter 6/25 - loss 2.80074112 - samples/sec: 116.31 - lr: 0.000293 +2021-03-26 05:39:16,671 epoch 72 - iter 8/25 - loss 2.86493906 - samples/sec: 129.94 - lr: 0.000293 +2021-03-26 05:39:17,638 epoch 72 - iter 10/25 - loss 2.81937277 - samples/sec: 132.72 - lr: 0.000293 +2021-03-26 05:39:18,640 epoch 72 - iter 12/25 - loss 2.77459627 - samples/sec: 128.01 - lr: 0.000293 +2021-03-26 05:39:19,736 epoch 72 - iter 14/25 - loss 2.78194279 - samples/sec: 116.93 - lr: 0.000293 +2021-03-26 05:39:20,696 epoch 72 - iter 16/25 - loss 2.89242551 - samples/sec: 133.58 - lr: 0.000293 +2021-03-26 05:39:21,722 epoch 72 - iter 18/25 - loss 2.91030233 - samples/sec: 124.85 - lr: 0.000293 +2021-03-26 05:39:22,669 epoch 72 - iter 20/25 - loss 2.86490841 - samples/sec: 135.49 - lr: 0.000293 +2021-03-26 05:39:23,705 epoch 72 - iter 22/25 - loss 2.91277636 - samples/sec: 123.73 - lr: 0.000293 +2021-03-26 05:39:24,667 epoch 72 - iter 24/25 - loss 2.93769855 - samples/sec: 133.35 - lr: 0.000293 +2021-03-26 05:39:25,085 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:39:25,086 EPOCH 72 done: loss 2.9390 - lr 0.0002930 +2021-03-26 05:39:25,886 DEV : loss 6.382140159606934 - score 0.9079 +2021-03-26 05:39:25,912 BAD EPOCHS (no improvement): 4 +2021-03-26 05:39:25,913 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:39:26,871 epoch 73 - iter 2/25 - loss 3.32987905 - samples/sec: 133.92 - lr: 0.000146 +2021-03-26 05:39:27,962 epoch 73 - iter 4/25 - loss 3.21548235 - samples/sec: 117.60 - lr: 0.000146 +2021-03-26 05:39:29,020 epoch 73 - iter 6/25 - loss 3.13740551 - samples/sec: 121.14 - lr: 0.000146 +2021-03-26 05:39:30,091 epoch 73 - iter 8/25 - loss 3.15512255 - samples/sec: 119.77 - lr: 0.000146 +2021-03-26 05:39:31,171 epoch 73 - iter 10/25 - loss 3.23293400 - samples/sec: 118.88 - lr: 0.000146 +2021-03-26 05:39:32,197 epoch 73 - iter 12/25 - loss 3.15044584 - samples/sec: 125.06 - lr: 0.000146 +2021-03-26 05:39:33,139 epoch 73 - iter 14/25 - loss 3.13014708 - samples/sec: 136.04 - lr: 0.000146 +2021-03-26 05:39:34,193 epoch 73 - iter 16/25 - loss 3.11161730 - samples/sec: 121.54 - lr: 0.000146 +2021-03-26 05:39:35,187 epoch 73 - iter 18/25 - loss 3.10345639 - samples/sec: 129.02 - lr: 0.000146 +2021-03-26 05:39:36,135 epoch 73 - iter 20/25 - loss 3.00746253 - samples/sec: 135.28 - lr: 0.000146 +2021-03-26 05:39:37,089 epoch 73 - iter 22/25 - loss 3.05165423 - samples/sec: 134.50 - lr: 0.000146 +2021-03-26 05:39:38,073 epoch 73 - iter 24/25 - loss 3.08920673 - samples/sec: 130.18 - lr: 0.000146 +2021-03-26 05:39:38,504 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:39:38,505 EPOCH 73 done: loss 3.0432 - lr 0.0001465 +2021-03-26 05:39:39,286 DEV : loss 6.382096290588379 - score 0.9079 +2021-03-26 05:39:39,311 BAD EPOCHS (no improvement): 1 +2021-03-26 05:39:39,312 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:39:40,292 epoch 74 - iter 2/25 - loss 2.67405450 - samples/sec: 130.92 - lr: 0.000146 +2021-03-26 05:39:41,236 epoch 74 - iter 4/25 - loss 2.68950474 - samples/sec: 135.77 - lr: 0.000146 +2021-03-26 05:39:42,206 epoch 74 - iter 6/25 - loss 2.83899264 - samples/sec: 132.15 - lr: 0.000146 +2021-03-26 05:39:43,203 epoch 74 - iter 8/25 - loss 3.01522473 - samples/sec: 128.67 - lr: 0.000146 +2021-03-26 05:39:44,201 epoch 74 - iter 10/25 - loss 3.02439599 - samples/sec: 128.58 - lr: 0.000146 +2021-03-26 05:39:45,258 epoch 74 - iter 12/25 - loss 3.01054655 - samples/sec: 121.44 - lr: 0.000146 +2021-03-26 05:39:46,218 epoch 74 - iter 14/25 - loss 2.94204186 - samples/sec: 133.48 - lr: 0.000146 +2021-03-26 05:39:47,151 epoch 74 - iter 16/25 - loss 2.91268761 - samples/sec: 137.34 - lr: 0.000146 +2021-03-26 05:39:48,101 epoch 74 - iter 18/25 - loss 2.94408124 - samples/sec: 135.23 - lr: 0.000146 +2021-03-26 05:39:49,190 epoch 74 - iter 20/25 - loss 2.96560382 - samples/sec: 117.66 - lr: 0.000146 +2021-03-26 05:39:50,123 epoch 74 - iter 22/25 - loss 3.01023171 - samples/sec: 137.41 - lr: 0.000146 +2021-03-26 05:39:51,093 epoch 74 - iter 24/25 - loss 3.04150907 - samples/sec: 132.27 - lr: 0.000146 +2021-03-26 05:39:51,529 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:39:51,531 EPOCH 74 done: loss 3.0545 - lr 0.0001465 +2021-03-26 05:39:52,333 DEV : loss 6.38180685043335 - score 0.9079 +2021-03-26 05:39:52,362 BAD EPOCHS (no improvement): 2 +2021-03-26 05:39:52,362 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:39:53,381 epoch 75 - iter 2/25 - loss 2.16002882 - samples/sec: 125.80 - lr: 0.000146 +2021-03-26 05:39:54,398 epoch 75 - iter 4/25 - loss 2.68587881 - samples/sec: 126.03 - lr: 0.000146 +2021-03-26 05:39:55,475 epoch 75 - iter 6/25 - loss 2.60181061 - samples/sec: 119.08 - lr: 0.000146 +2021-03-26 05:39:56,405 epoch 75 - iter 8/25 - loss 2.68238357 - samples/sec: 137.75 - lr: 0.000146 +2021-03-26 05:39:57,382 epoch 75 - iter 10/25 - loss 2.71803734 - samples/sec: 131.21 - lr: 0.000146 +2021-03-26 05:39:58,316 epoch 75 - iter 12/25 - loss 2.79610946 - samples/sec: 137.31 - lr: 0.000146 +2021-03-26 05:39:59,316 epoch 75 - iter 14/25 - loss 2.81926460 - samples/sec: 128.11 - lr: 0.000146 +2021-03-26 05:40:00,277 epoch 75 - iter 16/25 - loss 2.81095479 - samples/sec: 133.54 - lr: 0.000146 +2021-03-26 05:40:01,370 epoch 75 - iter 18/25 - loss 2.88618902 - samples/sec: 117.26 - lr: 0.000146 +2021-03-26 05:40:02,382 epoch 75 - iter 20/25 - loss 2.91740313 - samples/sec: 126.69 - lr: 0.000146 +2021-03-26 05:40:03,315 epoch 75 - iter 22/25 - loss 2.92163591 - samples/sec: 137.48 - lr: 0.000146 +2021-03-26 05:40:04,340 epoch 75 - iter 24/25 - loss 2.91205307 - samples/sec: 125.07 - lr: 0.000146 +2021-03-26 05:40:04,764 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:40:04,765 EPOCH 75 done: loss 2.9227 - lr 0.0001465 +2021-03-26 05:40:05,574 DEV : loss 6.382327079772949 - score 0.9079 +2021-03-26 05:40:05,600 BAD EPOCHS (no improvement): 3 +2021-03-26 05:40:05,601 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:40:06,600 epoch 76 - iter 2/25 - loss 3.22646475 - samples/sec: 128.42 - lr: 0.000146 +2021-03-26 05:40:07,624 epoch 76 - iter 4/25 - loss 2.85521507 - samples/sec: 125.13 - lr: 0.000146 +2021-03-26 05:40:08,773 epoch 76 - iter 6/25 - loss 2.90632153 - samples/sec: 111.54 - lr: 0.000146 +2021-03-26 05:40:09,816 epoch 76 - iter 8/25 - loss 2.89671373 - samples/sec: 122.89 - lr: 0.000146 +2021-03-26 05:40:10,797 epoch 76 - iter 10/25 - loss 2.87506793 - samples/sec: 130.77 - lr: 0.000146 +2021-03-26 05:40:11,720 epoch 76 - iter 12/25 - loss 2.86794132 - samples/sec: 138.91 - lr: 0.000146 +2021-03-26 05:40:12,841 epoch 76 - iter 14/25 - loss 2.85794585 - samples/sec: 114.38 - lr: 0.000146 +2021-03-26 05:40:13,865 epoch 76 - iter 16/25 - loss 2.98309749 - samples/sec: 125.23 - lr: 0.000146 +2021-03-26 05:40:14,763 epoch 76 - iter 18/25 - loss 2.96132292 - samples/sec: 142.73 - lr: 0.000146 +2021-03-26 05:40:15,792 epoch 76 - iter 20/25 - loss 2.96512048 - samples/sec: 124.50 - lr: 0.000146 +2021-03-26 05:40:16,785 epoch 76 - iter 22/25 - loss 2.92676656 - samples/sec: 129.11 - lr: 0.000146 +2021-03-26 05:40:17,722 epoch 76 - iter 24/25 - loss 2.92380586 - samples/sec: 136.79 - lr: 0.000146 +2021-03-26 05:40:18,106 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:40:18,107 EPOCH 76 done: loss 2.9335 - lr 0.0001465 +2021-03-26 05:40:18,890 DEV : loss 6.382133483886719 - score 0.9079 +2021-03-26 05:40:18,917 BAD EPOCHS (no improvement): 4 +2021-03-26 05:40:18,917 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:40:18,918 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:40:18,918 learning rate too small - quitting training! +2021-03-26 05:40:18,918 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:40:28,321 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:40:28,321 Testing using best model ... +2021-03-26 05:40:28,322 loading file /home/tmp/megahedm/models/multipos/multipos_UDMADAR_4Diale-LEV_EGY_GLF_MGR__fasttext_flairbwfw__64__0.3_202103260519/best-model.pt +2021-03-26 05:40:35,778 0.9155 +2021-03-26 05:40:35,779 +Results: +- F-score (micro): 0.9114 +- F-score (macro): 0.5279 +- Accuracy (incl. no class): 0.9155 + +By class: + precision recall f1-score support + + NOUN 0.9394 0.9482 0.9438 425 + NUM 1.0000 0.9091 0.9524 22 + PRON 0.9856 0.9856 0.9856 209 + VERB 0.9532 0.9477 0.9504 172 + ADP 0.9444 0.9341 0.9392 91 + CCONJ 0.9762 1.0000 0.9880 82 + PROPN 0.9000 0.8182 0.8571 22 + ADV 0.9310 0.8504 0.8889 127 + DET 0.9574 0.9574 0.9574 47 + PUNCT 1.0000 1.0000 1.0000 25 + ADJ 0.8938 0.8783 0.8860 115 + AUX 0.9111 0.9318 0.9213 44 + PART 0.9242 0.9848 0.9535 198 + SCONJ 0.9762 1.0000 0.9880 41 + INTJ 0.8571 1.0000 0.9231 18 + PREP 0.9531 0.9683 0.9606 63 + NOUN+NSUFF 0.8644 0.8095 0.8361 63 + V+PRON 0.8421 0.7619 0.8000 42 + V+PRON+PRON 0.5000 0.3636 0.4211 11 + ADJ+NSUFF 0.6061 0.7407 0.6667 27 + PREP+V 1.0000 0.5000 0.6667 2 + NOUN+PRON 0.8082 0.9516 0.8741 62 + PREP+PART 0.5000 1.0000 0.6667 1 + V 0.9140 0.9043 0.9091 94 + PROG_PART+V 0.7750 0.9394 0.8493 33 + EOS 1.0000 1.0000 1.0000 70 + MENTION 0.9130 1.0000 0.9545 21 + PART+V 1.0000 0.0000 0.0000 3 + PREP+NOUN 0.7857 0.6111 0.6875 18 + PREP+PRON 0.9091 0.9524 0.9302 21 + PREP+DET+NOUN 0.9000 0.9000 0.9000 10 + URL 1.0000 1.0000 1.0000 3 + FOREIGN 1.0000 0.7500 0.8571 4 + CONJ+NOUN+NSUFF 0.7500 0.7500 0.7500 4 + CONJ+NOUN 0.8333 1.0000 0.9091 10 + DET+NOUN 0.9670 0.9565 0.9617 92 + CONJ+PART 0.9091 0.9091 0.9091 11 + CONJ+V 0.7143 0.6250 0.6667 8 + DET+NOUN+NSUFF 0.8148 0.9565 0.8800 23 + HASH 1.0000 0.8667 0.9286 15 + PUNC 1.0000 1.0000 1.0000 135 + CONJ 0.9459 0.9722 0.9589 36 + NOUN+CASE 1.0000 1.0000 1.0000 4 + CONJ+DET+NOUN 0.7143 1.0000 0.8333 5 + PREP+NOUN+NSUFF+PRON 0.0000 0.0000 0.0000 1 + PREP+NOUN+PRON 0.4286 0.5000 0.4615 6 + FUT_PART+V 1.0000 0.5000 0.6667 6 + CONJ+NOUN+PRON 0.8000 0.6667 0.7273 6 + ADJ+PREP+PRON 0.0000 1.0000 0.0000 0 + CONJ+V+PREP+PRON 1.0000 0.0000 0.0000 1 + CONJ+V+PRON 0.8333 1.0000 0.9091 5 + DET+ADJ 0.8333 0.9091 0.8696 11 + ADJ+PRON 0.5000 0.2500 0.3333 4 + PROG_PART+V+PREP+PRON 0.0000 0.0000 0.0000 3 + NOUN+NSUFF+PRON 0.7778 0.7000 0.7368 10 + PREP+PART+PRON 1.0000 0.6000 0.7500 5 + PREP+PRON+DET+NOUN 1.0000 0.0000 0.0000 1 + CONJ+NUM+NSUFF 1.0000 0.0000 0.0000 1 + CONJ+NUM 0.0000 1.0000 0.0000 0 + PROG_PART+V+PRON 0.6000 0.7500 0.6667 12 + PART+PRON 0.9333 0.8750 0.9032 16 + CONJ+PREP+NOUN+PRON 1.0000 0.0000 0.0000 1 + PART+ADJ 1.0000 0.0000 0.0000 1 + PART+NOUN 0.8000 1.0000 0.8889 4 + CONJ+PREP+NOUN 1.0000 0.0000 0.0000 1 + CONJ+PRON 1.0000 0.7500 0.8571 4 + CONJ+FUT_PART 1.0000 0.0000 0.0000 1 + PART+NOUN+PRON 0.0000 1.0000 0.0000 0 + PART+PREP 1.0000 1.0000 1.0000 2 + PREP+NOUN+NSUFF 0.6000 0.7500 0.6667 4 + PREP+DET+NOUN+NSUFF 0.0000 0.0000 0.0000 1 + V+PRON+PREP+PRON 0.5000 0.5000 0.5000 4 + EMOT 1.0000 0.9333 0.9655 15 + V+PREP+PRON 0.2000 0.3333 0.2500 3 + PART+V+NEG_PART 0.1667 0.3333 0.2222 3 + PROG_PART+V+PRON+PRON 0.0000 0.0000 0.0000 3 + DET+ADJ+NSUFF 1.0000 0.5000 0.6667 4 + FUT_PART 1.0000 1.0000 1.0000 4 + CONJ+DET+ADJ+NSUFF 1.0000 0.0000 0.0000 1 + CONJ+DET+NOUN+NSUFF 1.0000 0.0000 0.0000 1 + CONJ+ADV 1.0000 1.0000 1.0000 1 + PART+PART 1.0000 0.0000 0.0000 2 + CONJ+PREP 1.0000 1.0000 1.0000 3 + PART+PROG_PART 1.0000 0.0000 0.0000 1 + ADJ+CASE 0.5000 1.0000 0.6667 1 + CONJ+PART+ADJ 1.0000 0.0000 0.0000 1 + PROG_PART+V+NEG_PART 1.0000 0.0000 0.0000 1 + PART+V+NOUN 1.0000 0.0000 0.0000 1 + PART+V+PRON 1.0000 1.0000 1.0000 1 + CONJ+PART+PROG_PART+V 1.0000 0.0000 0.0000 1 + CONJ+PROG_PART+V 0.0000 1.0000 0.0000 0 + CONJ+PART+V+NEG_PART 1.0000 0.0000 0.0000 1 + CONJ+FUT_PART+V 0.0000 1.0000 0.0000 0 + PART+V+PREP+PRON+NEG_PART 1.0000 0.0000 0.0000 1 + PART+PROG_PART+V+NEG_PART 0.0000 0.0000 0.0000 3 + PART+V+PRON+NEG_PART 0.5000 0.4000 0.4444 5 + PART+PREP+NEG_PART 1.0000 1.0000 1.0000 3 + NOUN+CASE+PRON 0.0000 1.0000 0.0000 0 + PART+PREP+PRON+NEG_PART 0.0000 1.0000 0.0000 0 + DET+NUM 1.0000 0.0000 0.0000 1 + FUT_PART+V+PRON+PRON 1.0000 0.0000 0.0000 1 + FUT_PART+V+PRON 0.0000 1.0000 0.0000 0 +PART+PROG_PART+V+PRON+NEG_PART 1.0000 0.0000 0.0000 1 + PART+NOUN+PRON+NEG_PART 1.0000 0.0000 0.0000 1 + PART+NOUN+NEG_PART 1.0000 1.0000 1.0000 1 + CONJ+ADJ+NSUFF 0.0000 0.0000 0.0000 1 + PREP+ADJ 1.0000 0.0000 0.0000 1 + NOUN+CASE+NSUFF+NOUN+PRON 1.0000 0.0000 0.0000 1 + V+NEG_PART 1.0000 0.0000 0.0000 1 + ADJ+NSUFF+PRON 1.0000 0.0000 0.0000 1 + + micro avg 0.9114 0.9114 0.9114 2710 + macro avg 0.7731 0.6053 0.5279 2710 + weighted avg 0.9186 0.9114 0.9080 2710 + +2021-03-26 05:40:35,779 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:40:35,779 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:40:41,356 Reading data from ../../Datasets_adhoc/CSCS_corpus-GUC +2021-03-26 05:40:41,357 Train: ../../Datasets_adhoc/CSCS_corpus-GUC/all_participants.conllu +2021-03-26 05:40:41,357 Dev: None +2021-03-26 05:40:41,357 Test: None +2021-03-26 05:40:41,619 Reading data from ../../Datasets_adhoc/UD_MADAR +2021-03-26 05:40:41,620 Train: ../../Datasets_adhoc/UD_MADAR/ajp_madar-ud-test-edit.conllu +2021-03-26 05:40:41,620 Dev: None +2021-03-26 05:40:41,620 Test: None +2021-03-26 05:40:41,660 Reading data from ../../Datasets_adhoc/dialectal_arabic_resources-master +2021-03-26 05:40:41,661 Train: ../../Datasets_adhoc/dialectal_arabic_resources-master/seg_plus_pos_lev.txt +2021-03-26 05:40:41,661 Dev: None +2021-03-26 05:40:41,661 Test: None +2021-03-26 05:40:41,813 Reading data from ../../Datasets_adhoc/dialectal_arabic_resources-master +2021-03-26 05:40:41,813 Train: ../../Datasets_adhoc/dialectal_arabic_resources-master/seg_plus_pos_egy.txt +2021-03-26 05:40:41,813 Dev: None +2021-03-26 05:40:41,814 Test: None +2021-03-26 05:40:41,985 Reading data from ../../Datasets_adhoc/dialectal_arabic_resources-master +2021-03-26 05:40:41,985 Train: ../../Datasets_adhoc/dialectal_arabic_resources-master/seg_plus_pos_glf.txt +2021-03-26 05:40:41,986 Dev: None +2021-03-26 05:40:41,986 Test: None +2021-03-26 05:40:42,149 Reading data from ../../Datasets_adhoc/dialectal_arabic_resources-master +2021-03-26 05:40:42,149 Train: ../../Datasets_adhoc/dialectal_arabic_resources-master/seg_plus_pos_mgr.txt +2021-03-26 05:40:42,150 Dev: None +2021-03-26 05:40:42,150 Test: None +2021-03-26 05:40:42,309 Filtering long sentences +2021-03-26 05:40:42,352 MultiCorpus: 1574 train + 176 dev + 194 test sentences + - ColumnCorpus Corpus: 934 train + 104 dev + 115 test sentences + - ColumnCorpus Corpus: 81 train + 9 dev + 10 test sentences + - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences + - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences +2021-03-26 05:40:42,752 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:40:42,753 Model: "SequenceTagger( + (embeddings): StackedEmbeddings( + (list_embedding_0): WordEmbeddings('ar') + (list_embedding_1): FlairEmbeddings( + (lm): LanguageModel( + (drop): Dropout(p=0.1, inplace=False) + (encoder): Embedding(7125, 100) + (rnn): LSTM(100, 2048) + (decoder): Linear(in_features=2048, out_features=7125, bias=True) + ) + ) + (list_embedding_2): FlairEmbeddings( + (lm): LanguageModel( + (drop): Dropout(p=0.1, inplace=False) + (encoder): Embedding(7125, 100) + (rnn): LSTM(100, 2048) + (decoder): Linear(in_features=2048, out_features=7125, bias=True) + ) + ) + ) + (word_dropout): WordDropout(p=0.05) + (locked_dropout): LockedDropout(p=0.5) + (embedding2nn): Linear(in_features=4396, out_features=4396, bias=True) + (rnn): LSTM(4396, 256, batch_first=True, bidirectional=True) + (linear): Linear(in_features=512, out_features=206, bias=True) + (beta): 1.0 + (weights): None + (weight_tensor) None +)" +2021-03-26 05:40:42,753 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:40:42,754 Corpus: "MultiCorpus: 1574 train + 176 dev + 194 test sentences + - ColumnCorpus Corpus: 934 train + 104 dev + 115 test sentences + - ColumnCorpus Corpus: 81 train + 9 dev + 10 test sentences + - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences + - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences" +2021-03-26 05:40:42,754 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:40:42,754 Parameters: +2021-03-26 05:40:42,754 - learning_rate: "0.3" +2021-03-26 05:40:42,755 - mini_batch_size: "64" +2021-03-26 05:40:42,755 - patience: "3" +2021-03-26 05:40:42,755 - anneal_factor: "0.5" +2021-03-26 05:40:42,755 - max_epochs: "150" +2021-03-26 05:40:42,756 - shuffle: "True" +2021-03-26 05:40:42,756 - train_with_dev: "False" +2021-03-26 05:40:42,756 - batch_growth_annealing: "False" +2021-03-26 05:40:42,757 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:40:42,757 Model training base path: "/home/tmp/megahedm/models/multipos/multipos_UDMADAR_4Diale-LEV_EGY_GLF_MGR__fasttext_flairbwfw__64__0.3_202103260540" +2021-03-26 05:40:42,757 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:40:42,757 Device: cuda:0 +2021-03-26 05:40:42,758 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:40:42,758 Embeddings storage mode: cpu +2021-03-26 05:40:42,759 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:40:44,574 epoch 1 - iter 2/25 - loss 88.64369583 - samples/sec: 70.58 - lr: 0.300000 +2021-03-26 05:40:45,955 epoch 1 - iter 4/25 - loss 78.67724133 - samples/sec: 92.82 - lr: 0.300000 +2021-03-26 05:40:47,222 epoch 1 - iter 6/25 - loss 75.12040520 - samples/sec: 101.11 - lr: 0.300000 +2021-03-26 05:40:48,480 epoch 1 - iter 8/25 - loss 71.83301973 - samples/sec: 101.86 - lr: 0.300000 +2021-03-26 05:40:49,869 epoch 1 - iter 10/25 - loss 70.04709473 - samples/sec: 92.21 - lr: 0.300000 +2021-03-26 05:40:52,337 epoch 1 - iter 12/25 - loss 67.97456614 - samples/sec: 51.91 - lr: 0.300000 +2021-03-26 05:40:53,686 epoch 1 - iter 14/25 - loss 66.16142763 - samples/sec: 94.99 - lr: 0.300000 +2021-03-26 05:40:55,088 epoch 1 - iter 16/25 - loss 64.42413473 - samples/sec: 91.40 - lr: 0.300000 +2021-03-26 05:40:56,535 epoch 1 - iter 18/25 - loss 62.82821062 - samples/sec: 88.56 - lr: 0.300000 +2021-03-26 05:40:57,937 epoch 1 - iter 20/25 - loss 61.43530502 - samples/sec: 91.40 - lr: 0.300000 +2021-03-26 05:40:59,282 epoch 1 - iter 22/25 - loss 59.67912865 - samples/sec: 95.27 - lr: 0.300000 +2021-03-26 05:41:00,535 epoch 1 - iter 24/25 - loss 58.65658077 - samples/sec: 102.22 - lr: 0.300000 +2021-03-26 05:41:01,124 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:41:01,125 EPOCH 1 done: loss 58.1316 - lr 0.3000000 +2021-03-26 05:41:02,400 DEV : loss 44.08968734741211 - score 0.319 +2021-03-26 05:41:02,426 BAD EPOCHS (no improvement): 0 +2021-03-26 05:41:12,015 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:41:13,293 epoch 2 - iter 2/25 - loss 45.49032211 - samples/sec: 100.36 - lr: 0.300000 +2021-03-26 05:41:14,221 epoch 2 - iter 4/25 - loss 46.81260490 - samples/sec: 138.31 - lr: 0.300000 +2021-03-26 05:41:15,195 epoch 2 - iter 6/25 - loss 43.56631343 - samples/sec: 131.59 - lr: 0.300000 +2021-03-26 05:41:16,116 epoch 2 - iter 8/25 - loss 42.27197218 - samples/sec: 139.35 - lr: 0.300000 +2021-03-26 05:41:17,051 epoch 2 - iter 10/25 - loss 40.86771469 - samples/sec: 137.18 - lr: 0.300000 +2021-03-26 05:41:18,114 epoch 2 - iter 12/25 - loss 39.77363968 - samples/sec: 120.63 - lr: 0.300000 +2021-03-26 05:41:19,130 epoch 2 - iter 14/25 - loss 39.80917222 - samples/sec: 126.25 - lr: 0.300000 +2021-03-26 05:41:20,100 epoch 2 - iter 16/25 - loss 39.00103319 - samples/sec: 132.27 - lr: 0.300000 +2021-03-26 05:41:21,055 epoch 2 - iter 18/25 - loss 38.09381591 - samples/sec: 134.15 - lr: 0.300000 +2021-03-26 05:41:21,991 epoch 2 - iter 20/25 - loss 37.36236010 - samples/sec: 137.06 - lr: 0.300000 +2021-03-26 05:41:23,050 epoch 2 - iter 22/25 - loss 36.81540619 - samples/sec: 121.01 - lr: 0.300000 +2021-03-26 05:41:24,160 epoch 2 - iter 24/25 - loss 36.91029286 - samples/sec: 115.48 - lr: 0.300000 +2021-03-26 05:41:24,581 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:41:24,582 EPOCH 2 done: loss 36.7035 - lr 0.3000000 +2021-03-26 05:41:25,367 DEV : loss 31.3277645111084 - score 0.5139 +2021-03-26 05:41:25,390 BAD EPOCHS (no improvement): 0 +2021-03-26 05:41:35,160 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:41:36,210 epoch 3 - iter 2/25 - loss 33.64617920 - samples/sec: 122.03 - lr: 0.300000 +2021-03-26 05:41:37,266 epoch 3 - iter 4/25 - loss 29.12564039 - samples/sec: 121.46 - lr: 0.300000 +2021-03-26 05:41:38,298 epoch 3 - iter 6/25 - loss 29.82938671 - samples/sec: 124.21 - lr: 0.300000 +2021-03-26 05:41:39,277 epoch 3 - iter 8/25 - loss 29.37328362 - samples/sec: 130.98 - lr: 0.300000 +2021-03-26 05:41:40,286 epoch 3 - iter 10/25 - loss 28.70174255 - samples/sec: 126.98 - lr: 0.300000 +2021-03-26 05:41:41,223 epoch 3 - iter 12/25 - loss 28.44004679 - samples/sec: 136.95 - lr: 0.300000 +2021-03-26 05:41:42,218 epoch 3 - iter 14/25 - loss 28.44743238 - samples/sec: 128.82 - lr: 0.300000 +2021-03-26 05:41:43,188 epoch 3 - iter 16/25 - loss 27.92235672 - samples/sec: 132.22 - lr: 0.300000 +2021-03-26 05:41:44,221 epoch 3 - iter 18/25 - loss 27.86411593 - samples/sec: 124.04 - lr: 0.300000 +2021-03-26 05:41:45,210 epoch 3 - iter 20/25 - loss 27.75742226 - samples/sec: 129.62 - lr: 0.300000 +2021-03-26 05:41:46,376 epoch 3 - iter 22/25 - loss 27.58030839 - samples/sec: 109.93 - lr: 0.300000 +2021-03-26 05:41:47,380 epoch 3 - iter 24/25 - loss 27.43895706 - samples/sec: 127.80 - lr: 0.300000 +2021-03-26 05:41:47,785 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:41:47,786 EPOCH 3 done: loss 27.1920 - lr 0.3000000 +2021-03-26 05:41:48,593 DEV : loss 22.79669189453125 - score 0.6489 +2021-03-26 05:41:48,619 BAD EPOCHS (no improvement): 0 +2021-03-26 05:41:58,360 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:41:59,359 epoch 4 - iter 2/25 - loss 22.24100590 - samples/sec: 128.43 - lr: 0.300000 +2021-03-26 05:42:00,348 epoch 4 - iter 4/25 - loss 21.90831089 - samples/sec: 129.61 - lr: 0.300000 +2021-03-26 05:42:01,456 epoch 4 - iter 6/25 - loss 23.29270585 - samples/sec: 115.69 - lr: 0.300000 +2021-03-26 05:42:02,415 epoch 4 - iter 8/25 - loss 23.13200498 - samples/sec: 133.65 - lr: 0.300000 +2021-03-26 05:42:03,346 epoch 4 - iter 10/25 - loss 23.02010689 - samples/sec: 137.73 - lr: 0.300000 +2021-03-26 05:42:04,364 epoch 4 - iter 12/25 - loss 22.89023797 - samples/sec: 125.93 - lr: 0.300000 +2021-03-26 05:42:05,402 epoch 4 - iter 14/25 - loss 23.05532360 - samples/sec: 123.46 - lr: 0.300000 +2021-03-26 05:42:06,479 epoch 4 - iter 16/25 - loss 23.22665524 - samples/sec: 119.00 - lr: 0.300000 +2021-03-26 05:42:07,364 epoch 4 - iter 18/25 - loss 22.73062568 - samples/sec: 144.81 - lr: 0.300000 +2021-03-26 05:42:08,340 epoch 4 - iter 20/25 - loss 22.55190754 - samples/sec: 131.33 - lr: 0.300000 +2021-03-26 05:42:09,495 epoch 4 - iter 22/25 - loss 22.42480538 - samples/sec: 110.97 - lr: 0.300000 +2021-03-26 05:42:10,548 epoch 4 - iter 24/25 - loss 22.38524278 - samples/sec: 121.73 - lr: 0.300000 +2021-03-26 05:42:11,025 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:42:11,025 EPOCH 4 done: loss 22.1926 - lr 0.3000000 +2021-03-26 05:42:11,839 DEV : loss 17.91794204711914 - score 0.7292 +2021-03-26 05:42:11,862 BAD EPOCHS (no improvement): 0 +2021-03-26 05:42:21,724 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:42:22,788 epoch 5 - iter 2/25 - loss 18.36693954 - samples/sec: 120.56 - lr: 0.300000 +2021-03-26 05:42:23,768 epoch 5 - iter 4/25 - loss 18.09573936 - samples/sec: 131.00 - lr: 0.300000 +2021-03-26 05:42:24,762 epoch 5 - iter 6/25 - loss 17.68604310 - samples/sec: 128.88 - lr: 0.300000 +2021-03-26 05:42:25,791 epoch 5 - iter 8/25 - loss 18.63525987 - samples/sec: 124.45 - lr: 0.300000 +2021-03-26 05:42:26,695 epoch 5 - iter 10/25 - loss 18.71813297 - samples/sec: 142.48 - lr: 0.300000 +2021-03-26 05:42:27,701 epoch 5 - iter 12/25 - loss 18.54522006 - samples/sec: 127.54 - lr: 0.300000 +2021-03-26 05:42:28,692 epoch 5 - iter 14/25 - loss 18.73616600 - samples/sec: 129.53 - lr: 0.300000 +2021-03-26 05:42:29,767 epoch 5 - iter 16/25 - loss 18.44975418 - samples/sec: 119.20 - lr: 0.300000 +2021-03-26 05:42:30,710 epoch 5 - iter 18/25 - loss 18.59771268 - samples/sec: 135.96 - lr: 0.300000 +2021-03-26 05:42:31,781 epoch 5 - iter 20/25 - loss 18.54842658 - samples/sec: 119.61 - lr: 0.300000 +2021-03-26 05:42:32,744 epoch 5 - iter 22/25 - loss 18.40371821 - samples/sec: 133.15 - lr: 0.300000 +2021-03-26 05:42:33,699 epoch 5 - iter 24/25 - loss 18.12898660 - samples/sec: 134.20 - lr: 0.300000 +2021-03-26 05:42:34,145 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:42:34,146 EPOCH 5 done: loss 18.2116 - lr 0.3000000 +2021-03-26 05:42:34,937 DEV : loss 15.187065124511719 - score 0.7554 +2021-03-26 05:42:34,962 BAD EPOCHS (no improvement): 0 +2021-03-26 05:42:44,840 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:42:45,857 epoch 6 - iter 2/25 - loss 16.13490629 - samples/sec: 126.10 - lr: 0.300000 +2021-03-26 05:42:46,823 epoch 6 - iter 4/25 - loss 15.87294555 - samples/sec: 132.95 - lr: 0.300000 +2021-03-26 05:42:47,793 epoch 6 - iter 6/25 - loss 16.28924131 - samples/sec: 132.32 - lr: 0.300000 +2021-03-26 05:42:48,742 epoch 6 - iter 8/25 - loss 15.93548644 - samples/sec: 134.96 - lr: 0.300000 +2021-03-26 05:42:49,689 epoch 6 - iter 10/25 - loss 16.17079515 - samples/sec: 135.52 - lr: 0.300000 +2021-03-26 05:42:50,735 epoch 6 - iter 12/25 - loss 15.89895153 - samples/sec: 122.60 - lr: 0.300000 +2021-03-26 05:42:51,805 epoch 6 - iter 14/25 - loss 15.59099872 - samples/sec: 119.76 - lr: 0.300000 +2021-03-26 05:42:52,797 epoch 6 - iter 16/25 - loss 15.44279909 - samples/sec: 129.43 - lr: 0.300000 +2021-03-26 05:42:53,825 epoch 6 - iter 18/25 - loss 15.70377202 - samples/sec: 124.70 - lr: 0.300000 +2021-03-26 05:42:54,955 epoch 6 - iter 20/25 - loss 15.55591607 - samples/sec: 113.46 - lr: 0.300000 +2021-03-26 05:42:55,851 epoch 6 - iter 22/25 - loss 15.54194147 - samples/sec: 143.21 - lr: 0.300000 +2021-03-26 05:42:56,945 epoch 6 - iter 24/25 - loss 15.66215499 - samples/sec: 117.09 - lr: 0.300000 +2021-03-26 05:42:57,368 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:42:57,369 EPOCH 6 done: loss 15.8092 - lr 0.3000000 +2021-03-26 05:42:58,145 DEV : loss 14.743255615234375 - score 0.7597 +2021-03-26 05:42:58,165 BAD EPOCHS (no improvement): 0 +2021-03-26 05:43:08,028 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:43:09,057 epoch 7 - iter 2/25 - loss 18.11933041 - samples/sec: 124.73 - lr: 0.300000 +2021-03-26 05:43:09,997 epoch 7 - iter 4/25 - loss 16.30117321 - samples/sec: 136.42 - lr: 0.300000 +2021-03-26 05:43:11,305 epoch 7 - iter 6/25 - loss 15.62002357 - samples/sec: 97.93 - lr: 0.300000 +2021-03-26 05:43:12,397 epoch 7 - iter 8/25 - loss 15.28432643 - samples/sec: 117.40 - lr: 0.300000 +2021-03-26 05:43:13,491 epoch 7 - iter 10/25 - loss 15.35674391 - samples/sec: 117.19 - lr: 0.300000 +2021-03-26 05:43:14,526 epoch 7 - iter 12/25 - loss 15.29835645 - samples/sec: 123.82 - lr: 0.300000 +2021-03-26 05:43:15,600 epoch 7 - iter 14/25 - loss 15.13747549 - samples/sec: 119.34 - lr: 0.300000 +2021-03-26 05:43:16,588 epoch 7 - iter 16/25 - loss 15.07328975 - samples/sec: 130.04 - lr: 0.300000 +2021-03-26 05:43:17,676 epoch 7 - iter 18/25 - loss 14.88522440 - samples/sec: 117.79 - lr: 0.300000 +2021-03-26 05:43:18,673 epoch 7 - iter 20/25 - loss 14.82129803 - samples/sec: 128.68 - lr: 0.300000 +2021-03-26 05:43:19,643 epoch 7 - iter 22/25 - loss 14.80583009 - samples/sec: 132.12 - lr: 0.300000 +2021-03-26 05:43:20,617 epoch 7 - iter 24/25 - loss 14.60828722 - samples/sec: 131.68 - lr: 0.300000 +2021-03-26 05:43:21,020 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:43:21,021 EPOCH 7 done: loss 14.5242 - lr 0.3000000 +2021-03-26 05:43:21,793 DEV : loss 12.157855033874512 - score 0.8015 +2021-03-26 05:43:21,818 BAD EPOCHS (no improvement): 0 +2021-03-26 05:43:31,696 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:43:32,747 epoch 8 - iter 2/25 - loss 11.99634266 - samples/sec: 122.05 - lr: 0.300000 +2021-03-26 05:43:33,751 epoch 8 - iter 4/25 - loss 12.01598549 - samples/sec: 127.66 - lr: 0.300000 +2021-03-26 05:43:34,778 epoch 8 - iter 6/25 - loss 12.23479764 - samples/sec: 124.88 - lr: 0.300000 +2021-03-26 05:43:35,847 epoch 8 - iter 8/25 - loss 12.58600891 - samples/sec: 119.95 - lr: 0.300000 +2021-03-26 05:43:36,884 epoch 8 - iter 10/25 - loss 12.72332258 - samples/sec: 123.62 - lr: 0.300000 +2021-03-26 05:43:38,058 epoch 8 - iter 12/25 - loss 12.66574216 - samples/sec: 109.13 - lr: 0.300000 +2021-03-26 05:43:39,024 epoch 8 - iter 14/25 - loss 12.68500212 - samples/sec: 132.88 - lr: 0.300000 +2021-03-26 05:43:40,057 epoch 8 - iter 16/25 - loss 12.81532699 - samples/sec: 124.04 - lr: 0.300000 +2021-03-26 05:43:41,106 epoch 8 - iter 18/25 - loss 12.93446302 - samples/sec: 122.14 - lr: 0.300000 +2021-03-26 05:43:42,093 epoch 8 - iter 20/25 - loss 12.94684072 - samples/sec: 130.16 - lr: 0.300000 +2021-03-26 05:43:43,106 epoch 8 - iter 22/25 - loss 13.04992381 - samples/sec: 126.44 - lr: 0.300000 +2021-03-26 05:43:44,042 epoch 8 - iter 24/25 - loss 12.85443902 - samples/sec: 137.07 - lr: 0.300000 +2021-03-26 05:43:44,491 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:43:44,492 EPOCH 8 done: loss 12.8589 - lr 0.3000000 +2021-03-26 05:43:45,261 DEV : loss 11.195834159851074 - score 0.8192 +2021-03-26 05:43:45,281 BAD EPOCHS (no improvement): 0 +2021-03-26 05:43:54,888 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:43:55,945 epoch 9 - iter 2/25 - loss 12.01339865 - samples/sec: 121.34 - lr: 0.300000 +2021-03-26 05:43:57,004 epoch 9 - iter 4/25 - loss 12.19045806 - samples/sec: 121.05 - lr: 0.300000 +2021-03-26 05:43:58,061 epoch 9 - iter 6/25 - loss 11.92286936 - samples/sec: 121.42 - lr: 0.300000 +2021-03-26 05:43:58,999 epoch 9 - iter 8/25 - loss 12.26701045 - samples/sec: 136.70 - lr: 0.300000 +2021-03-26 05:44:00,040 epoch 9 - iter 10/25 - loss 12.15806046 - samples/sec: 123.15 - lr: 0.300000 +2021-03-26 05:44:01,017 epoch 9 - iter 12/25 - loss 12.16244268 - samples/sec: 131.21 - lr: 0.300000 +2021-03-26 05:44:01,965 epoch 9 - iter 14/25 - loss 12.17545857 - samples/sec: 135.17 - lr: 0.300000 +2021-03-26 05:44:02,965 epoch 9 - iter 16/25 - loss 12.02299416 - samples/sec: 128.17 - lr: 0.300000 +2021-03-26 05:44:04,124 epoch 9 - iter 18/25 - loss 12.25711123 - samples/sec: 110.49 - lr: 0.300000 +2021-03-26 05:44:05,086 epoch 9 - iter 20/25 - loss 12.06897812 - samples/sec: 133.46 - lr: 0.300000 +2021-03-26 05:44:06,318 epoch 9 - iter 22/25 - loss 11.92144212 - samples/sec: 104.08 - lr: 0.300000 +2021-03-26 05:44:07,365 epoch 9 - iter 24/25 - loss 11.91666170 - samples/sec: 122.50 - lr: 0.300000 +2021-03-26 05:44:07,806 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:44:07,806 EPOCH 9 done: loss 11.8426 - lr 0.3000000 +2021-03-26 05:44:08,597 DEV : loss 11.0265531539917 - score 0.8206 +2021-03-26 05:44:08,626 BAD EPOCHS (no improvement): 0 +2021-03-26 05:44:18,228 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:44:19,477 epoch 10 - iter 2/25 - loss 11.34327888 - samples/sec: 102.68 - lr: 0.300000 +2021-03-26 05:44:20,455 epoch 10 - iter 4/25 - loss 11.71344209 - samples/sec: 131.17 - lr: 0.300000 +2021-03-26 05:44:21,429 epoch 10 - iter 6/25 - loss 11.20282396 - samples/sec: 131.54 - lr: 0.300000 +2021-03-26 05:44:22,468 epoch 10 - iter 8/25 - loss 11.10128748 - samples/sec: 123.46 - lr: 0.300000 +2021-03-26 05:44:23,485 epoch 10 - iter 10/25 - loss 10.91363373 - samples/sec: 126.05 - lr: 0.300000 +2021-03-26 05:44:24,582 epoch 10 - iter 12/25 - loss 10.88611531 - samples/sec: 116.79 - lr: 0.300000 +2021-03-26 05:44:25,566 epoch 10 - iter 14/25 - loss 10.59839494 - samples/sec: 130.37 - lr: 0.300000 +2021-03-26 05:44:26,641 epoch 10 - iter 16/25 - loss 10.86341327 - samples/sec: 119.11 - lr: 0.300000 +2021-03-26 05:44:27,623 epoch 10 - iter 18/25 - loss 10.94241174 - samples/sec: 130.58 - lr: 0.300000 +2021-03-26 05:44:28,671 epoch 10 - iter 20/25 - loss 10.91952567 - samples/sec: 122.37 - lr: 0.300000 +2021-03-26 05:44:29,678 epoch 10 - iter 22/25 - loss 10.93466590 - samples/sec: 127.25 - lr: 0.300000 +2021-03-26 05:44:30,656 epoch 10 - iter 24/25 - loss 11.03978912 - samples/sec: 131.27 - lr: 0.300000 +2021-03-26 05:44:31,060 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:44:31,061 EPOCH 10 done: loss 10.9988 - lr 0.3000000 +2021-03-26 05:44:31,840 DEV : loss 10.144160270690918 - score 0.8356 +2021-03-26 05:44:31,863 BAD EPOCHS (no improvement): 0 +2021-03-26 05:44:41,520 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:44:42,617 epoch 11 - iter 2/25 - loss 10.94504309 - samples/sec: 116.96 - lr: 0.300000 +2021-03-26 05:44:43,667 epoch 11 - iter 4/25 - loss 11.22785974 - samples/sec: 122.18 - lr: 0.300000 +2021-03-26 05:44:44,765 epoch 11 - iter 6/25 - loss 11.41904799 - samples/sec: 116.63 - lr: 0.300000 +2021-03-26 05:44:45,811 epoch 11 - iter 8/25 - loss 11.12639284 - samples/sec: 122.71 - lr: 0.300000 +2021-03-26 05:44:46,856 epoch 11 - iter 10/25 - loss 11.28100567 - samples/sec: 122.59 - lr: 0.300000 +2021-03-26 05:44:47,977 epoch 11 - iter 12/25 - loss 11.00330440 - samples/sec: 114.46 - lr: 0.300000 +2021-03-26 05:44:49,008 epoch 11 - iter 14/25 - loss 10.97292021 - samples/sec: 124.53 - lr: 0.300000 +2021-03-26 05:44:50,077 epoch 11 - iter 16/25 - loss 10.63795060 - samples/sec: 119.83 - lr: 0.300000 +2021-03-26 05:44:51,066 epoch 11 - iter 18/25 - loss 10.61853668 - samples/sec: 129.69 - lr: 0.300000 +2021-03-26 05:44:51,989 epoch 11 - iter 20/25 - loss 10.53863492 - samples/sec: 138.88 - lr: 0.300000 +2021-03-26 05:44:52,966 epoch 11 - iter 22/25 - loss 10.60792611 - samples/sec: 131.20 - lr: 0.300000 +2021-03-26 05:44:53,908 epoch 11 - iter 24/25 - loss 10.64770297 - samples/sec: 135.97 - lr: 0.300000 +2021-03-26 05:44:54,259 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:44:54,260 EPOCH 11 done: loss 10.5094 - lr 0.3000000 +2021-03-26 05:44:55,041 DEV : loss 9.662548065185547 - score 0.842 +2021-03-26 05:44:55,063 BAD EPOCHS (no improvement): 0 +2021-03-26 05:45:04,767 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:45:05,837 epoch 12 - iter 2/25 - loss 10.17411470 - samples/sec: 119.89 - lr: 0.300000 +2021-03-26 05:45:06,893 epoch 12 - iter 4/25 - loss 9.86866117 - samples/sec: 121.47 - lr: 0.300000 +2021-03-26 05:45:07,879 epoch 12 - iter 6/25 - loss 10.10727294 - samples/sec: 130.06 - lr: 0.300000 +2021-03-26 05:45:08,896 epoch 12 - iter 8/25 - loss 9.61317658 - samples/sec: 125.98 - lr: 0.300000 +2021-03-26 05:45:09,979 epoch 12 - iter 10/25 - loss 9.45750113 - samples/sec: 118.34 - lr: 0.300000 +2021-03-26 05:45:11,065 epoch 12 - iter 12/25 - loss 9.53312612 - samples/sec: 118.03 - lr: 0.300000 +2021-03-26 05:45:12,024 epoch 12 - iter 14/25 - loss 9.49966478 - samples/sec: 133.62 - lr: 0.300000 +2021-03-26 05:45:13,058 epoch 12 - iter 16/25 - loss 9.55181575 - samples/sec: 124.01 - lr: 0.300000 +2021-03-26 05:45:14,023 epoch 12 - iter 18/25 - loss 9.56000879 - samples/sec: 132.73 - lr: 0.300000 +2021-03-26 05:45:14,936 epoch 12 - iter 20/25 - loss 9.61673174 - samples/sec: 140.71 - lr: 0.300000 +2021-03-26 05:45:16,021 epoch 12 - iter 22/25 - loss 9.69639319 - samples/sec: 118.08 - lr: 0.300000 +2021-03-26 05:45:17,042 epoch 12 - iter 24/25 - loss 9.75539422 - samples/sec: 125.71 - lr: 0.300000 +2021-03-26 05:45:17,450 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:45:17,450 EPOCH 12 done: loss 9.7850 - lr 0.3000000 +2021-03-26 05:45:18,218 DEV : loss 8.691134452819824 - score 0.8552 +2021-03-26 05:45:18,244 BAD EPOCHS (no improvement): 0 +2021-03-26 05:45:27,952 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:45:29,023 epoch 13 - iter 2/25 - loss 9.57857800 - samples/sec: 119.78 - lr: 0.300000 +2021-03-26 05:45:30,085 epoch 13 - iter 4/25 - loss 9.67772269 - samples/sec: 120.75 - lr: 0.300000 +2021-03-26 05:45:31,154 epoch 13 - iter 6/25 - loss 9.08809503 - samples/sec: 119.94 - lr: 0.300000 +2021-03-26 05:45:32,351 epoch 13 - iter 8/25 - loss 8.77041090 - samples/sec: 107.04 - lr: 0.300000 +2021-03-26 05:45:33,301 epoch 13 - iter 10/25 - loss 8.77164540 - samples/sec: 134.94 - lr: 0.300000 +2021-03-26 05:45:34,298 epoch 13 - iter 12/25 - loss 8.93865446 - samples/sec: 128.70 - lr: 0.300000 +2021-03-26 05:45:35,393 epoch 13 - iter 14/25 - loss 9.02500653 - samples/sec: 117.03 - lr: 0.300000 +2021-03-26 05:45:36,370 epoch 13 - iter 16/25 - loss 8.95800772 - samples/sec: 131.27 - lr: 0.300000 +2021-03-26 05:45:37,341 epoch 13 - iter 18/25 - loss 8.78139753 - samples/sec: 131.90 - lr: 0.300000 +2021-03-26 05:45:38,390 epoch 13 - iter 20/25 - loss 8.79425094 - samples/sec: 122.27 - lr: 0.300000 +2021-03-26 05:45:39,547 epoch 13 - iter 22/25 - loss 9.00863030 - samples/sec: 110.74 - lr: 0.300000 +2021-03-26 05:45:40,547 epoch 13 - iter 24/25 - loss 9.03554883 - samples/sec: 128.22 - lr: 0.300000 +2021-03-26 05:45:40,918 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:45:40,918 EPOCH 13 done: loss 9.1318 - lr 0.3000000 +2021-03-26 05:45:41,743 DEV : loss 9.024928092956543 - score 0.8556 +2021-03-26 05:45:41,765 BAD EPOCHS (no improvement): 0 +2021-03-26 05:45:51,573 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:45:52,696 epoch 14 - iter 2/25 - loss 8.50551748 - samples/sec: 114.32 - lr: 0.300000 +2021-03-26 05:45:53,691 epoch 14 - iter 4/25 - loss 8.25183105 - samples/sec: 129.07 - lr: 0.300000 +2021-03-26 05:45:54,698 epoch 14 - iter 6/25 - loss 8.82747555 - samples/sec: 127.24 - lr: 0.300000 +2021-03-26 05:45:55,723 epoch 14 - iter 8/25 - loss 8.81481767 - samples/sec: 125.19 - lr: 0.300000 +2021-03-26 05:45:56,715 epoch 14 - iter 10/25 - loss 8.86008492 - samples/sec: 129.27 - lr: 0.300000 +2021-03-26 05:45:57,773 epoch 14 - iter 12/25 - loss 8.99228557 - samples/sec: 121.11 - lr: 0.300000 +2021-03-26 05:45:58,924 epoch 14 - iter 14/25 - loss 8.92841278 - samples/sec: 111.53 - lr: 0.300000 +2021-03-26 05:45:59,921 epoch 14 - iter 16/25 - loss 8.81908402 - samples/sec: 128.65 - lr: 0.300000 +2021-03-26 05:46:00,917 epoch 14 - iter 18/25 - loss 8.85951858 - samples/sec: 128.72 - lr: 0.300000 +2021-03-26 05:46:01,962 epoch 14 - iter 20/25 - loss 8.76512609 - samples/sec: 122.65 - lr: 0.300000 +2021-03-26 05:46:02,926 epoch 14 - iter 22/25 - loss 8.77875579 - samples/sec: 132.88 - lr: 0.300000 +2021-03-26 05:46:03,941 epoch 14 - iter 24/25 - loss 8.72450026 - samples/sec: 126.29 - lr: 0.300000 +2021-03-26 05:46:04,361 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:46:04,362 EPOCH 14 done: loss 8.8084 - lr 0.3000000 +2021-03-26 05:46:05,158 DEV : loss 8.242410659790039 - score 0.8596 +2021-03-26 05:46:05,176 BAD EPOCHS (no improvement): 0 +2021-03-26 05:46:14,853 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:46:16,012 epoch 15 - iter 2/25 - loss 8.07281971 - samples/sec: 110.99 - lr: 0.300000 +2021-03-26 05:46:17,051 epoch 15 - iter 4/25 - loss 8.91634297 - samples/sec: 123.53 - lr: 0.300000 +2021-03-26 05:46:17,999 epoch 15 - iter 6/25 - loss 8.43717909 - samples/sec: 135.21 - lr: 0.300000 +2021-03-26 05:46:18,994 epoch 15 - iter 8/25 - loss 8.13764864 - samples/sec: 128.95 - lr: 0.300000 +2021-03-26 05:46:19,991 epoch 15 - iter 10/25 - loss 8.22757897 - samples/sec: 128.62 - lr: 0.300000 +2021-03-26 05:46:20,963 epoch 15 - iter 12/25 - loss 8.27018185 - samples/sec: 132.56 - lr: 0.300000 +2021-03-26 05:46:21,938 epoch 15 - iter 14/25 - loss 8.31585772 - samples/sec: 131.44 - lr: 0.300000 +2021-03-26 05:46:22,995 epoch 15 - iter 16/25 - loss 8.29172522 - samples/sec: 121.31 - lr: 0.300000 +2021-03-26 05:46:24,040 epoch 15 - iter 18/25 - loss 8.23328315 - samples/sec: 122.62 - lr: 0.300000 +2021-03-26 05:46:24,981 epoch 15 - iter 20/25 - loss 8.09761107 - samples/sec: 136.19 - lr: 0.300000 +2021-03-26 05:46:26,065 epoch 15 - iter 22/25 - loss 8.19500925 - samples/sec: 118.27 - lr: 0.300000 +2021-03-26 05:46:27,191 epoch 15 - iter 24/25 - loss 8.17269633 - samples/sec: 113.79 - lr: 0.300000 +2021-03-26 05:46:27,663 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:46:27,663 EPOCH 15 done: loss 8.2421 - lr 0.3000000 +2021-03-26 05:46:28,522 DEV : loss 8.071908950805664 - score 0.8716 +2021-03-26 05:46:28,548 BAD EPOCHS (no improvement): 0 +2021-03-26 05:46:38,366 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:46:39,495 epoch 16 - iter 2/25 - loss 6.56948996 - samples/sec: 113.55 - lr: 0.300000 +2021-03-26 05:46:40,582 epoch 16 - iter 4/25 - loss 7.08489966 - samples/sec: 117.99 - lr: 0.300000 +2021-03-26 05:46:41,569 epoch 16 - iter 6/25 - loss 7.12076028 - samples/sec: 129.85 - lr: 0.300000 +2021-03-26 05:46:42,539 epoch 16 - iter 8/25 - loss 7.15210068 - samples/sec: 132.06 - lr: 0.300000 +2021-03-26 05:46:43,591 epoch 16 - iter 10/25 - loss 7.48657627 - samples/sec: 121.91 - lr: 0.300000 +2021-03-26 05:46:44,556 epoch 16 - iter 12/25 - loss 7.65123530 - samples/sec: 133.57 - lr: 0.300000 +2021-03-26 05:46:45,508 epoch 16 - iter 14/25 - loss 7.54185397 - samples/sec: 134.63 - lr: 0.300000 +2021-03-26 05:46:46,493 epoch 16 - iter 16/25 - loss 7.41987136 - samples/sec: 130.13 - lr: 0.300000 +2021-03-26 05:46:47,513 epoch 16 - iter 18/25 - loss 7.63816534 - samples/sec: 125.67 - lr: 0.300000 +2021-03-26 05:46:48,487 epoch 16 - iter 20/25 - loss 7.70393589 - samples/sec: 131.57 - lr: 0.300000 +2021-03-26 05:46:49,485 epoch 16 - iter 22/25 - loss 7.68824291 - samples/sec: 128.48 - lr: 0.300000 +2021-03-26 05:46:50,458 epoch 16 - iter 24/25 - loss 7.76888394 - samples/sec: 131.66 - lr: 0.300000 +2021-03-26 05:46:50,926 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:46:50,926 EPOCH 16 done: loss 7.7148 - lr 0.3000000 +2021-03-26 05:46:51,699 DEV : loss 7.759528636932373 - score 0.8764 +2021-03-26 05:46:51,726 BAD EPOCHS (no improvement): 0 +2021-03-26 05:47:01,505 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:47:02,648 epoch 17 - iter 2/25 - loss 7.21234059 - samples/sec: 112.30 - lr: 0.300000 +2021-03-26 05:47:03,629 epoch 17 - iter 4/25 - loss 7.31978428 - samples/sec: 130.64 - lr: 0.300000 +2021-03-26 05:47:04,689 epoch 17 - iter 6/25 - loss 7.75275191 - samples/sec: 120.98 - lr: 0.300000 +2021-03-26 05:47:05,784 epoch 17 - iter 8/25 - loss 7.63426083 - samples/sec: 116.98 - lr: 0.300000 +2021-03-26 05:47:06,829 epoch 17 - iter 10/25 - loss 7.70618315 - samples/sec: 122.75 - lr: 0.300000 +2021-03-26 05:47:07,839 epoch 17 - iter 12/25 - loss 7.80201284 - samples/sec: 126.90 - lr: 0.300000 +2021-03-26 05:47:08,941 epoch 17 - iter 14/25 - loss 7.86180087 - samples/sec: 116.28 - lr: 0.300000 +2021-03-26 05:47:09,987 epoch 17 - iter 16/25 - loss 7.67045468 - samples/sec: 122.61 - lr: 0.300000 +2021-03-26 05:47:11,023 epoch 17 - iter 18/25 - loss 7.80948228 - samples/sec: 123.82 - lr: 0.300000 +2021-03-26 05:47:11,955 epoch 17 - iter 20/25 - loss 7.69441493 - samples/sec: 137.57 - lr: 0.300000 +2021-03-26 05:47:12,997 epoch 17 - iter 22/25 - loss 7.67484760 - samples/sec: 122.99 - lr: 0.300000 +2021-03-26 05:47:14,164 epoch 17 - iter 24/25 - loss 7.75634086 - samples/sec: 109.77 - lr: 0.300000 +2021-03-26 05:47:14,562 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:47:14,563 EPOCH 17 done: loss 7.7065 - lr 0.3000000 +2021-03-26 05:47:15,365 DEV : loss 7.736227989196777 - score 0.8774 +2021-03-26 05:47:15,387 BAD EPOCHS (no improvement): 0 +2021-03-26 05:47:25,233 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:47:26,346 epoch 18 - iter 2/25 - loss 7.17800355 - samples/sec: 115.22 - lr: 0.300000 +2021-03-26 05:47:27,483 epoch 18 - iter 4/25 - loss 7.56220233 - samples/sec: 112.90 - lr: 0.300000 +2021-03-26 05:47:28,563 epoch 18 - iter 6/25 - loss 7.32986403 - samples/sec: 118.70 - lr: 0.300000 +2021-03-26 05:47:29,565 epoch 18 - iter 8/25 - loss 7.14382195 - samples/sec: 127.95 - lr: 0.300000 +2021-03-26 05:47:30,537 epoch 18 - iter 10/25 - loss 7.13372774 - samples/sec: 131.85 - lr: 0.300000 +2021-03-26 05:47:31,619 epoch 18 - iter 12/25 - loss 7.36776002 - samples/sec: 118.43 - lr: 0.300000 +2021-03-26 05:47:32,635 epoch 18 - iter 14/25 - loss 7.26921167 - samples/sec: 126.40 - lr: 0.300000 +2021-03-26 05:47:33,691 epoch 18 - iter 16/25 - loss 7.35486242 - samples/sec: 121.31 - lr: 0.300000 +2021-03-26 05:47:34,902 epoch 18 - iter 18/25 - loss 7.30507800 - samples/sec: 105.82 - lr: 0.300000 +2021-03-26 05:47:35,916 epoch 18 - iter 20/25 - loss 7.24085248 - samples/sec: 126.64 - lr: 0.300000 +2021-03-26 05:47:36,889 epoch 18 - iter 22/25 - loss 7.21271409 - samples/sec: 131.65 - lr: 0.300000 +2021-03-26 05:47:37,888 epoch 18 - iter 24/25 - loss 7.22516036 - samples/sec: 128.35 - lr: 0.300000 +2021-03-26 05:47:38,322 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:47:38,323 EPOCH 18 done: loss 7.2983 - lr 0.3000000 +2021-03-26 05:47:39,106 DEV : loss 7.408334732055664 - score 0.8796 +2021-03-26 05:47:39,132 BAD EPOCHS (no improvement): 0 +2021-03-26 05:47:48,970 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:47:49,936 epoch 19 - iter 2/25 - loss 7.55202436 - samples/sec: 132.73 - lr: 0.300000 +2021-03-26 05:47:50,894 epoch 19 - iter 4/25 - loss 6.82127225 - samples/sec: 133.93 - lr: 0.300000 +2021-03-26 05:47:51,881 epoch 19 - iter 6/25 - loss 6.78057869 - samples/sec: 130.03 - lr: 0.300000 +2021-03-26 05:47:53,154 epoch 19 - iter 8/25 - loss 6.69285744 - samples/sec: 100.75 - lr: 0.300000 +2021-03-26 05:47:54,148 epoch 19 - iter 10/25 - loss 6.59116917 - samples/sec: 128.99 - lr: 0.300000 +2021-03-26 05:47:55,029 epoch 19 - iter 12/25 - loss 6.69682733 - samples/sec: 145.48 - lr: 0.300000 +2021-03-26 05:47:56,012 epoch 19 - iter 14/25 - loss 6.73391271 - samples/sec: 130.40 - lr: 0.300000 +2021-03-26 05:47:57,053 epoch 19 - iter 16/25 - loss 6.77963328 - samples/sec: 123.15 - lr: 0.300000 +2021-03-26 05:47:58,029 epoch 19 - iter 18/25 - loss 6.72666068 - samples/sec: 131.55 - lr: 0.300000 +2021-03-26 05:47:59,114 epoch 19 - iter 20/25 - loss 6.93562422 - samples/sec: 118.20 - lr: 0.300000 +2021-03-26 05:48:00,115 epoch 19 - iter 22/25 - loss 6.89227057 - samples/sec: 128.10 - lr: 0.300000 +2021-03-26 05:48:01,107 epoch 19 - iter 24/25 - loss 6.89240589 - samples/sec: 129.35 - lr: 0.300000 +2021-03-26 05:48:01,518 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:48:01,520 EPOCH 19 done: loss 6.8797 - lr 0.3000000 +2021-03-26 05:48:02,335 DEV : loss 7.592113018035889 - score 0.8746 +2021-03-26 05:48:02,365 BAD EPOCHS (no improvement): 1 +2021-03-26 05:48:02,365 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:48:03,396 epoch 20 - iter 2/25 - loss 6.29856849 - samples/sec: 124.38 - lr: 0.300000 +2021-03-26 05:48:04,360 epoch 20 - iter 4/25 - loss 6.76293743 - samples/sec: 133.10 - lr: 0.300000 +2021-03-26 05:48:05,396 epoch 20 - iter 6/25 - loss 6.62074788 - samples/sec: 123.75 - lr: 0.300000 +2021-03-26 05:48:06,555 epoch 20 - iter 8/25 - loss 6.44405246 - samples/sec: 110.54 - lr: 0.300000 +2021-03-26 05:48:07,562 epoch 20 - iter 10/25 - loss 6.45065737 - samples/sec: 127.35 - lr: 0.300000 +2021-03-26 05:48:08,536 epoch 20 - iter 12/25 - loss 6.51046475 - samples/sec: 131.57 - lr: 0.300000 +2021-03-26 05:48:09,484 epoch 20 - iter 14/25 - loss 6.43319787 - samples/sec: 135.39 - lr: 0.300000 +2021-03-26 05:48:10,474 epoch 20 - iter 16/25 - loss 6.43069494 - samples/sec: 129.43 - lr: 0.300000 +2021-03-26 05:48:11,561 epoch 20 - iter 18/25 - loss 6.40164778 - samples/sec: 118.00 - lr: 0.300000 +2021-03-26 05:48:12,743 epoch 20 - iter 20/25 - loss 6.54254618 - samples/sec: 108.39 - lr: 0.300000 +2021-03-26 05:48:13,764 epoch 20 - iter 22/25 - loss 6.58177922 - samples/sec: 125.65 - lr: 0.300000 +2021-03-26 05:48:14,738 epoch 20 - iter 24/25 - loss 6.61301217 - samples/sec: 131.54 - lr: 0.300000 +2021-03-26 05:48:15,139 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:48:15,140 EPOCH 20 done: loss 6.5882 - lr 0.3000000 +2021-03-26 05:48:15,916 DEV : loss 7.316654205322266 - score 0.8808 +2021-03-26 05:48:15,942 BAD EPOCHS (no improvement): 0 +2021-03-26 05:48:25,531 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:48:26,515 epoch 21 - iter 2/25 - loss 6.55948019 - samples/sec: 130.53 - lr: 0.300000 +2021-03-26 05:48:27,655 epoch 21 - iter 4/25 - loss 5.92072499 - samples/sec: 112.40 - lr: 0.300000 +2021-03-26 05:48:28,666 epoch 21 - iter 6/25 - loss 5.60158626 - samples/sec: 126.75 - lr: 0.300000 +2021-03-26 05:48:29,621 epoch 21 - iter 8/25 - loss 5.78064823 - samples/sec: 134.21 - lr: 0.300000 +2021-03-26 05:48:30,595 epoch 21 - iter 10/25 - loss 5.86341887 - samples/sec: 131.69 - lr: 0.300000 +2021-03-26 05:48:31,640 epoch 21 - iter 12/25 - loss 5.97549959 - samples/sec: 122.62 - lr: 0.300000 +2021-03-26 05:48:32,697 epoch 21 - iter 14/25 - loss 6.21892132 - samples/sec: 121.24 - lr: 0.300000 +2021-03-26 05:48:33,902 epoch 21 - iter 16/25 - loss 6.26286507 - samples/sec: 106.35 - lr: 0.300000 +2021-03-26 05:48:35,248 epoch 21 - iter 18/25 - loss 6.13798594 - samples/sec: 95.30 - lr: 0.300000 +2021-03-26 05:48:36,248 epoch 21 - iter 20/25 - loss 6.16988015 - samples/sec: 128.15 - lr: 0.300000 +2021-03-26 05:48:37,287 epoch 21 - iter 22/25 - loss 6.20285286 - samples/sec: 123.43 - lr: 0.300000 +2021-03-26 05:48:38,356 epoch 21 - iter 24/25 - loss 6.24573507 - samples/sec: 119.83 - lr: 0.300000 +2021-03-26 05:48:38,960 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:48:38,961 EPOCH 21 done: loss 6.2932 - lr 0.3000000 +2021-03-26 05:48:39,802 DEV : loss 7.598691940307617 - score 0.8726 +2021-03-26 05:48:39,837 BAD EPOCHS (no improvement): 1 +2021-03-26 05:48:39,838 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:48:41,253 epoch 22 - iter 2/25 - loss 5.48124957 - samples/sec: 90.58 - lr: 0.300000 +2021-03-26 05:48:42,563 epoch 22 - iter 4/25 - loss 6.06442177 - samples/sec: 97.80 - lr: 0.300000 +2021-03-26 05:48:43,863 epoch 22 - iter 6/25 - loss 5.93726675 - samples/sec: 98.52 - lr: 0.300000 +2021-03-26 05:48:45,364 epoch 22 - iter 8/25 - loss 6.18622816 - samples/sec: 85.58 - lr: 0.300000 +2021-03-26 05:48:46,774 epoch 22 - iter 10/25 - loss 6.23740492 - samples/sec: 90.87 - lr: 0.300000 +2021-03-26 05:48:47,923 epoch 22 - iter 12/25 - loss 6.31357996 - samples/sec: 111.53 - lr: 0.300000 +2021-03-26 05:48:48,873 epoch 22 - iter 14/25 - loss 6.24927248 - samples/sec: 134.88 - lr: 0.300000 +2021-03-26 05:48:49,861 epoch 22 - iter 16/25 - loss 6.27198809 - samples/sec: 129.93 - lr: 0.300000 +2021-03-26 05:48:50,877 epoch 22 - iter 18/25 - loss 6.30164162 - samples/sec: 126.04 - lr: 0.300000 +2021-03-26 05:48:51,869 epoch 22 - iter 20/25 - loss 6.28241801 - samples/sec: 129.22 - lr: 0.300000 +2021-03-26 05:48:52,862 epoch 22 - iter 22/25 - loss 6.31069586 - samples/sec: 129.24 - lr: 0.300000 +2021-03-26 05:48:53,961 epoch 22 - iter 24/25 - loss 6.28890721 - samples/sec: 116.59 - lr: 0.300000 +2021-03-26 05:48:54,462 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:48:54,463 EPOCH 22 done: loss 6.3392 - lr 0.3000000 +2021-03-26 05:48:55,284 DEV : loss 7.424391746520996 - score 0.8802 +2021-03-26 05:48:55,315 BAD EPOCHS (no improvement): 2 +2021-03-26 05:48:55,315 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:48:56,413 epoch 23 - iter 2/25 - loss 7.48641682 - samples/sec: 116.85 - lr: 0.300000 +2021-03-26 05:48:57,395 epoch 23 - iter 4/25 - loss 6.89555621 - samples/sec: 130.61 - lr: 0.300000 +2021-03-26 05:48:58,460 epoch 23 - iter 6/25 - loss 6.57134056 - samples/sec: 120.40 - lr: 0.300000 +2021-03-26 05:48:59,660 epoch 23 - iter 8/25 - loss 6.16241872 - samples/sec: 106.79 - lr: 0.300000 +2021-03-26 05:49:00,733 epoch 23 - iter 10/25 - loss 6.01301031 - samples/sec: 119.43 - lr: 0.300000 +2021-03-26 05:49:01,727 epoch 23 - iter 12/25 - loss 5.96942111 - samples/sec: 129.08 - lr: 0.300000 +2021-03-26 05:49:02,695 epoch 23 - iter 14/25 - loss 5.98979133 - samples/sec: 132.53 - lr: 0.300000 +2021-03-26 05:49:03,756 epoch 23 - iter 16/25 - loss 5.93200183 - samples/sec: 120.81 - lr: 0.300000 +2021-03-26 05:49:04,735 epoch 23 - iter 18/25 - loss 5.90244452 - samples/sec: 130.93 - lr: 0.300000 +2021-03-26 05:49:05,761 epoch 23 - iter 20/25 - loss 5.99406066 - samples/sec: 124.90 - lr: 0.300000 +2021-03-26 05:49:06,817 epoch 23 - iter 22/25 - loss 6.01188855 - samples/sec: 121.40 - lr: 0.300000 +2021-03-26 05:49:07,786 epoch 23 - iter 24/25 - loss 6.00390444 - samples/sec: 132.41 - lr: 0.300000 +2021-03-26 05:49:08,255 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:49:08,256 EPOCH 23 done: loss 6.0797 - lr 0.3000000 +2021-03-26 05:49:09,056 DEV : loss 7.356023788452148 - score 0.883 +2021-03-26 05:49:09,075 BAD EPOCHS (no improvement): 0 +2021-03-26 05:49:18,746 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:49:19,866 epoch 24 - iter 2/25 - loss 5.56664753 - samples/sec: 114.59 - lr: 0.300000 +2021-03-26 05:49:20,854 epoch 24 - iter 4/25 - loss 5.72063935 - samples/sec: 129.66 - lr: 0.300000 +2021-03-26 05:49:21,812 epoch 24 - iter 6/25 - loss 5.76620873 - samples/sec: 133.94 - lr: 0.300000 +2021-03-26 05:49:23,020 epoch 24 - iter 8/25 - loss 5.84735155 - samples/sec: 106.04 - lr: 0.300000 +2021-03-26 05:49:24,401 epoch 24 - iter 10/25 - loss 5.96158013 - samples/sec: 92.83 - lr: 0.300000 +2021-03-26 05:49:25,393 epoch 24 - iter 12/25 - loss 5.80328266 - samples/sec: 129.21 - lr: 0.300000 +2021-03-26 05:49:26,386 epoch 24 - iter 14/25 - loss 5.73408553 - samples/sec: 129.11 - lr: 0.300000 +2021-03-26 05:49:27,388 epoch 24 - iter 16/25 - loss 5.67245895 - samples/sec: 127.98 - lr: 0.300000 +2021-03-26 05:49:28,392 epoch 24 - iter 18/25 - loss 5.72128481 - samples/sec: 127.61 - lr: 0.300000 +2021-03-26 05:49:29,443 epoch 24 - iter 20/25 - loss 5.75462203 - samples/sec: 121.95 - lr: 0.300000 +2021-03-26 05:49:30,462 epoch 24 - iter 22/25 - loss 5.76823217 - samples/sec: 125.81 - lr: 0.300000 +2021-03-26 05:49:31,465 epoch 24 - iter 24/25 - loss 5.79069185 - samples/sec: 127.82 - lr: 0.300000 +2021-03-26 05:49:31,910 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:49:31,911 EPOCH 24 done: loss 5.8127 - lr 0.3000000 +2021-03-26 05:49:32,690 DEV : loss 7.419638633728027 - score 0.8864 +2021-03-26 05:49:32,715 BAD EPOCHS (no improvement): 0 +2021-03-26 05:49:42,741 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:49:43,746 epoch 25 - iter 2/25 - loss 6.35212374 - samples/sec: 127.64 - lr: 0.300000 +2021-03-26 05:49:44,824 epoch 25 - iter 4/25 - loss 5.96967781 - samples/sec: 118.94 - lr: 0.300000 +2021-03-26 05:49:45,725 epoch 25 - iter 6/25 - loss 6.14810308 - samples/sec: 142.53 - lr: 0.300000 +2021-03-26 05:49:46,767 epoch 25 - iter 8/25 - loss 6.18986714 - samples/sec: 122.95 - lr: 0.300000 +2021-03-26 05:49:47,734 epoch 25 - iter 10/25 - loss 6.18832707 - samples/sec: 132.51 - lr: 0.300000 +2021-03-26 05:49:48,665 epoch 25 - iter 12/25 - loss 6.09619081 - samples/sec: 137.72 - lr: 0.300000 +2021-03-26 05:49:49,794 epoch 25 - iter 14/25 - loss 5.98943094 - samples/sec: 113.51 - lr: 0.300000 +2021-03-26 05:49:50,774 epoch 25 - iter 16/25 - loss 5.93080184 - samples/sec: 130.90 - lr: 0.300000 +2021-03-26 05:49:51,814 epoch 25 - iter 18/25 - loss 5.93358392 - samples/sec: 123.27 - lr: 0.300000 +2021-03-26 05:49:52,891 epoch 25 - iter 20/25 - loss 5.89371572 - samples/sec: 118.95 - lr: 0.300000 +2021-03-26 05:49:53,987 epoch 25 - iter 22/25 - loss 5.87030755 - samples/sec: 116.98 - lr: 0.300000 +2021-03-26 05:49:55,301 epoch 25 - iter 24/25 - loss 5.85457329 - samples/sec: 97.58 - lr: 0.300000 +2021-03-26 05:49:55,677 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:49:55,678 EPOCH 25 done: loss 5.7638 - lr 0.3000000 +2021-03-26 05:49:56,482 DEV : loss 7.5117692947387695 - score 0.8825 +2021-03-26 05:49:56,508 BAD EPOCHS (no improvement): 1 +2021-03-26 05:49:56,508 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:49:57,657 epoch 26 - iter 2/25 - loss 5.19982338 - samples/sec: 111.59 - lr: 0.300000 +2021-03-26 05:49:58,815 epoch 26 - iter 4/25 - loss 4.92103577 - samples/sec: 110.68 - lr: 0.300000 +2021-03-26 05:49:59,991 epoch 26 - iter 6/25 - loss 5.07783246 - samples/sec: 109.06 - lr: 0.300000 +2021-03-26 05:50:01,047 epoch 26 - iter 8/25 - loss 5.18258464 - samples/sec: 122.00 - lr: 0.300000 +2021-03-26 05:50:02,294 epoch 26 - iter 10/25 - loss 5.27087030 - samples/sec: 102.81 - lr: 0.300000 +2021-03-26 05:50:03,302 epoch 26 - iter 12/25 - loss 5.23783867 - samples/sec: 127.23 - lr: 0.300000 +2021-03-26 05:50:04,382 epoch 26 - iter 14/25 - loss 5.33069127 - samples/sec: 118.57 - lr: 0.300000 +2021-03-26 05:50:05,358 epoch 26 - iter 16/25 - loss 5.30787414 - samples/sec: 131.42 - lr: 0.300000 +2021-03-26 05:50:06,347 epoch 26 - iter 18/25 - loss 5.30061195 - samples/sec: 129.55 - lr: 0.300000 +2021-03-26 05:50:07,295 epoch 26 - iter 20/25 - loss 5.31607125 - samples/sec: 135.22 - lr: 0.300000 +2021-03-26 05:50:08,469 epoch 26 - iter 22/25 - loss 5.24197604 - samples/sec: 109.23 - lr: 0.300000 +2021-03-26 05:50:09,498 epoch 26 - iter 24/25 - loss 5.32497180 - samples/sec: 124.55 - lr: 0.300000 +2021-03-26 05:50:09,926 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:50:09,927 EPOCH 26 done: loss 5.2962 - lr 0.3000000 +2021-03-26 05:50:10,686 DEV : loss 7.333011150360107 - score 0.8881 +2021-03-26 05:50:10,710 BAD EPOCHS (no improvement): 0 +2021-03-26 05:50:20,333 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:50:21,431 epoch 27 - iter 2/25 - loss 5.16186333 - samples/sec: 116.78 - lr: 0.300000 +2021-03-26 05:50:22,464 epoch 27 - iter 4/25 - loss 5.12837744 - samples/sec: 124.20 - lr: 0.300000 +2021-03-26 05:50:23,572 epoch 27 - iter 6/25 - loss 5.31019672 - samples/sec: 115.61 - lr: 0.300000 +2021-03-26 05:50:24,600 epoch 27 - iter 8/25 - loss 5.32734174 - samples/sec: 124.69 - lr: 0.300000 +2021-03-26 05:50:25,613 epoch 27 - iter 10/25 - loss 5.22059751 - samples/sec: 126.63 - lr: 0.300000 +2021-03-26 05:50:26,594 epoch 27 - iter 12/25 - loss 5.27950998 - samples/sec: 130.75 - lr: 0.300000 +2021-03-26 05:50:27,597 epoch 27 - iter 14/25 - loss 5.41089095 - samples/sec: 127.69 - lr: 0.300000 +2021-03-26 05:50:28,609 epoch 27 - iter 16/25 - loss 5.40781000 - samples/sec: 126.67 - lr: 0.300000 +2021-03-26 05:50:29,656 epoch 27 - iter 18/25 - loss 5.39746579 - samples/sec: 122.47 - lr: 0.300000 +2021-03-26 05:50:30,589 epoch 27 - iter 20/25 - loss 5.43662884 - samples/sec: 137.43 - lr: 0.300000 +2021-03-26 05:50:31,638 epoch 27 - iter 22/25 - loss 5.37713861 - samples/sec: 122.16 - lr: 0.300000 +2021-03-26 05:50:32,594 epoch 27 - iter 24/25 - loss 5.36295988 - samples/sec: 134.30 - lr: 0.300000 +2021-03-26 05:50:32,996 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:50:32,996 EPOCH 27 done: loss 5.3616 - lr 0.3000000 +2021-03-26 05:50:33,787 DEV : loss 7.019782066345215 - score 0.8849 +2021-03-26 05:50:33,817 BAD EPOCHS (no improvement): 1 +2021-03-26 05:50:33,817 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:50:34,879 epoch 28 - iter 2/25 - loss 5.07017589 - samples/sec: 120.77 - lr: 0.300000 +2021-03-26 05:50:36,041 epoch 28 - iter 4/25 - loss 5.02670932 - samples/sec: 110.30 - lr: 0.300000 +2021-03-26 05:50:37,066 epoch 28 - iter 6/25 - loss 4.80902608 - samples/sec: 125.02 - lr: 0.300000 +2021-03-26 05:50:38,124 epoch 28 - iter 8/25 - loss 4.62447375 - samples/sec: 121.11 - lr: 0.300000 +2021-03-26 05:50:39,335 epoch 28 - iter 10/25 - loss 4.67827401 - samples/sec: 105.87 - lr: 0.300000 +2021-03-26 05:50:40,588 epoch 28 - iter 12/25 - loss 4.77806664 - samples/sec: 102.26 - lr: 0.300000 +2021-03-26 05:50:41,721 epoch 28 - iter 14/25 - loss 4.72924319 - samples/sec: 113.66 - lr: 0.300000 +2021-03-26 05:50:42,844 epoch 28 - iter 16/25 - loss 4.71131511 - samples/sec: 114.18 - lr: 0.300000 +2021-03-26 05:50:43,847 epoch 28 - iter 18/25 - loss 4.68717831 - samples/sec: 127.83 - lr: 0.300000 +2021-03-26 05:50:45,010 epoch 28 - iter 20/25 - loss 4.77009903 - samples/sec: 110.20 - lr: 0.300000 +2021-03-26 05:50:46,075 epoch 28 - iter 22/25 - loss 4.80764932 - samples/sec: 120.32 - lr: 0.300000 +2021-03-26 05:50:47,113 epoch 28 - iter 24/25 - loss 4.90641921 - samples/sec: 123.50 - lr: 0.300000 +2021-03-26 05:50:47,557 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:50:47,557 EPOCH 28 done: loss 4.9966 - lr 0.3000000 +2021-03-26 05:50:48,339 DEV : loss 7.182887077331543 - score 0.8918 +2021-03-26 05:50:48,364 BAD EPOCHS (no improvement): 0 +2021-03-26 05:50:58,235 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:50:59,334 epoch 29 - iter 2/25 - loss 4.95724106 - samples/sec: 116.78 - lr: 0.300000 +2021-03-26 05:51:00,317 epoch 29 - iter 4/25 - loss 4.68853933 - samples/sec: 130.36 - lr: 0.300000 +2021-03-26 05:51:01,279 epoch 29 - iter 6/25 - loss 4.66039519 - samples/sec: 133.25 - lr: 0.300000 +2021-03-26 05:51:02,244 epoch 29 - iter 8/25 - loss 4.51463768 - samples/sec: 132.87 - lr: 0.300000 +2021-03-26 05:51:03,327 epoch 29 - iter 10/25 - loss 4.73710887 - samples/sec: 118.37 - lr: 0.300000 +2021-03-26 05:51:04,504 epoch 29 - iter 12/25 - loss 4.68225382 - samples/sec: 108.91 - lr: 0.300000 +2021-03-26 05:51:05,503 epoch 29 - iter 14/25 - loss 4.56697776 - samples/sec: 128.41 - lr: 0.300000 +2021-03-26 05:51:06,493 epoch 29 - iter 16/25 - loss 4.53189199 - samples/sec: 129.70 - lr: 0.300000 +2021-03-26 05:51:07,506 epoch 29 - iter 18/25 - loss 4.68264200 - samples/sec: 127.20 - lr: 0.300000 +2021-03-26 05:51:08,637 epoch 29 - iter 20/25 - loss 4.78868047 - samples/sec: 113.21 - lr: 0.300000 +2021-03-26 05:51:09,667 epoch 29 - iter 22/25 - loss 4.92078643 - samples/sec: 124.51 - lr: 0.300000 +2021-03-26 05:51:10,900 epoch 29 - iter 24/25 - loss 4.98505587 - samples/sec: 103.91 - lr: 0.300000 +2021-03-26 05:51:11,284 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:51:11,285 EPOCH 29 done: loss 5.0067 - lr 0.3000000 +2021-03-26 05:51:12,098 DEV : loss 7.243838310241699 - score 0.8918 +2021-03-26 05:51:12,128 BAD EPOCHS (no improvement): 1 +2021-03-26 05:51:12,129 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:51:13,162 epoch 30 - iter 2/25 - loss 4.69986725 - samples/sec: 124.15 - lr: 0.300000 +2021-03-26 05:51:14,296 epoch 30 - iter 4/25 - loss 4.84692991 - samples/sec: 112.95 - lr: 0.300000 +2021-03-26 05:51:15,280 epoch 30 - iter 6/25 - loss 4.66442370 - samples/sec: 130.35 - lr: 0.300000 +2021-03-26 05:51:16,392 epoch 30 - iter 8/25 - loss 4.60726658 - samples/sec: 115.31 - lr: 0.300000 +2021-03-26 05:51:17,411 epoch 30 - iter 10/25 - loss 4.60311129 - samples/sec: 125.83 - lr: 0.300000 +2021-03-26 05:51:18,448 epoch 30 - iter 12/25 - loss 4.72527681 - samples/sec: 123.73 - lr: 0.300000 +2021-03-26 05:51:19,475 epoch 30 - iter 14/25 - loss 4.62729422 - samples/sec: 124.77 - lr: 0.300000 +2021-03-26 05:51:21,828 epoch 30 - iter 16/25 - loss 4.68846346 - samples/sec: 54.46 - lr: 0.300000 +2021-03-26 05:51:22,903 epoch 30 - iter 18/25 - loss 4.74469176 - samples/sec: 119.27 - lr: 0.300000 +2021-03-26 05:51:23,885 epoch 30 - iter 20/25 - loss 4.70663766 - samples/sec: 130.50 - lr: 0.300000 +2021-03-26 05:51:25,067 epoch 30 - iter 22/25 - loss 4.72499332 - samples/sec: 108.44 - lr: 0.300000 +2021-03-26 05:51:26,152 epoch 30 - iter 24/25 - loss 4.69499653 - samples/sec: 118.15 - lr: 0.300000 +2021-03-26 05:51:26,532 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:51:26,532 EPOCH 30 done: loss 4.6847 - lr 0.3000000 +2021-03-26 05:51:27,342 DEV : loss 7.096852779388428 - score 0.8954 +2021-03-26 05:51:27,365 BAD EPOCHS (no improvement): 0 +2021-03-26 05:51:37,234 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:51:38,257 epoch 31 - iter 2/25 - loss 4.87494373 - samples/sec: 125.43 - lr: 0.300000 +2021-03-26 05:51:39,293 epoch 31 - iter 4/25 - loss 4.54467654 - samples/sec: 123.83 - lr: 0.300000 +2021-03-26 05:51:40,261 epoch 31 - iter 6/25 - loss 4.44595257 - samples/sec: 132.41 - lr: 0.300000 +2021-03-26 05:51:41,212 epoch 31 - iter 8/25 - loss 4.43987417 - samples/sec: 134.97 - lr: 0.300000 +2021-03-26 05:51:42,199 epoch 31 - iter 10/25 - loss 4.30324354 - samples/sec: 129.78 - lr: 0.300000 +2021-03-26 05:51:43,255 epoch 31 - iter 12/25 - loss 4.50878437 - samples/sec: 121.38 - lr: 0.300000 +2021-03-26 05:51:44,323 epoch 31 - iter 14/25 - loss 4.55510497 - samples/sec: 120.11 - lr: 0.300000 +2021-03-26 05:51:45,461 epoch 31 - iter 16/25 - loss 4.60535118 - samples/sec: 112.62 - lr: 0.300000 +2021-03-26 05:51:46,501 epoch 31 - iter 18/25 - loss 4.64036245 - samples/sec: 123.17 - lr: 0.300000 +2021-03-26 05:51:47,513 epoch 31 - iter 20/25 - loss 4.69187005 - samples/sec: 126.69 - lr: 0.300000 +2021-03-26 05:51:48,481 epoch 31 - iter 22/25 - loss 4.67161454 - samples/sec: 132.40 - lr: 0.300000 +2021-03-26 05:51:49,496 epoch 31 - iter 24/25 - loss 4.69764324 - samples/sec: 126.34 - lr: 0.300000 +2021-03-26 05:51:49,914 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:51:49,915 EPOCH 31 done: loss 4.7288 - lr 0.3000000 +2021-03-26 05:51:50,690 DEV : loss 6.818641662597656 - score 0.8986 +2021-03-26 05:51:50,716 BAD EPOCHS (no improvement): 0 +2021-03-26 05:52:00,339 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:52:01,491 epoch 32 - iter 2/25 - loss 4.50873399 - samples/sec: 111.29 - lr: 0.300000 +2021-03-26 05:52:02,534 epoch 32 - iter 4/25 - loss 4.65008414 - samples/sec: 123.03 - lr: 0.300000 +2021-03-26 05:52:03,521 epoch 32 - iter 6/25 - loss 4.54596925 - samples/sec: 129.96 - lr: 0.300000 +2021-03-26 05:52:04,630 epoch 32 - iter 8/25 - loss 4.67705244 - samples/sec: 115.53 - lr: 0.300000 +2021-03-26 05:52:05,581 epoch 32 - iter 10/25 - loss 4.58810802 - samples/sec: 134.90 - lr: 0.300000 +2021-03-26 05:52:06,535 epoch 32 - iter 12/25 - loss 4.48243205 - samples/sec: 134.26 - lr: 0.300000 +2021-03-26 05:52:07,522 epoch 32 - iter 14/25 - loss 4.34981622 - samples/sec: 130.01 - lr: 0.300000 +2021-03-26 05:52:08,685 epoch 32 - iter 16/25 - loss 4.30454047 - samples/sec: 110.29 - lr: 0.300000 +2021-03-26 05:52:09,695 epoch 32 - iter 18/25 - loss 4.40030144 - samples/sec: 127.12 - lr: 0.300000 +2021-03-26 05:52:10,720 epoch 32 - iter 20/25 - loss 4.42493467 - samples/sec: 125.11 - lr: 0.300000 +2021-03-26 05:52:11,691 epoch 32 - iter 22/25 - loss 4.41666954 - samples/sec: 132.00 - lr: 0.300000 +2021-03-26 05:52:12,622 epoch 32 - iter 24/25 - loss 4.46280225 - samples/sec: 137.75 - lr: 0.300000 +2021-03-26 05:52:13,004 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:52:13,005 EPOCH 32 done: loss 4.4738 - lr 0.3000000 +2021-03-26 05:52:13,773 DEV : loss 6.889939308166504 - score 0.8994 +2021-03-26 05:52:13,798 BAD EPOCHS (no improvement): 0 +2021-03-26 05:52:23,499 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:52:24,515 epoch 33 - iter 2/25 - loss 4.49444866 - samples/sec: 126.26 - lr: 0.300000 +2021-03-26 05:52:25,520 epoch 33 - iter 4/25 - loss 4.15728247 - samples/sec: 127.56 - lr: 0.300000 +2021-03-26 05:52:26,490 epoch 33 - iter 6/25 - loss 4.49758712 - samples/sec: 132.10 - lr: 0.300000 +2021-03-26 05:52:27,420 epoch 33 - iter 8/25 - loss 4.29979959 - samples/sec: 137.95 - lr: 0.300000 +2021-03-26 05:52:28,443 epoch 33 - iter 10/25 - loss 4.28199461 - samples/sec: 125.34 - lr: 0.300000 +2021-03-26 05:52:29,503 epoch 33 - iter 12/25 - loss 4.25882107 - samples/sec: 120.95 - lr: 0.300000 +2021-03-26 05:52:30,482 epoch 33 - iter 14/25 - loss 4.39515034 - samples/sec: 131.00 - lr: 0.300000 +2021-03-26 05:52:31,428 epoch 33 - iter 16/25 - loss 4.38297199 - samples/sec: 135.72 - lr: 0.300000 +2021-03-26 05:52:32,521 epoch 33 - iter 18/25 - loss 4.39067970 - samples/sec: 117.24 - lr: 0.300000 +2021-03-26 05:52:33,692 epoch 33 - iter 20/25 - loss 4.40039575 - samples/sec: 109.58 - lr: 0.300000 +2021-03-26 05:52:34,733 epoch 33 - iter 22/25 - loss 4.41531073 - samples/sec: 123.12 - lr: 0.300000 +2021-03-26 05:52:35,734 epoch 33 - iter 24/25 - loss 4.43675790 - samples/sec: 128.11 - lr: 0.300000 +2021-03-26 05:52:36,210 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:52:36,212 EPOCH 33 done: loss 4.4681 - lr 0.3000000 +2021-03-26 05:52:36,990 DEV : loss 6.704212188720703 - score 0.899 +2021-03-26 05:52:37,016 BAD EPOCHS (no improvement): 1 +2021-03-26 05:52:37,017 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:52:38,049 epoch 34 - iter 2/25 - loss 4.85452342 - samples/sec: 124.26 - lr: 0.300000 +2021-03-26 05:52:39,022 epoch 34 - iter 4/25 - loss 4.34141463 - samples/sec: 131.71 - lr: 0.300000 +2021-03-26 05:52:40,061 epoch 34 - iter 6/25 - loss 4.44528925 - samples/sec: 123.40 - lr: 0.300000 +2021-03-26 05:52:41,019 epoch 34 - iter 8/25 - loss 4.39014325 - samples/sec: 133.80 - lr: 0.300000 +2021-03-26 05:52:42,082 epoch 34 - iter 10/25 - loss 4.40972173 - samples/sec: 120.60 - lr: 0.300000 +2021-03-26 05:52:43,066 epoch 34 - iter 12/25 - loss 4.44770990 - samples/sec: 130.30 - lr: 0.300000 +2021-03-26 05:52:44,164 epoch 34 - iter 14/25 - loss 4.42394963 - samples/sec: 116.79 - lr: 0.300000 +2021-03-26 05:52:45,102 epoch 34 - iter 16/25 - loss 4.28590170 - samples/sec: 136.59 - lr: 0.300000 +2021-03-26 05:52:46,081 epoch 34 - iter 18/25 - loss 4.33693907 - samples/sec: 130.91 - lr: 0.300000 +2021-03-26 05:52:47,060 epoch 34 - iter 20/25 - loss 4.30833759 - samples/sec: 131.04 - lr: 0.300000 +2021-03-26 05:52:48,102 epoch 34 - iter 22/25 - loss 4.32592613 - samples/sec: 123.03 - lr: 0.300000 +2021-03-26 05:52:49,156 epoch 34 - iter 24/25 - loss 4.35851695 - samples/sec: 121.67 - lr: 0.300000 +2021-03-26 05:52:49,577 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:52:49,577 EPOCH 34 done: loss 4.3686 - lr 0.3000000 +2021-03-26 05:52:50,360 DEV : loss 6.8909149169921875 - score 0.9 +2021-03-26 05:52:50,378 BAD EPOCHS (no improvement): 0 +2021-03-26 05:52:59,953 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:53:00,923 epoch 35 - iter 2/25 - loss 4.03293800 - samples/sec: 132.31 - lr: 0.300000 +2021-03-26 05:53:01,966 epoch 35 - iter 4/25 - loss 4.25911993 - samples/sec: 123.01 - lr: 0.300000 +2021-03-26 05:53:02,931 epoch 35 - iter 6/25 - loss 4.52007488 - samples/sec: 132.70 - lr: 0.300000 +2021-03-26 05:53:03,865 epoch 35 - iter 8/25 - loss 4.61368671 - samples/sec: 137.34 - lr: 0.300000 +2021-03-26 05:53:04,867 epoch 35 - iter 10/25 - loss 4.53169215 - samples/sec: 127.88 - lr: 0.300000 +2021-03-26 05:53:05,865 epoch 35 - iter 12/25 - loss 4.48455348 - samples/sec: 128.38 - lr: 0.300000 +2021-03-26 05:53:06,842 epoch 35 - iter 14/25 - loss 4.45766437 - samples/sec: 131.18 - lr: 0.300000 +2021-03-26 05:53:07,998 epoch 35 - iter 16/25 - loss 4.30519691 - samples/sec: 110.88 - lr: 0.300000 +2021-03-26 05:53:09,055 epoch 35 - iter 18/25 - loss 4.27180097 - samples/sec: 121.38 - lr: 0.300000 +2021-03-26 05:53:10,090 epoch 35 - iter 20/25 - loss 4.19139869 - samples/sec: 123.76 - lr: 0.300000 +2021-03-26 05:53:11,007 epoch 35 - iter 22/25 - loss 4.17696692 - samples/sec: 139.89 - lr: 0.300000 +2021-03-26 05:53:11,997 epoch 35 - iter 24/25 - loss 4.14253498 - samples/sec: 129.48 - lr: 0.300000 +2021-03-26 05:53:12,410 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:53:12,410 EPOCH 35 done: loss 4.1496 - lr 0.3000000 +2021-03-26 05:53:13,190 DEV : loss 6.916918754577637 - score 0.8928 +2021-03-26 05:53:13,215 BAD EPOCHS (no improvement): 1 +2021-03-26 05:53:13,216 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:53:14,245 epoch 36 - iter 2/25 - loss 3.75446880 - samples/sec: 124.71 - lr: 0.300000 +2021-03-26 05:53:15,430 epoch 36 - iter 4/25 - loss 4.07209581 - samples/sec: 108.10 - lr: 0.300000 +2021-03-26 05:53:16,350 epoch 36 - iter 6/25 - loss 4.02450681 - samples/sec: 139.43 - lr: 0.300000 +2021-03-26 05:53:17,365 epoch 36 - iter 8/25 - loss 3.84632128 - samples/sec: 126.28 - lr: 0.300000 +2021-03-26 05:53:18,361 epoch 36 - iter 10/25 - loss 3.93511500 - samples/sec: 128.67 - lr: 0.300000 +2021-03-26 05:53:19,307 epoch 36 - iter 12/25 - loss 3.98764678 - samples/sec: 135.51 - lr: 0.300000 +2021-03-26 05:53:20,246 epoch 36 - iter 14/25 - loss 4.08523498 - samples/sec: 136.38 - lr: 0.300000 +2021-03-26 05:53:21,269 epoch 36 - iter 16/25 - loss 4.14107890 - samples/sec: 125.35 - lr: 0.300000 +2021-03-26 05:53:22,221 epoch 36 - iter 18/25 - loss 4.20975999 - samples/sec: 134.58 - lr: 0.300000 +2021-03-26 05:53:23,149 epoch 36 - iter 20/25 - loss 4.16134720 - samples/sec: 138.34 - lr: 0.300000 +2021-03-26 05:53:24,232 epoch 36 - iter 22/25 - loss 4.14724630 - samples/sec: 118.33 - lr: 0.300000 +2021-03-26 05:53:25,199 epoch 36 - iter 24/25 - loss 4.13445437 - samples/sec: 132.58 - lr: 0.300000 +2021-03-26 05:53:25,669 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:53:25,670 EPOCH 36 done: loss 4.1594 - lr 0.3000000 +2021-03-26 05:53:26,453 DEV : loss 6.872126579284668 - score 0.9006 +2021-03-26 05:53:26,471 BAD EPOCHS (no improvement): 0 +2021-03-26 05:53:35,962 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:53:37,012 epoch 37 - iter 2/25 - loss 4.32914829 - samples/sec: 122.13 - lr: 0.300000 +2021-03-26 05:53:38,126 epoch 37 - iter 4/25 - loss 3.83761263 - samples/sec: 115.03 - lr: 0.300000 +2021-03-26 05:53:39,136 epoch 37 - iter 6/25 - loss 3.83161823 - samples/sec: 126.89 - lr: 0.300000 +2021-03-26 05:53:40,095 epoch 37 - iter 8/25 - loss 3.91476762 - samples/sec: 133.59 - lr: 0.300000 +2021-03-26 05:53:41,096 epoch 37 - iter 10/25 - loss 4.02498407 - samples/sec: 128.16 - lr: 0.300000 +2021-03-26 05:53:42,295 epoch 37 - iter 12/25 - loss 3.99303458 - samples/sec: 106.84 - lr: 0.300000 +2021-03-26 05:53:43,421 epoch 37 - iter 14/25 - loss 4.16374413 - samples/sec: 113.83 - lr: 0.300000 +2021-03-26 05:53:44,493 epoch 37 - iter 16/25 - loss 4.11268929 - samples/sec: 119.59 - lr: 0.300000 +2021-03-26 05:53:45,482 epoch 37 - iter 18/25 - loss 4.08254796 - samples/sec: 130.27 - lr: 0.300000 +2021-03-26 05:53:46,607 epoch 37 - iter 20/25 - loss 4.03820162 - samples/sec: 114.08 - lr: 0.300000 +2021-03-26 05:53:47,599 epoch 37 - iter 22/25 - loss 3.96956437 - samples/sec: 129.42 - lr: 0.300000 +2021-03-26 05:53:48,570 epoch 37 - iter 24/25 - loss 3.88680551 - samples/sec: 132.00 - lr: 0.300000 +2021-03-26 05:53:49,025 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:53:49,026 EPOCH 37 done: loss 3.9496 - lr 0.3000000 +2021-03-26 05:53:49,826 DEV : loss 6.98248815536499 - score 0.8924 +2021-03-26 05:53:49,851 BAD EPOCHS (no improvement): 1 +2021-03-26 05:53:49,852 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:53:50,995 epoch 38 - iter 2/25 - loss 4.03832924 - samples/sec: 112.15 - lr: 0.300000 +2021-03-26 05:53:51,988 epoch 38 - iter 4/25 - loss 3.38519096 - samples/sec: 129.16 - lr: 0.300000 +2021-03-26 05:53:52,950 epoch 38 - iter 6/25 - loss 3.72530087 - samples/sec: 133.17 - lr: 0.300000 +2021-03-26 05:53:54,155 epoch 38 - iter 8/25 - loss 3.67750609 - samples/sec: 106.36 - lr: 0.300000 +2021-03-26 05:53:55,130 epoch 38 - iter 10/25 - loss 3.67888119 - samples/sec: 131.49 - lr: 0.300000 +2021-03-26 05:53:56,153 epoch 38 - iter 12/25 - loss 3.70259313 - samples/sec: 125.41 - lr: 0.300000 +2021-03-26 05:53:57,202 epoch 38 - iter 14/25 - loss 3.85171386 - samples/sec: 122.12 - lr: 0.300000 +2021-03-26 05:53:58,218 epoch 38 - iter 16/25 - loss 3.86089347 - samples/sec: 126.26 - lr: 0.300000 +2021-03-26 05:53:59,198 epoch 38 - iter 18/25 - loss 3.88625879 - samples/sec: 130.94 - lr: 0.300000 +2021-03-26 05:54:00,240 epoch 38 - iter 20/25 - loss 3.87472203 - samples/sec: 123.05 - lr: 0.300000 +2021-03-26 05:54:01,183 epoch 38 - iter 22/25 - loss 3.76388594 - samples/sec: 135.94 - lr: 0.300000 +2021-03-26 05:54:02,138 epoch 38 - iter 24/25 - loss 3.86877202 - samples/sec: 134.32 - lr: 0.300000 +2021-03-26 05:54:02,580 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:54:02,580 EPOCH 38 done: loss 3.8287 - lr 0.3000000 +2021-03-26 05:54:03,366 DEV : loss 7.446681976318359 - score 0.8976 +2021-03-26 05:54:03,387 BAD EPOCHS (no improvement): 2 +2021-03-26 05:54:03,388 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:54:04,454 epoch 39 - iter 2/25 - loss 3.38113582 - samples/sec: 120.22 - lr: 0.300000 +2021-03-26 05:54:05,495 epoch 39 - iter 4/25 - loss 3.34563071 - samples/sec: 123.35 - lr: 0.300000 +2021-03-26 05:54:06,533 epoch 39 - iter 6/25 - loss 3.38525800 - samples/sec: 123.45 - lr: 0.300000 +2021-03-26 05:54:07,545 epoch 39 - iter 8/25 - loss 3.45749402 - samples/sec: 126.65 - lr: 0.300000 +2021-03-26 05:54:08,568 epoch 39 - iter 10/25 - loss 3.58208604 - samples/sec: 125.35 - lr: 0.300000 +2021-03-26 05:54:09,585 epoch 39 - iter 12/25 - loss 3.58137528 - samples/sec: 126.12 - lr: 0.300000 +2021-03-26 05:54:10,726 epoch 39 - iter 14/25 - loss 3.57692300 - samples/sec: 112.27 - lr: 0.300000 +2021-03-26 05:54:11,666 epoch 39 - iter 16/25 - loss 3.63784450 - samples/sec: 136.50 - lr: 0.300000 +2021-03-26 05:54:12,659 epoch 39 - iter 18/25 - loss 3.71754699 - samples/sec: 129.00 - lr: 0.300000 +2021-03-26 05:54:13,624 epoch 39 - iter 20/25 - loss 3.71790321 - samples/sec: 132.81 - lr: 0.300000 +2021-03-26 05:54:14,604 epoch 39 - iter 22/25 - loss 3.71617423 - samples/sec: 130.85 - lr: 0.300000 +2021-03-26 05:54:15,692 epoch 39 - iter 24/25 - loss 3.82966572 - samples/sec: 117.87 - lr: 0.300000 +2021-03-26 05:54:16,136 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:54:16,136 EPOCH 39 done: loss 3.7891 - lr 0.3000000 +2021-03-26 05:54:16,942 DEV : loss 6.701484680175781 - score 0.9033 +2021-03-26 05:54:16,966 BAD EPOCHS (no improvement): 0 +2021-03-26 05:54:26,975 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:54:28,066 epoch 40 - iter 2/25 - loss 3.69630694 - samples/sec: 117.74 - lr: 0.300000 +2021-03-26 05:54:29,004 epoch 40 - iter 4/25 - loss 3.42667872 - samples/sec: 136.54 - lr: 0.300000 +2021-03-26 05:54:30,129 epoch 40 - iter 6/25 - loss 3.64207637 - samples/sec: 113.97 - lr: 0.300000 +2021-03-26 05:54:31,179 epoch 40 - iter 8/25 - loss 3.56710127 - samples/sec: 122.08 - lr: 0.300000 +2021-03-26 05:54:32,170 epoch 40 - iter 10/25 - loss 3.51173098 - samples/sec: 129.30 - lr: 0.300000 +2021-03-26 05:54:33,167 epoch 40 - iter 12/25 - loss 3.52370836 - samples/sec: 128.57 - lr: 0.300000 +2021-03-26 05:54:34,201 epoch 40 - iter 14/25 - loss 3.64333616 - samples/sec: 124.09 - lr: 0.300000 +2021-03-26 05:54:35,280 epoch 40 - iter 16/25 - loss 3.65618315 - samples/sec: 118.80 - lr: 0.300000 +2021-03-26 05:54:36,367 epoch 40 - iter 18/25 - loss 3.66785228 - samples/sec: 118.29 - lr: 0.300000 +2021-03-26 05:54:37,362 epoch 40 - iter 20/25 - loss 3.67650574 - samples/sec: 128.92 - lr: 0.300000 +2021-03-26 05:54:38,307 epoch 40 - iter 22/25 - loss 3.73330291 - samples/sec: 135.55 - lr: 0.300000 +2021-03-26 05:54:39,254 epoch 40 - iter 24/25 - loss 3.69615401 - samples/sec: 135.49 - lr: 0.300000 +2021-03-26 05:54:39,667 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:54:39,667 EPOCH 40 done: loss 3.7016 - lr 0.3000000 +2021-03-26 05:54:40,457 DEV : loss 6.90902853012085 - score 0.8994 +2021-03-26 05:54:40,475 BAD EPOCHS (no improvement): 1 +2021-03-26 05:54:40,476 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:54:41,494 epoch 41 - iter 2/25 - loss 3.64263535 - samples/sec: 125.98 - lr: 0.300000 +2021-03-26 05:54:42,432 epoch 41 - iter 4/25 - loss 3.54664224 - samples/sec: 136.59 - lr: 0.300000 +2021-03-26 05:54:43,418 epoch 41 - iter 6/25 - loss 3.35265764 - samples/sec: 130.06 - lr: 0.300000 +2021-03-26 05:54:44,393 epoch 41 - iter 8/25 - loss 3.50885856 - samples/sec: 131.48 - lr: 0.300000 +2021-03-26 05:54:45,393 epoch 41 - iter 10/25 - loss 3.50825651 - samples/sec: 128.29 - lr: 0.300000 +2021-03-26 05:54:46,391 epoch 41 - iter 12/25 - loss 3.50486132 - samples/sec: 128.37 - lr: 0.300000 +2021-03-26 05:54:47,467 epoch 41 - iter 14/25 - loss 3.58481785 - samples/sec: 119.20 - lr: 0.300000 +2021-03-26 05:54:48,393 epoch 41 - iter 16/25 - loss 3.61369254 - samples/sec: 138.41 - lr: 0.300000 +2021-03-26 05:54:49,453 epoch 41 - iter 18/25 - loss 3.66834854 - samples/sec: 120.89 - lr: 0.300000 +2021-03-26 05:54:50,430 epoch 41 - iter 20/25 - loss 3.67402879 - samples/sec: 131.42 - lr: 0.300000 +2021-03-26 05:54:51,403 epoch 41 - iter 22/25 - loss 3.65755617 - samples/sec: 131.61 - lr: 0.300000 +2021-03-26 05:54:52,552 epoch 41 - iter 24/25 - loss 3.67933555 - samples/sec: 111.63 - lr: 0.300000 +2021-03-26 05:54:52,980 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:54:52,981 EPOCH 41 done: loss 3.6777 - lr 0.3000000 +2021-03-26 05:54:53,759 DEV : loss 7.189461708068848 - score 0.8944 +2021-03-26 05:54:53,777 BAD EPOCHS (no improvement): 2 +2021-03-26 05:54:53,778 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:54:54,731 epoch 42 - iter 2/25 - loss 3.30671835 - samples/sec: 134.50 - lr: 0.300000 +2021-03-26 05:54:55,799 epoch 42 - iter 4/25 - loss 3.40699106 - samples/sec: 120.18 - lr: 0.300000 +2021-03-26 05:54:56,863 epoch 42 - iter 6/25 - loss 3.40852686 - samples/sec: 120.49 - lr: 0.300000 +2021-03-26 05:54:57,894 epoch 42 - iter 8/25 - loss 3.28201953 - samples/sec: 124.31 - lr: 0.300000 +2021-03-26 05:54:58,942 epoch 42 - iter 10/25 - loss 3.39735591 - samples/sec: 122.35 - lr: 0.300000 +2021-03-26 05:55:00,003 epoch 42 - iter 12/25 - loss 3.49690189 - samples/sec: 120.76 - lr: 0.300000 +2021-03-26 05:55:01,012 epoch 42 - iter 14/25 - loss 3.53133660 - samples/sec: 127.05 - lr: 0.300000 +2021-03-26 05:55:02,057 epoch 42 - iter 16/25 - loss 3.58082850 - samples/sec: 122.79 - lr: 0.300000 +2021-03-26 05:55:03,084 epoch 42 - iter 18/25 - loss 3.54918623 - samples/sec: 124.72 - lr: 0.300000 +2021-03-26 05:55:03,990 epoch 42 - iter 20/25 - loss 3.49929521 - samples/sec: 141.54 - lr: 0.300000 +2021-03-26 05:55:04,965 epoch 42 - iter 22/25 - loss 3.52338445 - samples/sec: 131.47 - lr: 0.300000 +2021-03-26 05:55:06,106 epoch 42 - iter 24/25 - loss 3.55230616 - samples/sec: 112.34 - lr: 0.300000 +2021-03-26 05:55:06,499 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:55:06,500 EPOCH 42 done: loss 3.5816 - lr 0.3000000 +2021-03-26 05:55:07,294 DEV : loss 7.039179801940918 - score 0.8952 +2021-03-26 05:55:07,320 BAD EPOCHS (no improvement): 3 +2021-03-26 05:55:07,320 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:55:08,321 epoch 43 - iter 2/25 - loss 4.29221988 - samples/sec: 128.18 - lr: 0.300000 +2021-03-26 05:55:09,370 epoch 43 - iter 4/25 - loss 3.51265657 - samples/sec: 122.15 - lr: 0.300000 +2021-03-26 05:55:10,306 epoch 43 - iter 6/25 - loss 3.35663565 - samples/sec: 137.22 - lr: 0.300000 +2021-03-26 05:55:11,343 epoch 43 - iter 8/25 - loss 3.41256848 - samples/sec: 123.64 - lr: 0.300000 +2021-03-26 05:55:12,365 epoch 43 - iter 10/25 - loss 3.50655358 - samples/sec: 125.49 - lr: 0.300000 +2021-03-26 05:55:13,370 epoch 43 - iter 12/25 - loss 3.55152710 - samples/sec: 127.64 - lr: 0.300000 +2021-03-26 05:55:14,378 epoch 43 - iter 14/25 - loss 3.63799204 - samples/sec: 127.13 - lr: 0.300000 +2021-03-26 05:55:15,412 epoch 43 - iter 16/25 - loss 3.52342936 - samples/sec: 123.97 - lr: 0.300000 +2021-03-26 05:55:16,394 epoch 43 - iter 18/25 - loss 3.52740237 - samples/sec: 130.74 - lr: 0.300000 +2021-03-26 05:55:17,507 epoch 43 - iter 20/25 - loss 3.50906000 - samples/sec: 115.22 - lr: 0.300000 +2021-03-26 05:55:18,504 epoch 43 - iter 22/25 - loss 3.45362682 - samples/sec: 128.72 - lr: 0.300000 +2021-03-26 05:55:19,522 epoch 43 - iter 24/25 - loss 3.39437827 - samples/sec: 125.84 - lr: 0.300000 +2021-03-26 05:55:19,977 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:55:19,978 EPOCH 43 done: loss 3.4366 - lr 0.3000000 +2021-03-26 05:55:20,754 DEV : loss 6.94519567489624 - score 0.8942 +2021-03-26 05:55:20,777 BAD EPOCHS (no improvement): 4 +2021-03-26 05:55:20,778 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:55:21,913 epoch 44 - iter 2/25 - loss 3.44202733 - samples/sec: 112.97 - lr: 0.150000 +2021-03-26 05:55:22,894 epoch 44 - iter 4/25 - loss 3.22575986 - samples/sec: 130.56 - lr: 0.150000 +2021-03-26 05:55:23,893 epoch 44 - iter 6/25 - loss 3.08975744 - samples/sec: 128.30 - lr: 0.150000 +2021-03-26 05:55:24,957 epoch 44 - iter 8/25 - loss 3.07908553 - samples/sec: 120.50 - lr: 0.150000 +2021-03-26 05:55:25,926 epoch 44 - iter 10/25 - loss 3.09396088 - samples/sec: 132.28 - lr: 0.150000 +2021-03-26 05:55:26,981 epoch 44 - iter 12/25 - loss 3.13797561 - samples/sec: 121.52 - lr: 0.150000 +2021-03-26 05:55:28,053 epoch 44 - iter 14/25 - loss 3.04971560 - samples/sec: 119.52 - lr: 0.150000 +2021-03-26 05:55:29,089 epoch 44 - iter 16/25 - loss 3.02506705 - samples/sec: 123.69 - lr: 0.150000 +2021-03-26 05:55:30,105 epoch 44 - iter 18/25 - loss 3.03164853 - samples/sec: 126.20 - lr: 0.150000 +2021-03-26 05:55:31,151 epoch 44 - iter 20/25 - loss 2.96418314 - samples/sec: 122.63 - lr: 0.150000 +2021-03-26 05:55:32,211 epoch 44 - iter 22/25 - loss 2.97567326 - samples/sec: 121.02 - lr: 0.150000 +2021-03-26 05:55:33,208 epoch 44 - iter 24/25 - loss 3.00339601 - samples/sec: 128.47 - lr: 0.150000 +2021-03-26 05:55:33,678 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:55:33,679 EPOCH 44 done: loss 3.0213 - lr 0.1500000 +2021-03-26 05:55:34,556 DEV : loss 6.775360107421875 - score 0.9004 +2021-03-26 05:55:34,580 BAD EPOCHS (no improvement): 1 +2021-03-26 05:55:34,581 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:55:35,661 epoch 45 - iter 2/25 - loss 2.80927062 - samples/sec: 118.79 - lr: 0.150000 +2021-03-26 05:55:36,710 epoch 45 - iter 4/25 - loss 2.83806175 - samples/sec: 122.24 - lr: 0.150000 +2021-03-26 05:55:37,665 epoch 45 - iter 6/25 - loss 3.05464844 - samples/sec: 134.23 - lr: 0.150000 +2021-03-26 05:55:38,787 epoch 45 - iter 8/25 - loss 3.02497697 - samples/sec: 114.24 - lr: 0.150000 +2021-03-26 05:55:39,785 epoch 45 - iter 10/25 - loss 3.01038718 - samples/sec: 129.14 - lr: 0.150000 +2021-03-26 05:55:40,810 epoch 45 - iter 12/25 - loss 3.01171037 - samples/sec: 125.01 - lr: 0.150000 +2021-03-26 05:55:41,864 epoch 45 - iter 14/25 - loss 3.17433233 - samples/sec: 121.65 - lr: 0.150000 +2021-03-26 05:55:42,834 epoch 45 - iter 16/25 - loss 3.11452146 - samples/sec: 132.06 - lr: 0.150000 +2021-03-26 05:55:43,966 epoch 45 - iter 18/25 - loss 3.12591218 - samples/sec: 113.19 - lr: 0.150000 +2021-03-26 05:55:45,011 epoch 45 - iter 20/25 - loss 3.17873971 - samples/sec: 122.68 - lr: 0.150000 +2021-03-26 05:55:46,026 epoch 45 - iter 22/25 - loss 3.17179457 - samples/sec: 126.29 - lr: 0.150000 +2021-03-26 05:55:47,002 epoch 45 - iter 24/25 - loss 3.18488333 - samples/sec: 131.43 - lr: 0.150000 +2021-03-26 05:55:47,407 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:55:47,408 EPOCH 45 done: loss 3.1822 - lr 0.1500000 +2021-03-26 05:55:48,211 DEV : loss 6.951686859130859 - score 0.9 +2021-03-26 05:55:48,250 BAD EPOCHS (no improvement): 2 +2021-03-26 05:55:48,251 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:55:49,256 epoch 46 - iter 2/25 - loss 3.24467790 - samples/sec: 127.61 - lr: 0.150000 +2021-03-26 05:55:50,189 epoch 46 - iter 4/25 - loss 3.12835592 - samples/sec: 137.50 - lr: 0.150000 +2021-03-26 05:55:51,315 epoch 46 - iter 6/25 - loss 3.09644337 - samples/sec: 113.96 - lr: 0.150000 +2021-03-26 05:55:52,403 epoch 46 - iter 8/25 - loss 3.07437801 - samples/sec: 117.79 - lr: 0.150000 +2021-03-26 05:55:53,467 epoch 46 - iter 10/25 - loss 3.15177259 - samples/sec: 120.44 - lr: 0.150000 +2021-03-26 05:55:54,547 epoch 46 - iter 12/25 - loss 3.13121780 - samples/sec: 118.66 - lr: 0.150000 +2021-03-26 05:55:55,598 epoch 46 - iter 14/25 - loss 3.08818018 - samples/sec: 121.96 - lr: 0.150000 +2021-03-26 05:55:56,626 epoch 46 - iter 16/25 - loss 3.08784597 - samples/sec: 124.73 - lr: 0.150000 +2021-03-26 05:55:57,688 epoch 46 - iter 18/25 - loss 3.09030705 - samples/sec: 120.84 - lr: 0.150000 +2021-03-26 05:55:58,777 epoch 46 - iter 20/25 - loss 3.11507770 - samples/sec: 117.72 - lr: 0.150000 +2021-03-26 05:55:59,779 epoch 46 - iter 22/25 - loss 3.08774398 - samples/sec: 128.02 - lr: 0.150000 +2021-03-26 05:56:00,811 epoch 46 - iter 24/25 - loss 3.04504895 - samples/sec: 124.23 - lr: 0.150000 +2021-03-26 05:56:01,215 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:56:01,216 EPOCH 46 done: loss 3.0049 - lr 0.1500000 +2021-03-26 05:56:02,006 DEV : loss 6.751981258392334 - score 0.9 +2021-03-26 05:56:02,031 BAD EPOCHS (no improvement): 3 +2021-03-26 05:56:02,032 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:56:02,968 epoch 47 - iter 2/25 - loss 2.74569619 - samples/sec: 137.08 - lr: 0.150000 +2021-03-26 05:56:04,009 epoch 47 - iter 4/25 - loss 3.09366006 - samples/sec: 123.16 - lr: 0.150000 +2021-03-26 05:56:05,056 epoch 47 - iter 6/25 - loss 2.99806523 - samples/sec: 122.60 - lr: 0.150000 +2021-03-26 05:56:06,063 epoch 47 - iter 8/25 - loss 2.96191627 - samples/sec: 127.32 - lr: 0.150000 +2021-03-26 05:56:07,134 epoch 47 - iter 10/25 - loss 2.90949333 - samples/sec: 119.73 - lr: 0.150000 +2021-03-26 05:56:08,134 epoch 47 - iter 12/25 - loss 2.87643186 - samples/sec: 128.13 - lr: 0.150000 +2021-03-26 05:56:09,313 epoch 47 - iter 14/25 - loss 2.81540270 - samples/sec: 108.75 - lr: 0.150000 +2021-03-26 05:56:10,362 epoch 47 - iter 16/25 - loss 2.88302286 - samples/sec: 122.20 - lr: 0.150000 +2021-03-26 05:56:11,369 epoch 47 - iter 18/25 - loss 2.93328830 - samples/sec: 127.35 - lr: 0.150000 +2021-03-26 05:56:12,420 epoch 47 - iter 20/25 - loss 2.91642751 - samples/sec: 121.90 - lr: 0.150000 +2021-03-26 05:56:13,327 epoch 47 - iter 22/25 - loss 2.92303073 - samples/sec: 141.27 - lr: 0.150000 +2021-03-26 05:56:14,309 epoch 47 - iter 24/25 - loss 2.90322950 - samples/sec: 130.61 - lr: 0.150000 +2021-03-26 05:56:14,758 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:56:14,759 EPOCH 47 done: loss 2.9369 - lr 0.1500000 +2021-03-26 05:56:15,537 DEV : loss 6.756464958190918 - score 0.902 +2021-03-26 05:56:15,562 BAD EPOCHS (no improvement): 4 +2021-03-26 05:56:15,563 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:56:16,562 epoch 48 - iter 2/25 - loss 2.99353659 - samples/sec: 128.20 - lr: 0.075000 +2021-03-26 05:56:17,661 epoch 48 - iter 4/25 - loss 2.75419962 - samples/sec: 116.69 - lr: 0.075000 +2021-03-26 05:56:18,679 epoch 48 - iter 6/25 - loss 2.74369395 - samples/sec: 125.97 - lr: 0.075000 +2021-03-26 05:56:19,754 epoch 48 - iter 8/25 - loss 2.73128772 - samples/sec: 119.34 - lr: 0.075000 +2021-03-26 05:56:20,688 epoch 48 - iter 10/25 - loss 2.61472061 - samples/sec: 137.29 - lr: 0.075000 +2021-03-26 05:56:21,723 epoch 48 - iter 12/25 - loss 2.70608862 - samples/sec: 123.87 - lr: 0.075000 +2021-03-26 05:56:22,649 epoch 48 - iter 14/25 - loss 2.71813272 - samples/sec: 138.55 - lr: 0.075000 +2021-03-26 05:56:23,595 epoch 48 - iter 16/25 - loss 2.72055332 - samples/sec: 135.69 - lr: 0.075000 +2021-03-26 05:56:24,628 epoch 48 - iter 18/25 - loss 2.67474678 - samples/sec: 124.10 - lr: 0.075000 +2021-03-26 05:56:25,615 epoch 48 - iter 20/25 - loss 2.65116678 - samples/sec: 129.82 - lr: 0.075000 +2021-03-26 05:56:26,612 epoch 48 - iter 22/25 - loss 2.65450315 - samples/sec: 128.66 - lr: 0.075000 +2021-03-26 05:56:27,641 epoch 48 - iter 24/25 - loss 2.64664620 - samples/sec: 124.63 - lr: 0.075000 +2021-03-26 05:56:28,043 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:56:28,044 EPOCH 48 done: loss 2.6319 - lr 0.0750000 +2021-03-26 05:56:28,873 DEV : loss 6.665907859802246 - score 0.9054 +2021-03-26 05:56:28,901 BAD EPOCHS (no improvement): 0 +2021-03-26 05:56:38,673 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:56:39,835 epoch 49 - iter 2/25 - loss 2.66986585 - samples/sec: 110.39 - lr: 0.075000 +2021-03-26 05:56:40,851 epoch 49 - iter 4/25 - loss 2.79160547 - samples/sec: 126.40 - lr: 0.075000 +2021-03-26 05:56:41,909 epoch 49 - iter 6/25 - loss 2.87581944 - samples/sec: 121.07 - lr: 0.075000 +2021-03-26 05:56:42,900 epoch 49 - iter 8/25 - loss 2.73760222 - samples/sec: 129.40 - lr: 0.075000 +2021-03-26 05:56:43,859 epoch 49 - iter 10/25 - loss 2.65547196 - samples/sec: 133.67 - lr: 0.075000 +2021-03-26 05:56:44,835 epoch 49 - iter 12/25 - loss 2.72983141 - samples/sec: 131.25 - lr: 0.075000 +2021-03-26 05:56:45,777 epoch 49 - iter 14/25 - loss 2.76542152 - samples/sec: 136.19 - lr: 0.075000 +2021-03-26 05:56:46,686 epoch 49 - iter 16/25 - loss 2.71359808 - samples/sec: 141.01 - lr: 0.075000 +2021-03-26 05:56:47,654 epoch 49 - iter 18/25 - loss 2.68785766 - samples/sec: 132.43 - lr: 0.075000 +2021-03-26 05:56:48,665 epoch 49 - iter 20/25 - loss 2.70319249 - samples/sec: 126.81 - lr: 0.075000 +2021-03-26 05:56:49,680 epoch 49 - iter 22/25 - loss 2.72296827 - samples/sec: 126.19 - lr: 0.075000 +2021-03-26 05:56:50,751 epoch 49 - iter 24/25 - loss 2.70314044 - samples/sec: 119.85 - lr: 0.075000 +2021-03-26 05:56:51,173 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:56:51,174 EPOCH 49 done: loss 2.7032 - lr 0.0750000 +2021-03-26 05:56:51,962 DEV : loss 6.667111396789551 - score 0.9042 +2021-03-26 05:56:51,980 BAD EPOCHS (no improvement): 1 +2021-03-26 05:56:51,981 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:56:53,059 epoch 50 - iter 2/25 - loss 2.86935806 - samples/sec: 118.93 - lr: 0.075000 +2021-03-26 05:56:54,103 epoch 50 - iter 4/25 - loss 2.68313444 - samples/sec: 122.80 - lr: 0.075000 +2021-03-26 05:56:55,167 epoch 50 - iter 6/25 - loss 2.58666865 - samples/sec: 120.48 - lr: 0.075000 +2021-03-26 05:56:56,130 epoch 50 - iter 8/25 - loss 2.52996308 - samples/sec: 132.96 - lr: 0.075000 +2021-03-26 05:56:57,128 epoch 50 - iter 10/25 - loss 2.59376135 - samples/sec: 128.46 - lr: 0.075000 +2021-03-26 05:56:58,198 epoch 50 - iter 12/25 - loss 2.67243918 - samples/sec: 119.83 - lr: 0.075000 +2021-03-26 05:56:59,246 epoch 50 - iter 14/25 - loss 2.64159279 - samples/sec: 122.25 - lr: 0.075000 +2021-03-26 05:57:00,280 epoch 50 - iter 16/25 - loss 2.61747177 - samples/sec: 124.03 - lr: 0.075000 +2021-03-26 05:57:01,333 epoch 50 - iter 18/25 - loss 2.63498076 - samples/sec: 121.65 - lr: 0.075000 +2021-03-26 05:57:02,449 epoch 50 - iter 20/25 - loss 2.62919823 - samples/sec: 114.84 - lr: 0.075000 +2021-03-26 05:57:03,553 epoch 50 - iter 22/25 - loss 2.61773355 - samples/sec: 116.14 - lr: 0.075000 +2021-03-26 05:57:04,578 epoch 50 - iter 24/25 - loss 2.62837592 - samples/sec: 125.06 - lr: 0.075000 +2021-03-26 05:57:05,003 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:57:05,004 EPOCH 50 done: loss 2.5942 - lr 0.0750000 +2021-03-26 05:57:05,774 DEV : loss 6.875411033630371 - score 0.9 +2021-03-26 05:57:05,799 BAD EPOCHS (no improvement): 2 +2021-03-26 05:57:05,800 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:57:06,859 epoch 51 - iter 2/25 - loss 2.71137524 - samples/sec: 121.06 - lr: 0.075000 +2021-03-26 05:57:07,886 epoch 51 - iter 4/25 - loss 2.77748650 - samples/sec: 124.85 - lr: 0.075000 +2021-03-26 05:57:09,027 epoch 51 - iter 6/25 - loss 2.73082197 - samples/sec: 112.36 - lr: 0.075000 +2021-03-26 05:57:10,070 epoch 51 - iter 8/25 - loss 2.65448046 - samples/sec: 122.80 - lr: 0.075000 +2021-03-26 05:57:11,067 epoch 51 - iter 10/25 - loss 2.59957895 - samples/sec: 128.71 - lr: 0.075000 +2021-03-26 05:57:12,223 epoch 51 - iter 12/25 - loss 2.62106667 - samples/sec: 110.86 - lr: 0.075000 +2021-03-26 05:57:13,274 epoch 51 - iter 14/25 - loss 2.60977428 - samples/sec: 121.94 - lr: 0.075000 +2021-03-26 05:57:14,214 epoch 51 - iter 16/25 - loss 2.58111915 - samples/sec: 136.37 - lr: 0.075000 +2021-03-26 05:57:15,317 epoch 51 - iter 18/25 - loss 2.58968723 - samples/sec: 116.18 - lr: 0.075000 +2021-03-26 05:57:16,302 epoch 51 - iter 20/25 - loss 2.55809421 - samples/sec: 130.33 - lr: 0.075000 +2021-03-26 05:57:17,395 epoch 51 - iter 22/25 - loss 2.54893119 - samples/sec: 117.22 - lr: 0.075000 +2021-03-26 05:57:18,324 epoch 51 - iter 24/25 - loss 2.52646407 - samples/sec: 138.03 - lr: 0.075000 +2021-03-26 05:57:18,765 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:57:18,766 EPOCH 51 done: loss 2.5377 - lr 0.0750000 +2021-03-26 05:57:19,530 DEV : loss 6.827261924743652 - score 0.9048 +2021-03-26 05:57:19,555 BAD EPOCHS (no improvement): 3 +2021-03-26 05:57:19,556 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:57:20,486 epoch 52 - iter 2/25 - loss 2.62309206 - samples/sec: 137.95 - lr: 0.075000 +2021-03-26 05:57:21,560 epoch 52 - iter 4/25 - loss 2.72515994 - samples/sec: 119.25 - lr: 0.075000 +2021-03-26 05:57:22,545 epoch 52 - iter 6/25 - loss 2.54840565 - samples/sec: 130.31 - lr: 0.075000 +2021-03-26 05:57:23,689 epoch 52 - iter 8/25 - loss 2.53041878 - samples/sec: 112.02 - lr: 0.075000 +2021-03-26 05:57:24,729 epoch 52 - iter 10/25 - loss 2.59141297 - samples/sec: 123.23 - lr: 0.075000 +2021-03-26 05:57:25,787 epoch 52 - iter 12/25 - loss 2.63222436 - samples/sec: 121.13 - lr: 0.075000 +2021-03-26 05:57:26,746 epoch 52 - iter 14/25 - loss 2.52890756 - samples/sec: 133.69 - lr: 0.075000 +2021-03-26 05:57:27,744 epoch 52 - iter 16/25 - loss 2.55082352 - samples/sec: 128.52 - lr: 0.075000 +2021-03-26 05:57:28,683 epoch 52 - iter 18/25 - loss 2.52916779 - samples/sec: 136.66 - lr: 0.075000 +2021-03-26 05:57:29,610 epoch 52 - iter 20/25 - loss 2.53436933 - samples/sec: 138.29 - lr: 0.075000 +2021-03-26 05:57:30,691 epoch 52 - iter 22/25 - loss 2.54360505 - samples/sec: 118.58 - lr: 0.075000 +2021-03-26 05:57:31,760 epoch 52 - iter 24/25 - loss 2.58355175 - samples/sec: 120.47 - lr: 0.075000 +2021-03-26 05:57:32,163 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:57:32,163 EPOCH 52 done: loss 2.6029 - lr 0.0750000 +2021-03-26 05:57:32,968 DEV : loss 6.790990829467773 - score 0.9012 +2021-03-26 05:57:32,997 BAD EPOCHS (no improvement): 4 +2021-03-26 05:57:32,998 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:57:34,101 epoch 53 - iter 2/25 - loss 2.89371443 - samples/sec: 116.19 - lr: 0.037500 +2021-03-26 05:57:35,161 epoch 53 - iter 4/25 - loss 2.57764858 - samples/sec: 120.96 - lr: 0.037500 +2021-03-26 05:57:36,153 epoch 53 - iter 6/25 - loss 2.53048742 - samples/sec: 129.16 - lr: 0.037500 +2021-03-26 05:57:37,191 epoch 53 - iter 8/25 - loss 2.55369267 - samples/sec: 123.51 - lr: 0.037500 +2021-03-26 05:57:38,300 epoch 53 - iter 10/25 - loss 2.53974638 - samples/sec: 116.08 - lr: 0.037500 +2021-03-26 05:57:39,345 epoch 53 - iter 12/25 - loss 2.58391945 - samples/sec: 122.86 - lr: 0.037500 +2021-03-26 05:57:40,439 epoch 53 - iter 14/25 - loss 2.63310596 - samples/sec: 117.18 - lr: 0.037500 +2021-03-26 05:57:41,538 epoch 53 - iter 16/25 - loss 2.67614898 - samples/sec: 116.64 - lr: 0.037500 +2021-03-26 05:57:42,489 epoch 53 - iter 18/25 - loss 2.66936096 - samples/sec: 135.00 - lr: 0.037500 +2021-03-26 05:57:43,426 epoch 53 - iter 20/25 - loss 2.61346560 - samples/sec: 136.88 - lr: 0.037500 +2021-03-26 05:57:44,415 epoch 53 - iter 22/25 - loss 2.58636801 - samples/sec: 129.55 - lr: 0.037500 +2021-03-26 05:57:45,403 epoch 53 - iter 24/25 - loss 2.63187471 - samples/sec: 129.77 - lr: 0.037500 +2021-03-26 05:57:45,784 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:57:45,785 EPOCH 53 done: loss 2.6194 - lr 0.0375000 +2021-03-26 05:57:46,555 DEV : loss 6.770951747894287 - score 0.9048 +2021-03-26 05:57:46,574 BAD EPOCHS (no improvement): 1 +2021-03-26 05:57:46,575 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:57:47,623 epoch 54 - iter 2/25 - loss 2.28589094 - samples/sec: 122.26 - lr: 0.037500 +2021-03-26 05:57:48,591 epoch 54 - iter 4/25 - loss 2.56535906 - samples/sec: 132.52 - lr: 0.037500 +2021-03-26 05:57:49,602 epoch 54 - iter 6/25 - loss 2.55666661 - samples/sec: 126.74 - lr: 0.037500 +2021-03-26 05:57:50,590 epoch 54 - iter 8/25 - loss 2.60720205 - samples/sec: 129.76 - lr: 0.037500 +2021-03-26 05:57:51,707 epoch 54 - iter 10/25 - loss 2.68733101 - samples/sec: 114.79 - lr: 0.037500 +2021-03-26 05:57:52,840 epoch 54 - iter 12/25 - loss 2.67932904 - samples/sec: 113.16 - lr: 0.037500 +2021-03-26 05:57:53,903 epoch 54 - iter 14/25 - loss 2.65863521 - samples/sec: 120.56 - lr: 0.037500 +2021-03-26 05:57:54,898 epoch 54 - iter 16/25 - loss 2.67555900 - samples/sec: 128.76 - lr: 0.037500 +2021-03-26 05:57:55,990 epoch 54 - iter 18/25 - loss 2.66907661 - samples/sec: 117.46 - lr: 0.037500 +2021-03-26 05:57:56,986 epoch 54 - iter 20/25 - loss 2.65760111 - samples/sec: 128.68 - lr: 0.037500 +2021-03-26 05:57:58,209 epoch 54 - iter 22/25 - loss 2.64885169 - samples/sec: 104.75 - lr: 0.037500 +2021-03-26 05:57:59,237 epoch 54 - iter 24/25 - loss 2.65786774 - samples/sec: 124.80 - lr: 0.037500 +2021-03-26 05:57:59,634 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:57:59,635 EPOCH 54 done: loss 2.6281 - lr 0.0375000 +2021-03-26 05:58:00,394 DEV : loss 6.769601821899414 - score 0.9036 +2021-03-26 05:58:00,419 BAD EPOCHS (no improvement): 2 +2021-03-26 05:58:00,420 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:58:01,385 epoch 55 - iter 2/25 - loss 2.57250500 - samples/sec: 132.96 - lr: 0.037500 +2021-03-26 05:58:02,363 epoch 55 - iter 4/25 - loss 2.56209105 - samples/sec: 131.00 - lr: 0.037500 +2021-03-26 05:58:03,519 epoch 55 - iter 6/25 - loss 2.55206974 - samples/sec: 110.90 - lr: 0.037500 +2021-03-26 05:58:04,451 epoch 55 - iter 8/25 - loss 2.63297400 - samples/sec: 137.55 - lr: 0.037500 +2021-03-26 05:58:05,438 epoch 55 - iter 10/25 - loss 2.64202976 - samples/sec: 129.83 - lr: 0.037500 +2021-03-26 05:58:06,455 epoch 55 - iter 12/25 - loss 2.58580096 - samples/sec: 126.06 - lr: 0.037500 +2021-03-26 05:58:07,536 epoch 55 - iter 14/25 - loss 2.55834146 - samples/sec: 118.54 - lr: 0.037500 +2021-03-26 05:58:08,675 epoch 55 - iter 16/25 - loss 2.57096677 - samples/sec: 112.55 - lr: 0.037500 +2021-03-26 05:58:09,650 epoch 55 - iter 18/25 - loss 2.56309658 - samples/sec: 131.40 - lr: 0.037500 +2021-03-26 05:58:10,805 epoch 55 - iter 20/25 - loss 2.53701287 - samples/sec: 111.09 - lr: 0.037500 +2021-03-26 05:58:12,006 epoch 55 - iter 22/25 - loss 2.56861101 - samples/sec: 106.71 - lr: 0.037500 +2021-03-26 05:58:13,012 epoch 55 - iter 24/25 - loss 2.53881068 - samples/sec: 127.36 - lr: 0.037500 +2021-03-26 05:58:13,431 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:58:13,433 EPOCH 55 done: loss 2.5274 - lr 0.0375000 +2021-03-26 05:58:14,198 DEV : loss 6.722831726074219 - score 0.9034 +2021-03-26 05:58:14,222 BAD EPOCHS (no improvement): 3 +2021-03-26 05:58:14,223 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:58:15,235 epoch 56 - iter 2/25 - loss 3.06336522 - samples/sec: 126.75 - lr: 0.037500 +2021-03-26 05:58:16,235 epoch 56 - iter 4/25 - loss 2.66866070 - samples/sec: 128.31 - lr: 0.037500 +2021-03-26 05:58:17,254 epoch 56 - iter 6/25 - loss 2.59741449 - samples/sec: 125.72 - lr: 0.037500 +2021-03-26 05:58:18,230 epoch 56 - iter 8/25 - loss 2.50272012 - samples/sec: 131.39 - lr: 0.037500 +2021-03-26 05:58:19,459 epoch 56 - iter 10/25 - loss 2.50687180 - samples/sec: 104.40 - lr: 0.037500 +2021-03-26 05:58:20,490 epoch 56 - iter 12/25 - loss 2.49787994 - samples/sec: 124.27 - lr: 0.037500 +2021-03-26 05:58:21,569 epoch 56 - iter 14/25 - loss 2.48121374 - samples/sec: 118.80 - lr: 0.037500 +2021-03-26 05:58:22,634 epoch 56 - iter 16/25 - loss 2.48644322 - samples/sec: 120.45 - lr: 0.037500 +2021-03-26 05:58:23,633 epoch 56 - iter 18/25 - loss 2.49157370 - samples/sec: 128.30 - lr: 0.037500 +2021-03-26 05:58:24,723 epoch 56 - iter 20/25 - loss 2.48946971 - samples/sec: 117.56 - lr: 0.037500 +2021-03-26 05:58:25,809 epoch 56 - iter 22/25 - loss 2.49483615 - samples/sec: 118.10 - lr: 0.037500 +2021-03-26 05:58:26,827 epoch 56 - iter 24/25 - loss 2.50274724 - samples/sec: 125.92 - lr: 0.037500 +2021-03-26 05:58:27,323 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:58:27,324 EPOCH 56 done: loss 2.4933 - lr 0.0375000 +2021-03-26 05:58:28,121 DEV : loss 6.749120235443115 - score 0.9046 +2021-03-26 05:58:28,147 BAD EPOCHS (no improvement): 4 +2021-03-26 05:58:28,148 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:58:29,243 epoch 57 - iter 2/25 - loss 2.20577013 - samples/sec: 117.11 - lr: 0.018750 +2021-03-26 05:58:30,440 epoch 57 - iter 4/25 - loss 2.51310360 - samples/sec: 107.03 - lr: 0.018750 +2021-03-26 05:58:31,442 epoch 57 - iter 6/25 - loss 2.57224830 - samples/sec: 127.91 - lr: 0.018750 +2021-03-26 05:58:32,495 epoch 57 - iter 8/25 - loss 2.51049601 - samples/sec: 121.98 - lr: 0.018750 +2021-03-26 05:58:33,803 epoch 57 - iter 10/25 - loss 2.53363069 - samples/sec: 97.94 - lr: 0.018750 +2021-03-26 05:58:34,897 epoch 57 - iter 12/25 - loss 2.46666911 - samples/sec: 117.34 - lr: 0.018750 +2021-03-26 05:58:35,940 epoch 57 - iter 14/25 - loss 2.48624198 - samples/sec: 122.90 - lr: 0.018750 +2021-03-26 05:58:37,023 epoch 57 - iter 16/25 - loss 2.53736014 - samples/sec: 118.44 - lr: 0.018750 +2021-03-26 05:58:37,958 epoch 57 - iter 18/25 - loss 2.54605499 - samples/sec: 137.17 - lr: 0.018750 +2021-03-26 05:58:38,985 epoch 57 - iter 20/25 - loss 2.56636576 - samples/sec: 124.85 - lr: 0.018750 +2021-03-26 05:58:40,040 epoch 57 - iter 22/25 - loss 2.59158968 - samples/sec: 121.54 - lr: 0.018750 +2021-03-26 05:58:41,065 epoch 57 - iter 24/25 - loss 2.56697516 - samples/sec: 125.03 - lr: 0.018750 +2021-03-26 05:58:41,446 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:58:41,447 EPOCH 57 done: loss 2.5322 - lr 0.0187500 +2021-03-26 05:58:42,214 DEV : loss 6.725451946258545 - score 0.9042 +2021-03-26 05:58:42,240 BAD EPOCHS (no improvement): 1 +2021-03-26 05:58:42,241 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:58:43,198 epoch 58 - iter 2/25 - loss 2.39105058 - samples/sec: 133.89 - lr: 0.018750 +2021-03-26 05:58:44,218 epoch 58 - iter 4/25 - loss 2.44910604 - samples/sec: 125.74 - lr: 0.018750 +2021-03-26 05:58:45,178 epoch 58 - iter 6/25 - loss 2.47911966 - samples/sec: 133.55 - lr: 0.018750 +2021-03-26 05:58:46,280 epoch 58 - iter 8/25 - loss 2.48556039 - samples/sec: 116.26 - lr: 0.018750 +2021-03-26 05:58:47,264 epoch 58 - iter 10/25 - loss 2.55912685 - samples/sec: 130.36 - lr: 0.018750 +2021-03-26 05:58:48,195 epoch 58 - iter 12/25 - loss 2.43912268 - samples/sec: 137.76 - lr: 0.018750 +2021-03-26 05:58:49,124 epoch 58 - iter 14/25 - loss 2.48512311 - samples/sec: 138.01 - lr: 0.018750 +2021-03-26 05:58:50,078 epoch 58 - iter 16/25 - loss 2.50301301 - samples/sec: 134.37 - lr: 0.018750 +2021-03-26 05:58:51,198 epoch 58 - iter 18/25 - loss 2.48813513 - samples/sec: 114.42 - lr: 0.018750 +2021-03-26 05:58:52,204 epoch 58 - iter 20/25 - loss 2.48385862 - samples/sec: 127.48 - lr: 0.018750 +2021-03-26 05:58:53,224 epoch 58 - iter 22/25 - loss 2.44755840 - samples/sec: 125.61 - lr: 0.018750 +2021-03-26 05:58:54,287 epoch 58 - iter 24/25 - loss 2.44767216 - samples/sec: 120.70 - lr: 0.018750 +2021-03-26 05:58:54,755 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:58:54,755 EPOCH 58 done: loss 2.4693 - lr 0.0187500 +2021-03-26 05:58:55,549 DEV : loss 6.709056854248047 - score 0.9034 +2021-03-26 05:58:55,568 BAD EPOCHS (no improvement): 2 +2021-03-26 05:58:55,568 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:58:56,501 epoch 59 - iter 2/25 - loss 2.32147455 - samples/sec: 137.39 - lr: 0.018750 +2021-03-26 05:58:57,557 epoch 59 - iter 4/25 - loss 2.47985613 - samples/sec: 121.48 - lr: 0.018750 +2021-03-26 05:58:58,499 epoch 59 - iter 6/25 - loss 2.58878783 - samples/sec: 136.14 - lr: 0.018750 +2021-03-26 05:58:59,479 epoch 59 - iter 8/25 - loss 2.46861984 - samples/sec: 130.79 - lr: 0.018750 +2021-03-26 05:59:00,465 epoch 59 - iter 10/25 - loss 2.48236761 - samples/sec: 130.03 - lr: 0.018750 +2021-03-26 05:59:01,412 epoch 59 - iter 12/25 - loss 2.56132956 - samples/sec: 135.33 - lr: 0.018750 +2021-03-26 05:59:02,467 epoch 59 - iter 14/25 - loss 2.55461029 - samples/sec: 121.54 - lr: 0.018750 +2021-03-26 05:59:03,520 epoch 59 - iter 16/25 - loss 2.49694520 - samples/sec: 121.79 - lr: 0.018750 +2021-03-26 05:59:04,445 epoch 59 - iter 18/25 - loss 2.47332254 - samples/sec: 138.50 - lr: 0.018750 +2021-03-26 05:59:05,536 epoch 59 - iter 20/25 - loss 2.46696305 - samples/sec: 117.49 - lr: 0.018750 +2021-03-26 05:59:06,508 epoch 59 - iter 22/25 - loss 2.48456865 - samples/sec: 132.12 - lr: 0.018750 +2021-03-26 05:59:07,548 epoch 59 - iter 24/25 - loss 2.47716364 - samples/sec: 123.35 - lr: 0.018750 +2021-03-26 05:59:07,990 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:59:07,991 EPOCH 59 done: loss 2.4872 - lr 0.0187500 +2021-03-26 05:59:08,798 DEV : loss 6.735278129577637 - score 0.905 +2021-03-26 05:59:08,823 BAD EPOCHS (no improvement): 3 +2021-03-26 05:59:08,824 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:59:09,809 epoch 60 - iter 2/25 - loss 1.87354106 - samples/sec: 130.31 - lr: 0.018750 +2021-03-26 05:59:10,874 epoch 60 - iter 4/25 - loss 1.79272857 - samples/sec: 120.32 - lr: 0.018750 +2021-03-26 05:59:12,177 epoch 60 - iter 6/25 - loss 2.07879815 - samples/sec: 98.42 - lr: 0.018750 +2021-03-26 05:59:13,083 epoch 60 - iter 8/25 - loss 2.28630097 - samples/sec: 141.74 - lr: 0.018750 +2021-03-26 05:59:14,132 epoch 60 - iter 10/25 - loss 2.30058919 - samples/sec: 122.18 - lr: 0.018750 +2021-03-26 05:59:15,020 epoch 60 - iter 12/25 - loss 2.32238834 - samples/sec: 144.28 - lr: 0.018750 +2021-03-26 05:59:16,011 epoch 60 - iter 14/25 - loss 2.34343993 - samples/sec: 129.45 - lr: 0.018750 +2021-03-26 05:59:16,958 epoch 60 - iter 16/25 - loss 2.39775931 - samples/sec: 135.37 - lr: 0.018750 +2021-03-26 05:59:18,001 epoch 60 - iter 18/25 - loss 2.39503040 - samples/sec: 122.85 - lr: 0.018750 +2021-03-26 05:59:19,016 epoch 60 - iter 20/25 - loss 2.40990252 - samples/sec: 126.40 - lr: 0.018750 +2021-03-26 05:59:20,003 epoch 60 - iter 22/25 - loss 2.41649341 - samples/sec: 129.95 - lr: 0.018750 +2021-03-26 05:59:20,937 epoch 60 - iter 24/25 - loss 2.42599278 - samples/sec: 137.34 - lr: 0.018750 +2021-03-26 05:59:21,408 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:59:21,408 EPOCH 60 done: loss 2.4178 - lr 0.0187500 +2021-03-26 05:59:22,184 DEV : loss 6.739635467529297 - score 0.9058 +2021-03-26 05:59:22,209 BAD EPOCHS (no improvement): 0 +2021-03-26 05:59:31,905 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:59:33,017 epoch 61 - iter 2/25 - loss 2.60011214 - samples/sec: 115.35 - lr: 0.018750 +2021-03-26 05:59:34,162 epoch 61 - iter 4/25 - loss 2.52214578 - samples/sec: 112.02 - lr: 0.018750 +2021-03-26 05:59:35,247 epoch 61 - iter 6/25 - loss 2.38890078 - samples/sec: 118.01 - lr: 0.018750 +2021-03-26 05:59:36,290 epoch 61 - iter 8/25 - loss 2.56866135 - samples/sec: 122.99 - lr: 0.018750 +2021-03-26 05:59:37,174 epoch 61 - iter 10/25 - loss 2.39370874 - samples/sec: 144.91 - lr: 0.018750 +2021-03-26 05:59:38,153 epoch 61 - iter 12/25 - loss 2.40410967 - samples/sec: 131.03 - lr: 0.018750 +2021-03-26 05:59:39,170 epoch 61 - iter 14/25 - loss 2.44239428 - samples/sec: 126.04 - lr: 0.018750 +2021-03-26 05:59:40,165 epoch 61 - iter 16/25 - loss 2.39614346 - samples/sec: 128.80 - lr: 0.018750 +2021-03-26 05:59:41,158 epoch 61 - iter 18/25 - loss 2.44974503 - samples/sec: 129.07 - lr: 0.018750 +2021-03-26 05:59:42,118 epoch 61 - iter 20/25 - loss 2.46324108 - samples/sec: 133.63 - lr: 0.018750 +2021-03-26 05:59:43,074 epoch 61 - iter 22/25 - loss 2.44603983 - samples/sec: 134.01 - lr: 0.018750 +2021-03-26 05:59:44,176 epoch 61 - iter 24/25 - loss 2.48440110 - samples/sec: 116.33 - lr: 0.018750 +2021-03-26 05:59:44,584 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:59:44,585 EPOCH 61 done: loss 2.4975 - lr 0.0187500 +2021-03-26 05:59:45,362 DEV : loss 6.744481563568115 - score 0.9054 +2021-03-26 05:59:45,384 BAD EPOCHS (no improvement): 1 +2021-03-26 05:59:45,385 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:59:46,348 epoch 62 - iter 2/25 - loss 2.30863678 - samples/sec: 133.17 - lr: 0.018750 +2021-03-26 05:59:47,293 epoch 62 - iter 4/25 - loss 2.49650627 - samples/sec: 135.68 - lr: 0.018750 +2021-03-26 05:59:48,201 epoch 62 - iter 6/25 - loss 2.44710239 - samples/sec: 141.16 - lr: 0.018750 +2021-03-26 05:59:49,260 epoch 62 - iter 8/25 - loss 2.52125069 - samples/sec: 121.07 - lr: 0.018750 +2021-03-26 05:59:50,292 epoch 62 - iter 10/25 - loss 2.56077802 - samples/sec: 124.27 - lr: 0.018750 +2021-03-26 05:59:51,382 epoch 62 - iter 12/25 - loss 2.61480961 - samples/sec: 117.56 - lr: 0.018750 +2021-03-26 05:59:52,553 epoch 62 - iter 14/25 - loss 2.61970052 - samples/sec: 109.63 - lr: 0.018750 +2021-03-26 05:59:53,515 epoch 62 - iter 16/25 - loss 2.57987718 - samples/sec: 133.27 - lr: 0.018750 +2021-03-26 05:59:54,569 epoch 62 - iter 18/25 - loss 2.53430665 - samples/sec: 121.62 - lr: 0.018750 +2021-03-26 05:59:55,656 epoch 62 - iter 20/25 - loss 2.52618066 - samples/sec: 117.95 - lr: 0.018750 +2021-03-26 05:59:56,860 epoch 62 - iter 22/25 - loss 2.53868995 - samples/sec: 106.47 - lr: 0.018750 +2021-03-26 05:59:58,000 epoch 62 - iter 24/25 - loss 2.54573082 - samples/sec: 112.49 - lr: 0.018750 +2021-03-26 05:59:58,443 ---------------------------------------------------------------------------------------------------- +2021-03-26 05:59:58,445 EPOCH 62 done: loss 2.5331 - lr 0.0187500 +2021-03-26 05:59:59,231 DEV : loss 6.735997676849365 - score 0.9042 +2021-03-26 05:59:59,257 BAD EPOCHS (no improvement): 2 +2021-03-26 05:59:59,257 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:00:00,271 epoch 63 - iter 2/25 - loss 2.81712377 - samples/sec: 126.50 - lr: 0.018750 +2021-03-26 06:00:01,333 epoch 63 - iter 4/25 - loss 2.43051547 - samples/sec: 120.77 - lr: 0.018750 +2021-03-26 06:00:02,362 epoch 63 - iter 6/25 - loss 2.45722274 - samples/sec: 124.62 - lr: 0.018750 +2021-03-26 06:00:03,332 epoch 63 - iter 8/25 - loss 2.45625201 - samples/sec: 132.27 - lr: 0.018750 +2021-03-26 06:00:04,390 epoch 63 - iter 10/25 - loss 2.40530462 - samples/sec: 121.10 - lr: 0.018750 +2021-03-26 06:00:05,365 epoch 63 - iter 12/25 - loss 2.45525324 - samples/sec: 131.63 - lr: 0.018750 +2021-03-26 06:00:06,373 epoch 63 - iter 14/25 - loss 2.35274683 - samples/sec: 127.06 - lr: 0.018750 +2021-03-26 06:00:07,455 epoch 63 - iter 16/25 - loss 2.37795563 - samples/sec: 118.49 - lr: 0.018750 +2021-03-26 06:00:08,433 epoch 63 - iter 18/25 - loss 2.35921160 - samples/sec: 131.10 - lr: 0.018750 +2021-03-26 06:00:09,563 epoch 63 - iter 20/25 - loss 2.44812641 - samples/sec: 113.42 - lr: 0.018750 +2021-03-26 06:00:10,507 epoch 63 - iter 22/25 - loss 2.43963941 - samples/sec: 135.85 - lr: 0.018750 +2021-03-26 06:00:11,492 epoch 63 - iter 24/25 - loss 2.40271173 - samples/sec: 130.19 - lr: 0.018750 +2021-03-26 06:00:11,884 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:00:11,886 EPOCH 63 done: loss 2.4120 - lr 0.0187500 +2021-03-26 06:00:12,680 DEV : loss 6.751418113708496 - score 0.9046 +2021-03-26 06:00:12,710 BAD EPOCHS (no improvement): 3 +2021-03-26 06:00:12,711 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:00:15,095 epoch 64 - iter 2/25 - loss 2.59013653 - samples/sec: 53.73 - lr: 0.018750 +2021-03-26 06:00:16,160 epoch 64 - iter 4/25 - loss 2.39185470 - samples/sec: 120.47 - lr: 0.018750 +2021-03-26 06:00:17,149 epoch 64 - iter 6/25 - loss 2.35445166 - samples/sec: 129.73 - lr: 0.018750 +2021-03-26 06:00:18,163 epoch 64 - iter 8/25 - loss 2.37706771 - samples/sec: 126.32 - lr: 0.018750 +2021-03-26 06:00:19,163 epoch 64 - iter 10/25 - loss 2.35224319 - samples/sec: 128.21 - lr: 0.018750 +2021-03-26 06:00:20,164 epoch 64 - iter 12/25 - loss 2.30195101 - samples/sec: 128.08 - lr: 0.018750 +2021-03-26 06:00:21,419 epoch 64 - iter 14/25 - loss 2.37853982 - samples/sec: 102.04 - lr: 0.018750 +2021-03-26 06:00:22,547 epoch 64 - iter 16/25 - loss 2.38729794 - samples/sec: 113.68 - lr: 0.018750 +2021-03-26 06:00:23,553 epoch 64 - iter 18/25 - loss 2.38070179 - samples/sec: 127.44 - lr: 0.018750 +2021-03-26 06:00:24,569 epoch 64 - iter 20/25 - loss 2.38173290 - samples/sec: 126.15 - lr: 0.018750 +2021-03-26 06:00:25,591 epoch 64 - iter 22/25 - loss 2.42094920 - samples/sec: 125.41 - lr: 0.018750 +2021-03-26 06:00:26,649 epoch 64 - iter 24/25 - loss 2.47213012 - samples/sec: 121.22 - lr: 0.018750 +2021-03-26 06:00:27,065 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:00:27,065 EPOCH 64 done: loss 2.4368 - lr 0.0187500 +2021-03-26 06:00:27,871 DEV : loss 6.746072292327881 - score 0.9046 +2021-03-26 06:00:27,895 BAD EPOCHS (no improvement): 4 +2021-03-26 06:00:27,896 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:00:28,950 epoch 65 - iter 2/25 - loss 2.09257662 - samples/sec: 121.68 - lr: 0.009375 +2021-03-26 06:00:29,918 epoch 65 - iter 4/25 - loss 2.38884521 - samples/sec: 132.42 - lr: 0.009375 +2021-03-26 06:00:30,886 epoch 65 - iter 6/25 - loss 2.30170250 - samples/sec: 132.44 - lr: 0.009375 +2021-03-26 06:00:31,959 epoch 65 - iter 8/25 - loss 2.26360065 - samples/sec: 119.53 - lr: 0.009375 +2021-03-26 06:00:32,959 epoch 65 - iter 10/25 - loss 2.35580001 - samples/sec: 128.25 - lr: 0.009375 +2021-03-26 06:00:33,896 epoch 65 - iter 12/25 - loss 2.32859574 - samples/sec: 136.83 - lr: 0.009375 +2021-03-26 06:00:34,950 epoch 65 - iter 14/25 - loss 2.38266451 - samples/sec: 121.70 - lr: 0.009375 +2021-03-26 06:00:35,865 epoch 65 - iter 16/25 - loss 2.34344883 - samples/sec: 140.21 - lr: 0.009375 +2021-03-26 06:00:36,890 epoch 65 - iter 18/25 - loss 2.32179760 - samples/sec: 125.11 - lr: 0.009375 +2021-03-26 06:00:37,933 epoch 65 - iter 20/25 - loss 2.34824256 - samples/sec: 123.40 - lr: 0.009375 +2021-03-26 06:00:38,990 epoch 65 - iter 22/25 - loss 2.35746112 - samples/sec: 121.18 - lr: 0.009375 +2021-03-26 06:00:40,052 epoch 65 - iter 24/25 - loss 2.37810714 - samples/sec: 120.75 - lr: 0.009375 +2021-03-26 06:00:40,446 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:00:40,448 EPOCH 65 done: loss 2.3485 - lr 0.0093750 +2021-03-26 06:00:41,222 DEV : loss 6.7548370361328125 - score 0.9054 +2021-03-26 06:00:41,248 BAD EPOCHS (no improvement): 1 +2021-03-26 06:00:41,249 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:00:42,273 epoch 66 - iter 2/25 - loss 2.60710657 - samples/sec: 125.14 - lr: 0.009375 +2021-03-26 06:00:43,227 epoch 66 - iter 4/25 - loss 2.56015295 - samples/sec: 134.33 - lr: 0.009375 +2021-03-26 06:00:44,116 epoch 66 - iter 6/25 - loss 2.44887527 - samples/sec: 144.16 - lr: 0.009375 +2021-03-26 06:00:45,162 epoch 66 - iter 8/25 - loss 2.41221850 - samples/sec: 122.61 - lr: 0.009375 +2021-03-26 06:00:46,120 epoch 66 - iter 10/25 - loss 2.33587884 - samples/sec: 133.87 - lr: 0.009375 +2021-03-26 06:00:47,126 epoch 66 - iter 12/25 - loss 2.41186844 - samples/sec: 127.46 - lr: 0.009375 +2021-03-26 06:00:48,053 epoch 66 - iter 14/25 - loss 2.38344255 - samples/sec: 138.38 - lr: 0.009375 +2021-03-26 06:00:48,997 epoch 66 - iter 16/25 - loss 2.40268778 - samples/sec: 135.89 - lr: 0.009375 +2021-03-26 06:00:50,225 epoch 66 - iter 18/25 - loss 2.46428358 - samples/sec: 104.37 - lr: 0.009375 +2021-03-26 06:00:51,411 epoch 66 - iter 20/25 - loss 2.52800439 - samples/sec: 108.15 - lr: 0.009375 +2021-03-26 06:00:52,363 epoch 66 - iter 22/25 - loss 2.48158739 - samples/sec: 134.83 - lr: 0.009375 +2021-03-26 06:00:53,364 epoch 66 - iter 24/25 - loss 2.51746005 - samples/sec: 127.92 - lr: 0.009375 +2021-03-26 06:00:53,785 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:00:53,786 EPOCH 66 done: loss 2.5028 - lr 0.0093750 +2021-03-26 06:00:54,567 DEV : loss 6.752009868621826 - score 0.9058 +2021-03-26 06:00:54,590 BAD EPOCHS (no improvement): 2 +2021-03-26 06:00:54,591 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:00:55,558 epoch 67 - iter 2/25 - loss 2.90654492 - samples/sec: 132.50 - lr: 0.009375 +2021-03-26 06:00:56,624 epoch 67 - iter 4/25 - loss 2.68288243 - samples/sec: 120.25 - lr: 0.009375 +2021-03-26 06:00:57,733 epoch 67 - iter 6/25 - loss 2.66686583 - samples/sec: 115.71 - lr: 0.009375 +2021-03-26 06:00:58,699 epoch 67 - iter 8/25 - loss 2.66637003 - samples/sec: 132.89 - lr: 0.009375 +2021-03-26 06:00:59,626 epoch 67 - iter 10/25 - loss 2.71289964 - samples/sec: 138.25 - lr: 0.009375 +2021-03-26 06:01:00,729 epoch 67 - iter 12/25 - loss 2.61172573 - samples/sec: 116.19 - lr: 0.009375 +2021-03-26 06:01:01,698 epoch 67 - iter 14/25 - loss 2.55450325 - samples/sec: 132.57 - lr: 0.009375 +2021-03-26 06:01:02,709 epoch 67 - iter 16/25 - loss 2.55706222 - samples/sec: 126.75 - lr: 0.009375 +2021-03-26 06:01:03,692 epoch 67 - iter 18/25 - loss 2.54990624 - samples/sec: 130.46 - lr: 0.009375 +2021-03-26 06:01:04,784 epoch 67 - iter 20/25 - loss 2.58042824 - samples/sec: 117.36 - lr: 0.009375 +2021-03-26 06:01:05,847 epoch 67 - iter 22/25 - loss 2.54599817 - samples/sec: 120.56 - lr: 0.009375 +2021-03-26 06:01:06,920 epoch 67 - iter 24/25 - loss 2.53072828 - samples/sec: 119.51 - lr: 0.009375 +2021-03-26 06:01:07,321 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:01:07,321 EPOCH 67 done: loss 2.5156 - lr 0.0093750 +2021-03-26 06:01:08,148 DEV : loss 6.734310150146484 - score 0.9054 +2021-03-26 06:01:08,168 BAD EPOCHS (no improvement): 3 +2021-03-26 06:01:08,169 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:01:09,213 epoch 68 - iter 2/25 - loss 2.69127226 - samples/sec: 122.66 - lr: 0.009375 +2021-03-26 06:01:10,181 epoch 68 - iter 4/25 - loss 2.73949641 - samples/sec: 132.55 - lr: 0.009375 +2021-03-26 06:01:11,141 epoch 68 - iter 6/25 - loss 2.49159026 - samples/sec: 133.70 - lr: 0.009375 +2021-03-26 06:01:12,088 epoch 68 - iter 8/25 - loss 2.57020289 - samples/sec: 135.36 - lr: 0.009375 +2021-03-26 06:01:13,095 epoch 68 - iter 10/25 - loss 2.51954174 - samples/sec: 127.28 - lr: 0.009375 +2021-03-26 06:01:14,181 epoch 68 - iter 12/25 - loss 2.44195040 - samples/sec: 118.04 - lr: 0.009375 +2021-03-26 06:01:15,190 epoch 68 - iter 14/25 - loss 2.47290715 - samples/sec: 127.08 - lr: 0.009375 +2021-03-26 06:01:16,358 epoch 68 - iter 16/25 - loss 2.44247594 - samples/sec: 109.69 - lr: 0.009375 +2021-03-26 06:01:17,437 epoch 68 - iter 18/25 - loss 2.41137531 - samples/sec: 118.88 - lr: 0.009375 +2021-03-26 06:01:18,445 epoch 68 - iter 20/25 - loss 2.37544742 - samples/sec: 127.30 - lr: 0.009375 +2021-03-26 06:01:19,648 epoch 68 - iter 22/25 - loss 2.36865735 - samples/sec: 106.46 - lr: 0.009375 +2021-03-26 06:01:20,609 epoch 68 - iter 24/25 - loss 2.39470330 - samples/sec: 133.59 - lr: 0.009375 +2021-03-26 06:01:21,010 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:01:21,011 EPOCH 68 done: loss 2.3852 - lr 0.0093750 +2021-03-26 06:01:21,771 DEV : loss 6.732729911804199 - score 0.9054 +2021-03-26 06:01:21,795 BAD EPOCHS (no improvement): 4 +2021-03-26 06:01:21,795 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:01:22,731 epoch 69 - iter 2/25 - loss 2.14961201 - samples/sec: 137.02 - lr: 0.004687 +2021-03-26 06:01:23,622 epoch 69 - iter 4/25 - loss 2.15720448 - samples/sec: 144.16 - lr: 0.004687 +2021-03-26 06:01:24,580 epoch 69 - iter 6/25 - loss 2.13730657 - samples/sec: 133.78 - lr: 0.004687 +2021-03-26 06:01:25,690 epoch 69 - iter 8/25 - loss 2.14602265 - samples/sec: 115.47 - lr: 0.004687 +2021-03-26 06:01:26,672 epoch 69 - iter 10/25 - loss 2.24792657 - samples/sec: 130.74 - lr: 0.004687 +2021-03-26 06:01:27,758 epoch 69 - iter 12/25 - loss 2.22681612 - samples/sec: 118.12 - lr: 0.004687 +2021-03-26 06:01:28,719 epoch 69 - iter 14/25 - loss 2.17326404 - samples/sec: 133.59 - lr: 0.004687 +2021-03-26 06:01:29,765 epoch 69 - iter 16/25 - loss 2.18757197 - samples/sec: 122.65 - lr: 0.004687 +2021-03-26 06:01:30,818 epoch 69 - iter 18/25 - loss 2.28958360 - samples/sec: 121.86 - lr: 0.004687 +2021-03-26 06:01:31,843 epoch 69 - iter 20/25 - loss 2.28681493 - samples/sec: 125.19 - lr: 0.004687 +2021-03-26 06:01:32,902 epoch 69 - iter 22/25 - loss 2.31756993 - samples/sec: 121.07 - lr: 0.004687 +2021-03-26 06:01:33,937 epoch 69 - iter 24/25 - loss 2.33851089 - samples/sec: 123.91 - lr: 0.004687 +2021-03-26 06:01:34,336 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:01:34,338 EPOCH 69 done: loss 2.3385 - lr 0.0046875 +2021-03-26 06:01:35,144 DEV : loss 6.749580383300781 - score 0.9066 +2021-03-26 06:01:35,169 BAD EPOCHS (no improvement): 0 +2021-03-26 06:01:44,939 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:01:46,012 epoch 70 - iter 2/25 - loss 2.96315742 - samples/sec: 119.62 - lr: 0.004687 +2021-03-26 06:01:47,104 epoch 70 - iter 4/25 - loss 2.70809591 - samples/sec: 117.51 - lr: 0.004687 +2021-03-26 06:01:48,137 epoch 70 - iter 6/25 - loss 2.69718925 - samples/sec: 124.12 - lr: 0.004687 +2021-03-26 06:01:49,195 epoch 70 - iter 8/25 - loss 2.65873078 - samples/sec: 121.17 - lr: 0.004687 +2021-03-26 06:01:50,229 epoch 70 - iter 10/25 - loss 2.57560856 - samples/sec: 123.94 - lr: 0.004687 +2021-03-26 06:01:51,311 epoch 70 - iter 12/25 - loss 2.63722142 - samples/sec: 118.57 - lr: 0.004687 +2021-03-26 06:01:52,425 epoch 70 - iter 14/25 - loss 2.60139622 - samples/sec: 115.20 - lr: 0.004687 +2021-03-26 06:01:53,531 epoch 70 - iter 16/25 - loss 2.61185963 - samples/sec: 115.89 - lr: 0.004687 +2021-03-26 06:01:54,471 epoch 70 - iter 18/25 - loss 2.58912293 - samples/sec: 136.41 - lr: 0.004687 +2021-03-26 06:01:55,460 epoch 70 - iter 20/25 - loss 2.60415198 - samples/sec: 129.59 - lr: 0.004687 +2021-03-26 06:01:56,472 epoch 70 - iter 22/25 - loss 2.56834055 - samples/sec: 126.68 - lr: 0.004687 +2021-03-26 06:01:57,547 epoch 70 - iter 24/25 - loss 2.56006884 - samples/sec: 119.35 - lr: 0.004687 +2021-03-26 06:01:57,992 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:01:57,992 EPOCH 70 done: loss 2.5346 - lr 0.0046875 +2021-03-26 06:01:58,765 DEV : loss 6.751224517822266 - score 0.9062 +2021-03-26 06:01:58,785 BAD EPOCHS (no improvement): 1 +2021-03-26 06:01:58,786 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:01:59,808 epoch 71 - iter 2/25 - loss 2.36810112 - samples/sec: 125.51 - lr: 0.004687 +2021-03-26 06:02:00,804 epoch 71 - iter 4/25 - loss 2.16877392 - samples/sec: 128.64 - lr: 0.004687 +2021-03-26 06:02:01,893 epoch 71 - iter 6/25 - loss 2.23906277 - samples/sec: 117.72 - lr: 0.004687 +2021-03-26 06:02:03,393 epoch 71 - iter 8/25 - loss 2.35320215 - samples/sec: 85.41 - lr: 0.004687 +2021-03-26 06:02:04,442 epoch 71 - iter 10/25 - loss 2.17347422 - samples/sec: 122.18 - lr: 0.004687 +2021-03-26 06:02:05,460 epoch 71 - iter 12/25 - loss 2.27554323 - samples/sec: 125.95 - lr: 0.004687 +2021-03-26 06:02:06,360 epoch 71 - iter 14/25 - loss 2.27385560 - samples/sec: 142.69 - lr: 0.004687 +2021-03-26 06:02:07,479 epoch 71 - iter 16/25 - loss 2.30233501 - samples/sec: 114.52 - lr: 0.004687 +2021-03-26 06:02:08,511 epoch 71 - iter 18/25 - loss 2.34775718 - samples/sec: 124.20 - lr: 0.004687 +2021-03-26 06:02:09,502 epoch 71 - iter 20/25 - loss 2.37440296 - samples/sec: 129.38 - lr: 0.004687 +2021-03-26 06:02:10,527 epoch 71 - iter 22/25 - loss 2.34015569 - samples/sec: 125.15 - lr: 0.004687 +2021-03-26 06:02:11,548 epoch 71 - iter 24/25 - loss 2.31703675 - samples/sec: 125.47 - lr: 0.004687 +2021-03-26 06:02:11,985 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:02:11,987 EPOCH 71 done: loss 2.3079 - lr 0.0046875 +2021-03-26 06:02:12,798 DEV : loss 6.757469177246094 - score 0.907 +2021-03-26 06:02:12,838 BAD EPOCHS (no improvement): 0 +2021-03-26 06:02:22,994 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:02:24,090 epoch 72 - iter 2/25 - loss 1.88978660 - samples/sec: 117.15 - lr: 0.004687 +2021-03-26 06:02:25,107 epoch 72 - iter 4/25 - loss 2.08690861 - samples/sec: 126.25 - lr: 0.004687 +2021-03-26 06:02:26,251 epoch 72 - iter 6/25 - loss 2.20860885 - samples/sec: 112.06 - lr: 0.004687 +2021-03-26 06:02:27,414 epoch 72 - iter 8/25 - loss 2.27037697 - samples/sec: 110.19 - lr: 0.004687 +2021-03-26 06:02:28,636 epoch 72 - iter 10/25 - loss 2.46181704 - samples/sec: 104.91 - lr: 0.004687 +2021-03-26 06:02:29,795 epoch 72 - iter 12/25 - loss 2.40252721 - samples/sec: 110.61 - lr: 0.004687 +2021-03-26 06:02:30,794 epoch 72 - iter 14/25 - loss 2.46973358 - samples/sec: 128.27 - lr: 0.004687 +2021-03-26 06:02:31,747 epoch 72 - iter 16/25 - loss 2.51864427 - samples/sec: 134.47 - lr: 0.004687 +2021-03-26 06:02:32,766 epoch 72 - iter 18/25 - loss 2.51311964 - samples/sec: 125.83 - lr: 0.004687 +2021-03-26 06:02:33,781 epoch 72 - iter 20/25 - loss 2.48758618 - samples/sec: 126.28 - lr: 0.004687 +2021-03-26 06:02:34,687 epoch 72 - iter 22/25 - loss 2.43633575 - samples/sec: 141.56 - lr: 0.004687 +2021-03-26 06:02:35,710 epoch 72 - iter 24/25 - loss 2.41600278 - samples/sec: 125.20 - lr: 0.004687 +2021-03-26 06:02:36,128 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:02:36,129 EPOCH 72 done: loss 2.4187 - lr 0.0046875 +2021-03-26 06:02:36,909 DEV : loss 6.750691890716553 - score 0.9062 +2021-03-26 06:02:36,935 BAD EPOCHS (no improvement): 1 +2021-03-26 06:02:36,936 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:02:37,935 epoch 73 - iter 2/25 - loss 2.43253404 - samples/sec: 128.30 - lr: 0.004687 +2021-03-26 06:02:39,029 epoch 73 - iter 4/25 - loss 2.42423633 - samples/sec: 117.15 - lr: 0.004687 +2021-03-26 06:02:40,048 epoch 73 - iter 6/25 - loss 2.41635050 - samples/sec: 125.78 - lr: 0.004687 +2021-03-26 06:02:41,083 epoch 73 - iter 8/25 - loss 2.35884486 - samples/sec: 123.78 - lr: 0.004687 +2021-03-26 06:02:42,091 epoch 73 - iter 10/25 - loss 2.36682016 - samples/sec: 127.16 - lr: 0.004687 +2021-03-26 06:02:43,234 epoch 73 - iter 12/25 - loss 2.33669632 - samples/sec: 112.15 - lr: 0.004687 +2021-03-26 06:02:44,171 epoch 73 - iter 14/25 - loss 2.29896097 - samples/sec: 136.88 - lr: 0.004687 +2021-03-26 06:02:45,190 epoch 73 - iter 16/25 - loss 2.35040139 - samples/sec: 125.73 - lr: 0.004687 +2021-03-26 06:02:46,184 epoch 73 - iter 18/25 - loss 2.40921725 - samples/sec: 129.00 - lr: 0.004687 +2021-03-26 06:02:47,238 epoch 73 - iter 20/25 - loss 2.44946590 - samples/sec: 121.50 - lr: 0.004687 +2021-03-26 06:02:48,285 epoch 73 - iter 22/25 - loss 2.42929926 - samples/sec: 122.49 - lr: 0.004687 +2021-03-26 06:02:49,298 epoch 73 - iter 24/25 - loss 2.42566876 - samples/sec: 126.67 - lr: 0.004687 +2021-03-26 06:02:49,682 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:02:49,682 EPOCH 73 done: loss 2.4041 - lr 0.0046875 +2021-03-26 06:02:50,473 DEV : loss 6.749738693237305 - score 0.9058 +2021-03-26 06:02:50,498 BAD EPOCHS (no improvement): 2 +2021-03-26 06:02:50,499 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:02:51,521 epoch 74 - iter 2/25 - loss 2.52992201 - samples/sec: 125.48 - lr: 0.004687 +2021-03-26 06:02:52,495 epoch 74 - iter 4/25 - loss 2.31653494 - samples/sec: 131.58 - lr: 0.004687 +2021-03-26 06:02:53,454 epoch 74 - iter 6/25 - loss 2.40114681 - samples/sec: 133.68 - lr: 0.004687 +2021-03-26 06:02:54,628 epoch 74 - iter 8/25 - loss 2.56239355 - samples/sec: 109.21 - lr: 0.004687 +2021-03-26 06:02:55,790 epoch 74 - iter 10/25 - loss 2.53644116 - samples/sec: 110.26 - lr: 0.004687 +2021-03-26 06:02:56,800 epoch 74 - iter 12/25 - loss 2.53346517 - samples/sec: 126.97 - lr: 0.004687 +2021-03-26 06:02:57,694 epoch 74 - iter 14/25 - loss 2.49255146 - samples/sec: 143.46 - lr: 0.004687 +2021-03-26 06:02:58,680 epoch 74 - iter 16/25 - loss 2.54234672 - samples/sec: 129.96 - lr: 0.004687 +2021-03-26 06:02:59,637 epoch 74 - iter 18/25 - loss 2.44926276 - samples/sec: 133.94 - lr: 0.004687 +2021-03-26 06:03:00,644 epoch 74 - iter 20/25 - loss 2.44066932 - samples/sec: 127.36 - lr: 0.004687 +2021-03-26 06:03:01,663 epoch 74 - iter 22/25 - loss 2.48960504 - samples/sec: 125.95 - lr: 0.004687 +2021-03-26 06:03:02,738 epoch 74 - iter 24/25 - loss 2.49861135 - samples/sec: 119.29 - lr: 0.004687 +2021-03-26 06:03:03,138 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:03:03,139 EPOCH 74 done: loss 2.4818 - lr 0.0046875 +2021-03-26 06:03:03,915 DEV : loss 6.755931854248047 - score 0.9062 +2021-03-26 06:03:03,941 BAD EPOCHS (no improvement): 3 +2021-03-26 06:03:03,942 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:03:04,893 epoch 75 - iter 2/25 - loss 2.53918266 - samples/sec: 134.96 - lr: 0.004687 +2021-03-26 06:03:05,906 epoch 75 - iter 4/25 - loss 2.29154232 - samples/sec: 126.53 - lr: 0.004687 +2021-03-26 06:03:06,923 epoch 75 - iter 6/25 - loss 2.21423191 - samples/sec: 126.16 - lr: 0.004687 +2021-03-26 06:03:08,088 epoch 75 - iter 8/25 - loss 2.15976694 - samples/sec: 110.02 - lr: 0.004687 +2021-03-26 06:03:09,092 epoch 75 - iter 10/25 - loss 2.17650853 - samples/sec: 127.66 - lr: 0.004687 +2021-03-26 06:03:10,091 epoch 75 - iter 12/25 - loss 2.24466210 - samples/sec: 128.36 - lr: 0.004687 +2021-03-26 06:03:11,241 epoch 75 - iter 14/25 - loss 2.26832461 - samples/sec: 111.53 - lr: 0.004687 +2021-03-26 06:03:12,515 epoch 75 - iter 16/25 - loss 2.31134140 - samples/sec: 100.63 - lr: 0.004687 +2021-03-26 06:03:13,511 epoch 75 - iter 18/25 - loss 2.32876862 - samples/sec: 128.86 - lr: 0.004687 +2021-03-26 06:03:14,546 epoch 75 - iter 20/25 - loss 2.32084221 - samples/sec: 123.96 - lr: 0.004687 +2021-03-26 06:03:15,564 epoch 75 - iter 22/25 - loss 2.30626246 - samples/sec: 126.04 - lr: 0.004687 +2021-03-26 06:03:16,665 epoch 75 - iter 24/25 - loss 2.32581308 - samples/sec: 116.42 - lr: 0.004687 +2021-03-26 06:03:17,029 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:03:17,030 EPOCH 75 done: loss 2.3309 - lr 0.0046875 +2021-03-26 06:03:17,793 DEV : loss 6.7594757080078125 - score 0.9058 +2021-03-26 06:03:17,819 BAD EPOCHS (no improvement): 4 +2021-03-26 06:03:17,820 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:03:18,848 epoch 76 - iter 2/25 - loss 2.15280581 - samples/sec: 124.71 - lr: 0.002344 +2021-03-26 06:03:19,792 epoch 76 - iter 4/25 - loss 2.35897440 - samples/sec: 135.76 - lr: 0.002344 +2021-03-26 06:03:20,733 epoch 76 - iter 6/25 - loss 2.32432910 - samples/sec: 136.21 - lr: 0.002344 +2021-03-26 06:03:21,692 epoch 76 - iter 8/25 - loss 2.36514843 - samples/sec: 133.77 - lr: 0.002344 +2021-03-26 06:03:22,782 epoch 76 - iter 10/25 - loss 2.41345239 - samples/sec: 117.65 - lr: 0.002344 +2021-03-26 06:03:23,884 epoch 76 - iter 12/25 - loss 2.36118354 - samples/sec: 116.41 - lr: 0.002344 +2021-03-26 06:03:24,894 epoch 76 - iter 14/25 - loss 2.37700932 - samples/sec: 127.10 - lr: 0.002344 +2021-03-26 06:03:25,917 epoch 76 - iter 16/25 - loss 2.37870736 - samples/sec: 125.32 - lr: 0.002344 +2021-03-26 06:03:26,964 epoch 76 - iter 18/25 - loss 2.34423612 - samples/sec: 122.33 - lr: 0.002344 +2021-03-26 06:03:27,945 epoch 76 - iter 20/25 - loss 2.35013263 - samples/sec: 130.63 - lr: 0.002344 +2021-03-26 06:03:29,025 epoch 76 - iter 22/25 - loss 2.34488303 - samples/sec: 118.68 - lr: 0.002344 +2021-03-26 06:03:30,051 epoch 76 - iter 24/25 - loss 2.38089673 - samples/sec: 124.92 - lr: 0.002344 +2021-03-26 06:03:30,461 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:03:30,462 EPOCH 76 done: loss 2.4297 - lr 0.0023437 +2021-03-26 06:03:31,220 DEV : loss 6.752521514892578 - score 0.907 +2021-03-26 06:03:31,246 BAD EPOCHS (no improvement): 0 +2021-03-26 06:03:40,746 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:03:41,763 epoch 77 - iter 2/25 - loss 2.50017357 - samples/sec: 126.27 - lr: 0.002344 +2021-03-26 06:03:42,741 epoch 77 - iter 4/25 - loss 2.22791696 - samples/sec: 130.96 - lr: 0.002344 +2021-03-26 06:03:43,758 epoch 77 - iter 6/25 - loss 2.39182313 - samples/sec: 126.02 - lr: 0.002344 +2021-03-26 06:03:44,691 epoch 77 - iter 8/25 - loss 2.38184157 - samples/sec: 137.71 - lr: 0.002344 +2021-03-26 06:03:45,694 epoch 77 - iter 10/25 - loss 2.44655206 - samples/sec: 127.74 - lr: 0.002344 +2021-03-26 06:03:46,789 epoch 77 - iter 12/25 - loss 2.40551968 - samples/sec: 117.06 - lr: 0.002344 +2021-03-26 06:03:47,802 epoch 77 - iter 14/25 - loss 2.38007069 - samples/sec: 126.62 - lr: 0.002344 +2021-03-26 06:03:48,909 epoch 77 - iter 16/25 - loss 2.43085665 - samples/sec: 115.78 - lr: 0.002344 +2021-03-26 06:03:49,932 epoch 77 - iter 18/25 - loss 2.45488136 - samples/sec: 125.38 - lr: 0.002344 +2021-03-26 06:03:50,914 epoch 77 - iter 20/25 - loss 2.43090633 - samples/sec: 130.50 - lr: 0.002344 +2021-03-26 06:03:51,955 epoch 77 - iter 22/25 - loss 2.38712107 - samples/sec: 123.25 - lr: 0.002344 +2021-03-26 06:03:52,997 epoch 77 - iter 24/25 - loss 2.41249888 - samples/sec: 123.06 - lr: 0.002344 +2021-03-26 06:03:53,411 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:03:53,412 EPOCH 77 done: loss 2.4017 - lr 0.0023437 +2021-03-26 06:03:54,214 DEV : loss 6.756526470184326 - score 0.9062 +2021-03-26 06:03:54,240 BAD EPOCHS (no improvement): 1 +2021-03-26 06:03:54,241 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:03:55,315 epoch 78 - iter 2/25 - loss 2.16830981 - samples/sec: 119.40 - lr: 0.002344 +2021-03-26 06:03:56,398 epoch 78 - iter 4/25 - loss 2.32413882 - samples/sec: 118.28 - lr: 0.002344 +2021-03-26 06:03:57,365 epoch 78 - iter 6/25 - loss 2.29382400 - samples/sec: 132.76 - lr: 0.002344 +2021-03-26 06:03:58,310 epoch 78 - iter 8/25 - loss 2.32180464 - samples/sec: 135.65 - lr: 0.002344 +2021-03-26 06:03:59,297 epoch 78 - iter 10/25 - loss 2.41712666 - samples/sec: 129.89 - lr: 0.002344 +2021-03-26 06:04:00,251 epoch 78 - iter 12/25 - loss 2.34759426 - samples/sec: 134.32 - lr: 0.002344 +2021-03-26 06:04:01,282 epoch 78 - iter 14/25 - loss 2.37938215 - samples/sec: 124.38 - lr: 0.002344 +2021-03-26 06:04:02,324 epoch 78 - iter 16/25 - loss 2.36263736 - samples/sec: 123.04 - lr: 0.002344 +2021-03-26 06:04:03,311 epoch 78 - iter 18/25 - loss 2.35715266 - samples/sec: 129.94 - lr: 0.002344 +2021-03-26 06:04:04,388 epoch 78 - iter 20/25 - loss 2.36736240 - samples/sec: 118.97 - lr: 0.002344 +2021-03-26 06:04:05,463 epoch 78 - iter 22/25 - loss 2.38804259 - samples/sec: 119.23 - lr: 0.002344 +2021-03-26 06:04:06,517 epoch 78 - iter 24/25 - loss 2.38424336 - samples/sec: 121.65 - lr: 0.002344 +2021-03-26 06:04:07,017 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:04:07,018 EPOCH 78 done: loss 2.3911 - lr 0.0023437 +2021-03-26 06:04:07,791 DEV : loss 6.753111839294434 - score 0.9066 +2021-03-26 06:04:07,817 BAD EPOCHS (no improvement): 2 +2021-03-26 06:04:07,818 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:04:08,882 epoch 79 - iter 2/25 - loss 2.51307201 - samples/sec: 120.52 - lr: 0.002344 +2021-03-26 06:04:09,937 epoch 79 - iter 4/25 - loss 2.67237234 - samples/sec: 121.44 - lr: 0.002344 +2021-03-26 06:04:10,994 epoch 79 - iter 6/25 - loss 2.48488834 - samples/sec: 121.36 - lr: 0.002344 +2021-03-26 06:04:12,163 epoch 79 - iter 8/25 - loss 2.51524417 - samples/sec: 109.72 - lr: 0.002344 +2021-03-26 06:04:13,153 epoch 79 - iter 10/25 - loss 2.38670518 - samples/sec: 129.48 - lr: 0.002344 +2021-03-26 06:04:14,192 epoch 79 - iter 12/25 - loss 2.34001416 - samples/sec: 123.40 - lr: 0.002344 +2021-03-26 06:04:15,247 epoch 79 - iter 14/25 - loss 2.27449070 - samples/sec: 121.54 - lr: 0.002344 +2021-03-26 06:04:16,275 epoch 79 - iter 16/25 - loss 2.31747055 - samples/sec: 124.71 - lr: 0.002344 +2021-03-26 06:04:17,248 epoch 79 - iter 18/25 - loss 2.27182851 - samples/sec: 131.83 - lr: 0.002344 +2021-03-26 06:04:18,200 epoch 79 - iter 20/25 - loss 2.28024085 - samples/sec: 134.52 - lr: 0.002344 +2021-03-26 06:04:19,199 epoch 79 - iter 22/25 - loss 2.29187663 - samples/sec: 128.35 - lr: 0.002344 +2021-03-26 06:04:20,251 epoch 79 - iter 24/25 - loss 2.32610110 - samples/sec: 121.97 - lr: 0.002344 +2021-03-26 06:04:20,656 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:04:20,658 EPOCH 79 done: loss 2.3529 - lr 0.0023437 +2021-03-26 06:04:21,461 DEV : loss 6.750301361083984 - score 0.9058 +2021-03-26 06:04:21,479 BAD EPOCHS (no improvement): 3 +2021-03-26 06:04:21,480 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:04:22,530 epoch 80 - iter 2/25 - loss 2.18168497 - samples/sec: 122.11 - lr: 0.002344 +2021-03-26 06:04:23,531 epoch 80 - iter 4/25 - loss 2.26245016 - samples/sec: 128.10 - lr: 0.002344 +2021-03-26 06:04:24,522 epoch 80 - iter 6/25 - loss 2.26572899 - samples/sec: 129.44 - lr: 0.002344 +2021-03-26 06:04:25,496 epoch 80 - iter 8/25 - loss 2.22850430 - samples/sec: 131.55 - lr: 0.002344 +2021-03-26 06:04:26,602 epoch 80 - iter 10/25 - loss 2.22804742 - samples/sec: 115.94 - lr: 0.002344 +2021-03-26 06:04:27,737 epoch 80 - iter 12/25 - loss 2.25041121 - samples/sec: 112.91 - lr: 0.002344 +2021-03-26 06:04:28,740 epoch 80 - iter 14/25 - loss 2.35656748 - samples/sec: 127.88 - lr: 0.002344 +2021-03-26 06:04:29,859 epoch 80 - iter 16/25 - loss 2.41935366 - samples/sec: 114.61 - lr: 0.002344 +2021-03-26 06:04:30,861 epoch 80 - iter 18/25 - loss 2.41030420 - samples/sec: 127.86 - lr: 0.002344 +2021-03-26 06:04:31,862 epoch 80 - iter 20/25 - loss 2.41881890 - samples/sec: 128.21 - lr: 0.002344 +2021-03-26 06:04:32,852 epoch 80 - iter 22/25 - loss 2.43735432 - samples/sec: 129.48 - lr: 0.002344 +2021-03-26 06:04:33,803 epoch 80 - iter 24/25 - loss 2.37674877 - samples/sec: 134.96 - lr: 0.002344 +2021-03-26 06:04:34,286 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:04:34,287 EPOCH 80 done: loss 2.4121 - lr 0.0023437 +2021-03-26 06:04:35,059 DEV : loss 6.750476837158203 - score 0.9062 +2021-03-26 06:04:35,076 BAD EPOCHS (no improvement): 4 +2021-03-26 06:04:35,077 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:04:36,051 epoch 81 - iter 2/25 - loss 2.33307743 - samples/sec: 131.62 - lr: 0.001172 +2021-03-26 06:04:37,034 epoch 81 - iter 4/25 - loss 2.36750239 - samples/sec: 130.42 - lr: 0.001172 +2021-03-26 06:04:38,019 epoch 81 - iter 6/25 - loss 2.15509367 - samples/sec: 130.15 - lr: 0.001172 +2021-03-26 06:04:39,004 epoch 81 - iter 8/25 - loss 2.14618689 - samples/sec: 130.31 - lr: 0.001172 +2021-03-26 06:04:39,985 epoch 81 - iter 10/25 - loss 2.29240470 - samples/sec: 130.81 - lr: 0.001172 +2021-03-26 06:04:41,226 epoch 81 - iter 12/25 - loss 2.39395547 - samples/sec: 103.67 - lr: 0.001172 +2021-03-26 06:04:42,262 epoch 81 - iter 14/25 - loss 2.49634773 - samples/sec: 123.76 - lr: 0.001172 +2021-03-26 06:04:43,200 epoch 81 - iter 16/25 - loss 2.48210439 - samples/sec: 136.78 - lr: 0.001172 +2021-03-26 06:04:44,216 epoch 81 - iter 18/25 - loss 2.53619586 - samples/sec: 126.17 - lr: 0.001172 +2021-03-26 06:04:45,281 epoch 81 - iter 20/25 - loss 2.53158684 - samples/sec: 120.34 - lr: 0.001172 +2021-03-26 06:04:46,511 epoch 81 - iter 22/25 - loss 2.54447044 - samples/sec: 104.27 - lr: 0.001172 +2021-03-26 06:04:47,627 epoch 81 - iter 24/25 - loss 2.54063163 - samples/sec: 115.01 - lr: 0.001172 +2021-03-26 06:04:48,035 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:04:48,036 EPOCH 81 done: loss 2.5130 - lr 0.0011719 +2021-03-26 06:04:48,863 DEV : loss 6.748684883117676 - score 0.9062 +2021-03-26 06:04:48,882 BAD EPOCHS (no improvement): 1 +2021-03-26 06:04:48,883 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:04:50,061 epoch 82 - iter 2/25 - loss 2.38473833 - samples/sec: 108.86 - lr: 0.001172 +2021-03-26 06:04:51,204 epoch 82 - iter 4/25 - loss 2.48814780 - samples/sec: 112.13 - lr: 0.001172 +2021-03-26 06:04:52,241 epoch 82 - iter 6/25 - loss 2.38537379 - samples/sec: 123.63 - lr: 0.001172 +2021-03-26 06:04:53,341 epoch 82 - iter 8/25 - loss 2.33337769 - samples/sec: 116.55 - lr: 0.001172 +2021-03-26 06:04:54,469 epoch 82 - iter 10/25 - loss 2.37745783 - samples/sec: 113.66 - lr: 0.001172 +2021-03-26 06:04:55,486 epoch 82 - iter 12/25 - loss 2.35676177 - samples/sec: 126.05 - lr: 0.001172 +2021-03-26 06:04:56,435 epoch 82 - iter 14/25 - loss 2.34542345 - samples/sec: 135.12 - lr: 0.001172 +2021-03-26 06:04:57,400 epoch 82 - iter 16/25 - loss 2.44239170 - samples/sec: 132.83 - lr: 0.001172 +2021-03-26 06:04:58,376 epoch 82 - iter 18/25 - loss 2.41721334 - samples/sec: 131.38 - lr: 0.001172 +2021-03-26 06:04:59,343 epoch 82 - iter 20/25 - loss 2.38730320 - samples/sec: 132.61 - lr: 0.001172 +2021-03-26 06:05:00,370 epoch 82 - iter 22/25 - loss 2.36518301 - samples/sec: 124.76 - lr: 0.001172 +2021-03-26 06:05:01,399 epoch 82 - iter 24/25 - loss 2.36671859 - samples/sec: 124.62 - lr: 0.001172 +2021-03-26 06:05:01,784 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:05:01,785 EPOCH 82 done: loss 2.3515 - lr 0.0011719 +2021-03-26 06:05:02,558 DEV : loss 6.749919891357422 - score 0.9054 +2021-03-26 06:05:02,576 BAD EPOCHS (no improvement): 2 +2021-03-26 06:05:02,577 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:05:03,579 epoch 83 - iter 2/25 - loss 2.50160813 - samples/sec: 127.96 - lr: 0.001172 +2021-03-26 06:05:04,592 epoch 83 - iter 4/25 - loss 2.49640393 - samples/sec: 126.56 - lr: 0.001172 +2021-03-26 06:05:05,683 epoch 83 - iter 6/25 - loss 2.44112500 - samples/sec: 117.45 - lr: 0.001172 +2021-03-26 06:05:06,603 epoch 83 - iter 8/25 - loss 2.29882008 - samples/sec: 139.55 - lr: 0.001172 +2021-03-26 06:05:07,521 epoch 83 - iter 10/25 - loss 2.32238986 - samples/sec: 139.72 - lr: 0.001172 +2021-03-26 06:05:08,533 epoch 83 - iter 12/25 - loss 2.32237995 - samples/sec: 126.68 - lr: 0.001172 +2021-03-26 06:05:09,491 epoch 83 - iter 14/25 - loss 2.42954922 - samples/sec: 133.91 - lr: 0.001172 +2021-03-26 06:05:10,504 epoch 83 - iter 16/25 - loss 2.42919302 - samples/sec: 126.52 - lr: 0.001172 +2021-03-26 06:05:11,528 epoch 83 - iter 18/25 - loss 2.42619081 - samples/sec: 125.13 - lr: 0.001172 +2021-03-26 06:05:12,633 epoch 83 - iter 20/25 - loss 2.42287891 - samples/sec: 115.97 - lr: 0.001172 +2021-03-26 06:05:13,594 epoch 83 - iter 22/25 - loss 2.42632379 - samples/sec: 133.48 - lr: 0.001172 +2021-03-26 06:05:14,561 epoch 83 - iter 24/25 - loss 2.42381104 - samples/sec: 132.57 - lr: 0.001172 +2021-03-26 06:05:15,115 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:05:15,117 EPOCH 83 done: loss 2.4270 - lr 0.0011719 +2021-03-26 06:05:15,905 DEV : loss 6.750192165374756 - score 0.9058 +2021-03-26 06:05:15,930 BAD EPOCHS (no improvement): 3 +2021-03-26 06:05:15,931 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:05:17,094 epoch 84 - iter 2/25 - loss 2.93648970 - samples/sec: 110.24 - lr: 0.001172 +2021-03-26 06:05:18,206 epoch 84 - iter 4/25 - loss 2.73000449 - samples/sec: 115.40 - lr: 0.001172 +2021-03-26 06:05:19,245 epoch 84 - iter 6/25 - loss 2.49694339 - samples/sec: 123.50 - lr: 0.001172 +2021-03-26 06:05:20,367 epoch 84 - iter 8/25 - loss 2.38217634 - samples/sec: 114.35 - lr: 0.001172 +2021-03-26 06:05:21,393 epoch 84 - iter 10/25 - loss 2.38356829 - samples/sec: 124.90 - lr: 0.001172 +2021-03-26 06:05:22,454 epoch 84 - iter 12/25 - loss 2.39194481 - samples/sec: 120.76 - lr: 0.001172 +2021-03-26 06:05:23,530 epoch 84 - iter 14/25 - loss 2.37829908 - samples/sec: 119.18 - lr: 0.001172 +2021-03-26 06:05:24,408 epoch 84 - iter 16/25 - loss 2.33745650 - samples/sec: 146.02 - lr: 0.001172 +2021-03-26 06:05:25,357 epoch 84 - iter 18/25 - loss 2.37947904 - samples/sec: 135.05 - lr: 0.001172 +2021-03-26 06:05:26,338 epoch 84 - iter 20/25 - loss 2.38875579 - samples/sec: 130.75 - lr: 0.001172 +2021-03-26 06:05:27,409 epoch 84 - iter 22/25 - loss 2.39732352 - samples/sec: 119.67 - lr: 0.001172 +2021-03-26 06:05:28,397 epoch 84 - iter 24/25 - loss 2.43036128 - samples/sec: 129.78 - lr: 0.001172 +2021-03-26 06:05:28,943 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:05:28,945 EPOCH 84 done: loss 2.4095 - lr 0.0011719 +2021-03-26 06:05:29,765 DEV : loss 6.74921989440918 - score 0.9054 +2021-03-26 06:05:29,796 BAD EPOCHS (no improvement): 4 +2021-03-26 06:05:29,796 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:05:30,862 epoch 85 - iter 2/25 - loss 2.46741247 - samples/sec: 120.43 - lr: 0.000586 +2021-03-26 06:05:31,847 epoch 85 - iter 4/25 - loss 2.50471973 - samples/sec: 130.04 - lr: 0.000586 +2021-03-26 06:05:32,920 epoch 85 - iter 6/25 - loss 2.49037719 - samples/sec: 119.44 - lr: 0.000586 +2021-03-26 06:05:34,014 epoch 85 - iter 8/25 - loss 2.46623042 - samples/sec: 117.19 - lr: 0.000586 +2021-03-26 06:05:35,145 epoch 85 - iter 10/25 - loss 2.53815739 - samples/sec: 113.39 - lr: 0.000586 +2021-03-26 06:05:36,090 epoch 85 - iter 12/25 - loss 2.43846828 - samples/sec: 135.55 - lr: 0.000586 +2021-03-26 06:05:37,054 epoch 85 - iter 14/25 - loss 2.50122946 - samples/sec: 133.03 - lr: 0.000586 +2021-03-26 06:05:38,042 epoch 85 - iter 16/25 - loss 2.51318491 - samples/sec: 129.71 - lr: 0.000586 +2021-03-26 06:05:39,062 epoch 85 - iter 18/25 - loss 2.51794383 - samples/sec: 125.81 - lr: 0.000586 +2021-03-26 06:05:40,066 epoch 85 - iter 20/25 - loss 2.51388612 - samples/sec: 127.59 - lr: 0.000586 +2021-03-26 06:05:41,068 epoch 85 - iter 22/25 - loss 2.50162070 - samples/sec: 127.99 - lr: 0.000586 +2021-03-26 06:05:42,209 epoch 85 - iter 24/25 - loss 2.50571299 - samples/sec: 112.36 - lr: 0.000586 +2021-03-26 06:05:42,646 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:05:42,647 EPOCH 85 done: loss 2.4964 - lr 0.0005859 +2021-03-26 06:05:43,416 DEV : loss 6.748666763305664 - score 0.9058 +2021-03-26 06:05:43,442 BAD EPOCHS (no improvement): 1 +2021-03-26 06:05:43,443 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:05:44,603 epoch 86 - iter 2/25 - loss 2.20638573 - samples/sec: 110.50 - lr: 0.000586 +2021-03-26 06:05:45,676 epoch 86 - iter 4/25 - loss 2.36967194 - samples/sec: 119.49 - lr: 0.000586 +2021-03-26 06:05:46,743 epoch 86 - iter 6/25 - loss 2.34573106 - samples/sec: 120.13 - lr: 0.000586 +2021-03-26 06:05:47,757 epoch 86 - iter 8/25 - loss 2.33546901 - samples/sec: 126.46 - lr: 0.000586 +2021-03-26 06:05:48,822 epoch 86 - iter 10/25 - loss 2.33539317 - samples/sec: 120.32 - lr: 0.000586 +2021-03-26 06:05:49,966 epoch 86 - iter 12/25 - loss 2.31430407 - samples/sec: 112.07 - lr: 0.000586 +2021-03-26 06:05:51,001 epoch 86 - iter 14/25 - loss 2.36735562 - samples/sec: 123.82 - lr: 0.000586 +2021-03-26 06:05:51,962 epoch 86 - iter 16/25 - loss 2.38468708 - samples/sec: 133.36 - lr: 0.000586 +2021-03-26 06:05:52,917 epoch 86 - iter 18/25 - loss 2.36988348 - samples/sec: 134.25 - lr: 0.000586 +2021-03-26 06:05:53,927 epoch 86 - iter 20/25 - loss 2.41210339 - samples/sec: 126.95 - lr: 0.000586 +2021-03-26 06:05:54,892 epoch 86 - iter 22/25 - loss 2.42297436 - samples/sec: 132.83 - lr: 0.000586 +2021-03-26 06:05:55,905 epoch 86 - iter 24/25 - loss 2.45856727 - samples/sec: 126.59 - lr: 0.000586 +2021-03-26 06:05:56,363 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:05:56,363 EPOCH 86 done: loss 2.4923 - lr 0.0005859 +2021-03-26 06:05:57,179 DEV : loss 6.748099327087402 - score 0.9054 +2021-03-26 06:05:57,214 BAD EPOCHS (no improvement): 2 +2021-03-26 06:05:57,214 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:05:58,279 epoch 87 - iter 2/25 - loss 2.76640713 - samples/sec: 120.45 - lr: 0.000586 +2021-03-26 06:05:59,409 epoch 87 - iter 4/25 - loss 2.25503093 - samples/sec: 113.45 - lr: 0.000586 +2021-03-26 06:06:00,409 epoch 87 - iter 6/25 - loss 2.23643498 - samples/sec: 128.20 - lr: 0.000586 +2021-03-26 06:06:01,516 epoch 87 - iter 8/25 - loss 2.42684510 - samples/sec: 115.74 - lr: 0.000586 +2021-03-26 06:06:02,479 epoch 87 - iter 10/25 - loss 2.44421749 - samples/sec: 133.10 - lr: 0.000586 +2021-03-26 06:06:03,457 epoch 87 - iter 12/25 - loss 2.56141780 - samples/sec: 131.10 - lr: 0.000586 +2021-03-26 06:06:04,589 epoch 87 - iter 14/25 - loss 2.52008237 - samples/sec: 113.18 - lr: 0.000586 +2021-03-26 06:06:05,551 epoch 87 - iter 16/25 - loss 2.52018423 - samples/sec: 133.33 - lr: 0.000586 +2021-03-26 06:06:06,588 epoch 87 - iter 18/25 - loss 2.54743120 - samples/sec: 123.54 - lr: 0.000586 +2021-03-26 06:06:07,548 epoch 87 - iter 20/25 - loss 2.50725095 - samples/sec: 133.75 - lr: 0.000586 +2021-03-26 06:06:08,610 epoch 87 - iter 22/25 - loss 2.54458821 - samples/sec: 120.70 - lr: 0.000586 +2021-03-26 06:06:09,664 epoch 87 - iter 24/25 - loss 2.53648801 - samples/sec: 121.68 - lr: 0.000586 +2021-03-26 06:06:10,095 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:06:10,096 EPOCH 87 done: loss 2.5160 - lr 0.0005859 +2021-03-26 06:06:10,915 DEV : loss 6.7480010986328125 - score 0.9054 +2021-03-26 06:06:10,942 BAD EPOCHS (no improvement): 3 +2021-03-26 06:06:10,942 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:06:11,965 epoch 88 - iter 2/25 - loss 2.35491943 - samples/sec: 125.32 - lr: 0.000586 +2021-03-26 06:06:13,099 epoch 88 - iter 4/25 - loss 2.46524775 - samples/sec: 112.98 - lr: 0.000586 +2021-03-26 06:06:14,226 epoch 88 - iter 6/25 - loss 2.32718579 - samples/sec: 113.69 - lr: 0.000586 +2021-03-26 06:06:15,238 epoch 88 - iter 8/25 - loss 2.20188485 - samples/sec: 126.78 - lr: 0.000586 +2021-03-26 06:06:16,358 epoch 88 - iter 10/25 - loss 2.20019561 - samples/sec: 114.55 - lr: 0.000586 +2021-03-26 06:06:17,428 epoch 88 - iter 12/25 - loss 2.24452155 - samples/sec: 119.80 - lr: 0.000586 +2021-03-26 06:06:18,464 epoch 88 - iter 14/25 - loss 2.23918286 - samples/sec: 123.74 - lr: 0.000586 +2021-03-26 06:06:19,371 epoch 88 - iter 16/25 - loss 2.28264054 - samples/sec: 141.38 - lr: 0.000586 +2021-03-26 06:06:20,329 epoch 88 - iter 18/25 - loss 2.28697127 - samples/sec: 133.84 - lr: 0.000586 +2021-03-26 06:06:21,244 epoch 88 - iter 20/25 - loss 2.33569470 - samples/sec: 140.01 - lr: 0.000586 +2021-03-26 06:06:22,231 epoch 88 - iter 22/25 - loss 2.35366069 - samples/sec: 129.84 - lr: 0.000586 +2021-03-26 06:06:23,213 epoch 88 - iter 24/25 - loss 2.34399285 - samples/sec: 130.59 - lr: 0.000586 +2021-03-26 06:06:23,692 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:06:23,694 EPOCH 88 done: loss 2.3643 - lr 0.0005859 +2021-03-26 06:06:24,466 DEV : loss 6.7463812828063965 - score 0.9058 +2021-03-26 06:06:24,488 BAD EPOCHS (no improvement): 4 +2021-03-26 06:06:24,489 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:06:25,540 epoch 89 - iter 2/25 - loss 2.73587209 - samples/sec: 121.92 - lr: 0.000293 +2021-03-26 06:06:26,501 epoch 89 - iter 4/25 - loss 2.68548825 - samples/sec: 133.47 - lr: 0.000293 +2021-03-26 06:06:27,498 epoch 89 - iter 6/25 - loss 2.69872004 - samples/sec: 128.66 - lr: 0.000293 +2021-03-26 06:06:28,456 epoch 89 - iter 8/25 - loss 2.63567401 - samples/sec: 133.91 - lr: 0.000293 +2021-03-26 06:06:29,545 epoch 89 - iter 10/25 - loss 2.61476969 - samples/sec: 117.69 - lr: 0.000293 +2021-03-26 06:06:30,674 epoch 89 - iter 12/25 - loss 2.61334603 - samples/sec: 113.54 - lr: 0.000293 +2021-03-26 06:06:31,763 epoch 89 - iter 14/25 - loss 2.59073320 - samples/sec: 117.78 - lr: 0.000293 +2021-03-26 06:06:32,784 epoch 89 - iter 16/25 - loss 2.52647568 - samples/sec: 125.50 - lr: 0.000293 +2021-03-26 06:06:33,805 epoch 89 - iter 18/25 - loss 2.51834848 - samples/sec: 125.62 - lr: 0.000293 +2021-03-26 06:06:34,896 epoch 89 - iter 20/25 - loss 2.46717817 - samples/sec: 117.44 - lr: 0.000293 +2021-03-26 06:06:35,884 epoch 89 - iter 22/25 - loss 2.45679012 - samples/sec: 129.77 - lr: 0.000293 +2021-03-26 06:06:36,865 epoch 89 - iter 24/25 - loss 2.39095766 - samples/sec: 130.53 - lr: 0.000293 +2021-03-26 06:06:37,303 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:06:37,304 EPOCH 89 done: loss 2.3895 - lr 0.0002930 +2021-03-26 06:06:38,066 DEV : loss 6.746824264526367 - score 0.9058 +2021-03-26 06:06:38,087 BAD EPOCHS (no improvement): 1 +2021-03-26 06:06:38,088 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:06:39,103 epoch 90 - iter 2/25 - loss 2.29537117 - samples/sec: 126.45 - lr: 0.000293 +2021-03-26 06:06:40,283 epoch 90 - iter 4/25 - loss 2.12782717 - samples/sec: 108.60 - lr: 0.000293 +2021-03-26 06:06:41,405 epoch 90 - iter 6/25 - loss 2.28929679 - samples/sec: 114.16 - lr: 0.000293 +2021-03-26 06:06:42,392 epoch 90 - iter 8/25 - loss 2.35938233 - samples/sec: 129.89 - lr: 0.000293 +2021-03-26 06:06:43,303 epoch 90 - iter 10/25 - loss 2.32694597 - samples/sec: 140.65 - lr: 0.000293 +2021-03-26 06:06:44,302 epoch 90 - iter 12/25 - loss 2.37408519 - samples/sec: 128.31 - lr: 0.000293 +2021-03-26 06:06:45,326 epoch 90 - iter 14/25 - loss 2.40497916 - samples/sec: 125.28 - lr: 0.000293 +2021-03-26 06:06:46,292 epoch 90 - iter 16/25 - loss 2.43432885 - samples/sec: 132.67 - lr: 0.000293 +2021-03-26 06:06:47,297 epoch 90 - iter 18/25 - loss 2.41117813 - samples/sec: 127.55 - lr: 0.000293 +2021-03-26 06:06:48,362 epoch 90 - iter 20/25 - loss 2.34449456 - samples/sec: 120.35 - lr: 0.000293 +2021-03-26 06:06:49,500 epoch 90 - iter 22/25 - loss 2.32467386 - samples/sec: 112.72 - lr: 0.000293 +2021-03-26 06:06:50,504 epoch 90 - iter 24/25 - loss 2.35763361 - samples/sec: 127.72 - lr: 0.000293 +2021-03-26 06:06:51,075 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:06:51,075 EPOCH 90 done: loss 2.3738 - lr 0.0002930 +2021-03-26 06:06:51,866 DEV : loss 6.747221946716309 - score 0.9058 +2021-03-26 06:06:51,889 BAD EPOCHS (no improvement): 2 +2021-03-26 06:06:51,889 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:06:52,864 epoch 91 - iter 2/25 - loss 2.09060097 - samples/sec: 131.51 - lr: 0.000293 +2021-03-26 06:06:53,871 epoch 91 - iter 4/25 - loss 2.77089983 - samples/sec: 127.22 - lr: 0.000293 +2021-03-26 06:06:54,886 epoch 91 - iter 6/25 - loss 2.58453842 - samples/sec: 126.39 - lr: 0.000293 +2021-03-26 06:06:56,033 epoch 91 - iter 8/25 - loss 2.52161238 - samples/sec: 111.74 - lr: 0.000293 +2021-03-26 06:06:57,466 epoch 91 - iter 10/25 - loss 2.41345448 - samples/sec: 89.46 - lr: 0.000293 +2021-03-26 06:06:58,844 epoch 91 - iter 12/25 - loss 2.37983878 - samples/sec: 92.99 - lr: 0.000293 +2021-03-26 06:07:00,250 epoch 91 - iter 14/25 - loss 2.36699120 - samples/sec: 91.14 - lr: 0.000293 +2021-03-26 06:07:01,595 epoch 91 - iter 16/25 - loss 2.36627452 - samples/sec: 95.30 - lr: 0.000293 +2021-03-26 06:07:02,860 epoch 91 - iter 18/25 - loss 2.38054042 - samples/sec: 101.27 - lr: 0.000293 +2021-03-26 06:07:04,215 epoch 91 - iter 20/25 - loss 2.38775070 - samples/sec: 94.62 - lr: 0.000293 +2021-03-26 06:07:05,525 epoch 91 - iter 22/25 - loss 2.42452406 - samples/sec: 97.77 - lr: 0.000293 +2021-03-26 06:07:06,861 epoch 91 - iter 24/25 - loss 2.43479525 - samples/sec: 95.92 - lr: 0.000293 +2021-03-26 06:07:07,426 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:07:07,427 EPOCH 91 done: loss 2.4289 - lr 0.0002930 +2021-03-26 06:07:08,309 DEV : loss 6.747243404388428 - score 0.9058 +2021-03-26 06:07:08,345 BAD EPOCHS (no improvement): 3 +2021-03-26 06:07:08,346 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:07:09,714 epoch 92 - iter 2/25 - loss 2.56387246 - samples/sec: 93.67 - lr: 0.000293 +2021-03-26 06:07:11,135 epoch 92 - iter 4/25 - loss 2.47984356 - samples/sec: 90.18 - lr: 0.000293 +2021-03-26 06:07:12,472 epoch 92 - iter 6/25 - loss 2.34325659 - samples/sec: 95.81 - lr: 0.000293 +2021-03-26 06:07:13,501 epoch 92 - iter 8/25 - loss 2.34978437 - samples/sec: 124.61 - lr: 0.000293 +2021-03-26 06:07:14,436 epoch 92 - iter 10/25 - loss 2.38845448 - samples/sec: 137.32 - lr: 0.000293 +2021-03-26 06:07:15,382 epoch 92 - iter 12/25 - loss 2.38777759 - samples/sec: 135.50 - lr: 0.000293 +2021-03-26 06:07:16,411 epoch 92 - iter 14/25 - loss 2.46206682 - samples/sec: 124.52 - lr: 0.000293 +2021-03-26 06:07:17,485 epoch 92 - iter 16/25 - loss 2.48083726 - samples/sec: 119.50 - lr: 0.000293 +2021-03-26 06:07:18,507 epoch 92 - iter 18/25 - loss 2.50218729 - samples/sec: 125.45 - lr: 0.000293 +2021-03-26 06:07:19,597 epoch 92 - iter 20/25 - loss 2.49390016 - samples/sec: 117.58 - lr: 0.000293 +2021-03-26 06:07:20,652 epoch 92 - iter 22/25 - loss 2.48165541 - samples/sec: 121.53 - lr: 0.000293 +2021-03-26 06:07:21,588 epoch 92 - iter 24/25 - loss 2.45996833 - samples/sec: 137.24 - lr: 0.000293 +2021-03-26 06:07:22,022 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:07:22,023 EPOCH 92 done: loss 2.4582 - lr 0.0002930 +2021-03-26 06:07:22,843 DEV : loss 6.7463531494140625 - score 0.9058 +2021-03-26 06:07:22,865 BAD EPOCHS (no improvement): 4 +2021-03-26 06:07:22,866 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:07:23,916 epoch 93 - iter 2/25 - loss 2.83326006 - samples/sec: 122.07 - lr: 0.000146 +2021-03-26 06:07:24,990 epoch 93 - iter 4/25 - loss 2.57940161 - samples/sec: 119.39 - lr: 0.000146 +2021-03-26 06:07:26,116 epoch 93 - iter 6/25 - loss 2.51093678 - samples/sec: 113.84 - lr: 0.000146 +2021-03-26 06:07:27,157 epoch 93 - iter 8/25 - loss 2.54783881 - samples/sec: 123.21 - lr: 0.000146 +2021-03-26 06:07:28,087 epoch 93 - iter 10/25 - loss 2.52907882 - samples/sec: 137.85 - lr: 0.000146 +2021-03-26 06:07:29,180 epoch 93 - iter 12/25 - loss 2.53294051 - samples/sec: 117.34 - lr: 0.000146 +2021-03-26 06:07:30,152 epoch 93 - iter 14/25 - loss 2.58373092 - samples/sec: 131.92 - lr: 0.000146 +2021-03-26 06:07:31,101 epoch 93 - iter 16/25 - loss 2.49791358 - samples/sec: 135.12 - lr: 0.000146 +2021-03-26 06:07:32,036 epoch 93 - iter 18/25 - loss 2.52266951 - samples/sec: 137.17 - lr: 0.000146 +2021-03-26 06:07:33,002 epoch 93 - iter 20/25 - loss 2.55563478 - samples/sec: 132.64 - lr: 0.000146 +2021-03-26 06:07:33,961 epoch 93 - iter 22/25 - loss 2.51628619 - samples/sec: 133.82 - lr: 0.000146 +2021-03-26 06:07:35,016 epoch 93 - iter 24/25 - loss 2.50995046 - samples/sec: 121.45 - lr: 0.000146 +2021-03-26 06:07:35,421 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:07:35,422 EPOCH 93 done: loss 2.5024 - lr 0.0001465 +2021-03-26 06:07:36,224 DEV : loss 6.746026515960693 - score 0.9058 +2021-03-26 06:07:36,251 BAD EPOCHS (no improvement): 1 +2021-03-26 06:07:36,252 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:07:37,310 epoch 94 - iter 2/25 - loss 2.69705701 - samples/sec: 121.25 - lr: 0.000146 +2021-03-26 06:07:38,263 epoch 94 - iter 4/25 - loss 2.63027239 - samples/sec: 134.47 - lr: 0.000146 +2021-03-26 06:07:39,516 epoch 94 - iter 6/25 - loss 2.61560170 - samples/sec: 102.22 - lr: 0.000146 +2021-03-26 06:07:40,577 epoch 94 - iter 8/25 - loss 2.48729473 - samples/sec: 120.79 - lr: 0.000146 +2021-03-26 06:07:41,696 epoch 94 - iter 10/25 - loss 2.48122675 - samples/sec: 114.59 - lr: 0.000146 +2021-03-26 06:07:42,723 epoch 94 - iter 12/25 - loss 2.45633954 - samples/sec: 124.84 - lr: 0.000146 +2021-03-26 06:07:43,694 epoch 94 - iter 14/25 - loss 2.50454484 - samples/sec: 131.93 - lr: 0.000146 +2021-03-26 06:07:44,786 epoch 94 - iter 16/25 - loss 2.55948451 - samples/sec: 117.41 - lr: 0.000146 +2021-03-26 06:07:45,859 epoch 94 - iter 18/25 - loss 2.50821441 - samples/sec: 119.48 - lr: 0.000146 +2021-03-26 06:07:46,975 epoch 94 - iter 20/25 - loss 2.54369184 - samples/sec: 114.87 - lr: 0.000146 +2021-03-26 06:07:47,924 epoch 94 - iter 22/25 - loss 2.56217494 - samples/sec: 135.23 - lr: 0.000146 +2021-03-26 06:07:48,912 epoch 94 - iter 24/25 - loss 2.52295050 - samples/sec: 129.79 - lr: 0.000146 +2021-03-26 06:07:49,312 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:07:49,314 EPOCH 94 done: loss 2.5068 - lr 0.0001465 +2021-03-26 06:07:50,089 DEV : loss 6.7457427978515625 - score 0.9058 +2021-03-26 06:07:50,115 BAD EPOCHS (no improvement): 2 +2021-03-26 06:07:50,116 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:07:51,198 epoch 95 - iter 2/25 - loss 2.21429241 - samples/sec: 118.45 - lr: 0.000146 +2021-03-26 06:07:52,201 epoch 95 - iter 4/25 - loss 2.23874307 - samples/sec: 127.73 - lr: 0.000146 +2021-03-26 06:07:53,273 epoch 95 - iter 6/25 - loss 2.25965214 - samples/sec: 119.73 - lr: 0.000146 +2021-03-26 06:07:54,322 epoch 95 - iter 8/25 - loss 2.37573701 - samples/sec: 122.20 - lr: 0.000146 +2021-03-26 06:07:55,351 epoch 95 - iter 10/25 - loss 2.34428158 - samples/sec: 124.53 - lr: 0.000146 +2021-03-26 06:07:56,347 epoch 95 - iter 12/25 - loss 2.31626995 - samples/sec: 128.90 - lr: 0.000146 +2021-03-26 06:07:57,367 epoch 95 - iter 14/25 - loss 2.34982332 - samples/sec: 125.55 - lr: 0.000146 +2021-03-26 06:07:58,551 epoch 95 - iter 16/25 - loss 2.37993184 - samples/sec: 108.24 - lr: 0.000146 +2021-03-26 06:07:59,564 epoch 95 - iter 18/25 - loss 2.36954919 - samples/sec: 126.58 - lr: 0.000146 +2021-03-26 06:08:00,535 epoch 95 - iter 20/25 - loss 2.35135136 - samples/sec: 132.01 - lr: 0.000146 +2021-03-26 06:08:01,488 epoch 95 - iter 22/25 - loss 2.30894905 - samples/sec: 134.72 - lr: 0.000146 +2021-03-26 06:08:02,480 epoch 95 - iter 24/25 - loss 2.33361017 - samples/sec: 129.32 - lr: 0.000146 +2021-03-26 06:08:02,882 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:08:02,883 EPOCH 95 done: loss 2.3231 - lr 0.0001465 +2021-03-26 06:08:03,669 DEV : loss 6.745711326599121 - score 0.9058 +2021-03-26 06:08:03,694 BAD EPOCHS (no improvement): 3 +2021-03-26 06:08:03,694 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:08:04,616 epoch 96 - iter 2/25 - loss 2.30606425 - samples/sec: 139.22 - lr: 0.000146 +2021-03-26 06:08:05,646 epoch 96 - iter 4/25 - loss 2.47422612 - samples/sec: 124.52 - lr: 0.000146 +2021-03-26 06:08:06,612 epoch 96 - iter 6/25 - loss 2.45720597 - samples/sec: 132.77 - lr: 0.000146 +2021-03-26 06:08:07,633 epoch 96 - iter 8/25 - loss 2.44069329 - samples/sec: 125.46 - lr: 0.000146 +2021-03-26 06:08:08,753 epoch 96 - iter 10/25 - loss 2.38360977 - samples/sec: 114.49 - lr: 0.000146 +2021-03-26 06:08:09,991 epoch 96 - iter 12/25 - loss 2.33929481 - samples/sec: 103.49 - lr: 0.000146 +2021-03-26 06:08:11,153 epoch 96 - iter 14/25 - loss 2.29825859 - samples/sec: 110.32 - lr: 0.000146 +2021-03-26 06:08:12,130 epoch 96 - iter 16/25 - loss 2.28283183 - samples/sec: 131.24 - lr: 0.000146 +2021-03-26 06:08:13,162 epoch 96 - iter 18/25 - loss 2.28030713 - samples/sec: 124.31 - lr: 0.000146 +2021-03-26 06:08:14,189 epoch 96 - iter 20/25 - loss 2.30450888 - samples/sec: 124.70 - lr: 0.000146 +2021-03-26 06:08:15,193 epoch 96 - iter 22/25 - loss 2.33098056 - samples/sec: 127.78 - lr: 0.000146 +2021-03-26 06:08:16,322 epoch 96 - iter 24/25 - loss 2.31253702 - samples/sec: 113.43 - lr: 0.000146 +2021-03-26 06:08:16,783 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:08:16,784 EPOCH 96 done: loss 2.2834 - lr 0.0001465 +2021-03-26 06:08:17,592 DEV : loss 6.745593070983887 - score 0.9058 +2021-03-26 06:08:17,618 BAD EPOCHS (no improvement): 4 +2021-03-26 06:08:17,619 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:08:17,619 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:08:17,620 learning rate too small - quitting training! +2021-03-26 06:08:17,620 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:08:27,028 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:08:27,029 Testing using best model ... +2021-03-26 06:08:27,029 loading file /home/tmp/megahedm/models/multipos/multipos_UDMADAR_4Diale-LEV_EGY_GLF_MGR__fasttext_flairbwfw__64__0.3_202103260540/best-model.pt +2021-03-26 06:08:34,565 0.9157 +2021-03-26 06:08:34,565 +Results: +- F-score (micro): 0.912 +- F-score (macro): 0.5618 +- Accuracy (incl. no class): 0.9157 + +By class: + precision recall f1-score support + + NOUN 0.9314 0.9359 0.9336 421 + PRON 0.9732 0.9909 0.9820 220 + VERB 0.9359 0.9359 0.9359 156 + AUX 0.9200 0.9787 0.9485 47 + PROPN 0.9333 1.0000 0.9655 14 + ADV 0.9423 0.8909 0.9159 110 + ADP 0.9730 0.9643 0.9686 112 + ADJ 0.9174 0.8333 0.8734 120 + DET 0.9649 0.9016 0.9322 61 + CCONJ 1.0000 0.9620 0.9806 79 + NUM 0.8000 0.8889 0.8421 18 + PART 0.9241 0.9605 0.9419 228 + SCONJ 0.8684 0.9706 0.9167 34 + PUNCT 1.0000 1.0000 1.0000 22 + INTJ 1.0000 0.8750 0.9333 8 + MENTION 1.0000 1.0000 1.0000 21 + V 0.8990 0.9082 0.9036 98 + FUT_PART+V+PREP+PRON 1.0000 0.5000 0.6667 2 + PREP+ADV 0.0000 1.0000 0.0000 0 + ADJ+NSUFF 0.6897 0.9091 0.7843 22 + PREP+PRON 0.9259 0.8621 0.8929 29 + PUNC 0.9936 1.0000 0.9968 156 + EOS 1.0000 1.0000 1.0000 70 + DET+NOUN 0.9390 0.9872 0.9625 78 + PREP 0.9867 0.9610 0.9737 77 + HASH 0.9600 0.8889 0.9231 27 + PROG_PART+V 0.8919 0.9429 0.9167 35 + CONJ 1.0000 0.9762 0.9880 42 + NOUN+CASE 0.2500 0.5000 0.3333 2 + NOUN+PRON 0.8701 0.8816 0.8758 76 + V+PRON 0.6714 0.8103 0.7344 58 + PART+PRON 0.9167 0.8462 0.8800 13 + NOUN+NSUFF+PRON 0.7895 0.8333 0.8108 18 + PROG_PART+V+PRON 0.7500 0.7500 0.7500 16 + PREP+NOUN+NSUFF 0.3333 0.3333 0.3333 3 + NOUN+NSUFF 0.9167 0.8684 0.8919 38 + DET+NOUN+NSUFF 0.8400 0.8400 0.8400 25 + FOREIGN 1.0000 1.0000 1.0000 2 + CONJ+NOUN 0.7273 0.8889 0.8000 9 + CONJ+PART 0.8462 1.0000 0.9167 11 + PART+NSUFF 0.0000 1.0000 0.0000 0 + PREP+PART 1.0000 0.0000 0.0000 2 + ADJ+NSUFF+PRON 0.5000 0.5000 0.5000 2 + PREP+NOUN 0.6923 0.8182 0.7500 11 + PART+NOUN 0.6667 0.3333 0.4444 6 + PREP+PART+PRON 1.0000 1.0000 1.0000 3 + PREP+DET+ADJ 1.0000 0.0000 0.0000 1 + PREP+DET+NOUN 0.7500 0.6667 0.7059 9 + PREP+DET+NOUN+NSUFF 0.5000 0.6667 0.5714 3 + CONJ+V+PRON 0.2857 0.5000 0.3636 4 + CONJ+V 0.3333 0.6667 0.4444 3 + CONJ+DET+NOUN 0.3333 0.6667 0.4444 3 + CONJ+NOUN+NSUFF+PRON 1.0000 0.0000 0.0000 1 + CONJ+NOUN+PRON 0.7143 0.8333 0.7692 6 + PART+V+PRON+PRON 1.0000 0.0000 0.0000 1 + ADJ+PRON 0.3333 0.2000 0.2500 5 + V+PRON+PRON 0.6000 0.3000 0.4000 10 + CONJ+DET+NOUN+NSUFF 1.0000 0.0000 0.0000 1 + DET+ADJ+NSUFF 0.5000 0.5000 0.5000 6 + EMOT 0.8571 1.0000 0.9231 6 + CONJ+PREP 1.0000 0.5000 0.6667 2 + PRON+DET+NOUN 0.0000 1.0000 0.0000 0 + CONJ+PRON 1.0000 1.0000 1.0000 4 + DET+ADJ 1.0000 0.8182 0.9000 11 + PART+V+PRON+NEG_PART 0.5000 1.0000 0.6667 2 + V+PRON+PREP+PRON 0.5000 0.5000 0.5000 2 + PROG_PART+V+PREP+PRON 0.0000 0.0000 0.0000 1 + CONJ+PROG_PART+V 1.0000 0.5000 0.6667 2 + URL 1.0000 1.0000 1.0000 3 + NOUN+PRON+NEG_PART 1.0000 0.0000 0.0000 1 +PART+PROG_PART+V+NEG_PART 0.5000 0.5000 0.5000 2 + FUT_PART+V 0.9167 0.9167 0.9167 12 + PRON+NOUN 1.0000 0.0000 0.0000 1 + NUM+NSUFF 0.0000 0.0000 0.0000 3 + CONJ+PREP+PRON 1.0000 1.0000 1.0000 1 + FUT_PART 1.0000 1.0000 1.0000 3 + PREP+NOUN+PRON 0.6667 0.5714 0.6154 7 + PART+V 1.0000 0.0000 0.0000 2 + V+PREP+PRON 0.6667 0.4000 0.5000 5 + CONJ+V+PRON+PREP+PRON 1.0000 0.0000 0.0000 1 + CONJ+DET+ADJ 0.0000 1.0000 0.0000 0 + PART+NOUN+PRON 1.0000 0.0000 0.0000 1 + ADJ+NSUFF+PREP+PRON 1.0000 0.0000 0.0000 1 + CONJ+NOUN+NSUFF 0.6667 1.0000 0.8000 2 + CONJ+PART+PROG_PART+V 1.0000 0.0000 0.0000 1 + ADV+NSUFF 1.0000 1.0000 1.0000 1 +PART+V+PRON+PRON+NEG_PART 1.0000 0.0000 0.0000 1 +PART+V+PREP+PRON+NEG_PART 0.0000 1.0000 0.0000 0 + PART+PREP+NEG_PART 1.0000 1.0000 1.0000 2 + ADJ+CASE 1.0000 0.5000 0.6667 2 + CONJ+ADV+NSUFF 1.0000 0.0000 0.0000 1 + DET+NUM 1.0000 1.0000 1.0000 1 + CONJ+PROG_PART+V+PRON 1.0000 0.0000 0.0000 1 + PREP+DET+NOUN+NSUFF+PART 1.0000 0.0000 0.0000 1 + CONJ+PREP+DET+NOUN+NSUFF 1.0000 0.0000 0.0000 1 + PART+NOUN+NEG_PART 1.0000 1.0000 1.0000 1 + FUT_PART+V+PRON 1.0000 0.2500 0.4000 4 + CONJ+ADJ+NSUFF 1.0000 0.0000 0.0000 2 + NOUN+CASE+PRON 1.0000 1.0000 1.0000 1 + CONJ+NOUN+CASE 1.0000 0.0000 0.0000 1 + CONJ+PART+V 0.0000 1.0000 0.0000 0 + PART+PREP+PRON 1.0000 0.0000 0.0000 1 + V+NSUFF 1.0000 0.0000 0.0000 1 + PREP+NOUN+NSUFF+PRON 1.0000 0.0000 0.0000 1 + PROG_PART+V+PRON+PRON 1.0000 0.0000 0.0000 1 + PART+NOUN+NSUFF 1.0000 0.0000 0.0000 1 + PART+V+NEG_PART 1.0000 1.0000 1.0000 1 + + micro avg 0.9125 0.9115 0.9120 2757 + macro avg 0.8015 0.6232 0.5618 2757 + weighted avg 0.9195 0.9115 0.9086 2757 + +2021-03-26 06:08:34,566 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:08:34,566 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:08:41,131 Reading data from ../../Datasets_adhoc/CSCS_corpus-GUC +2021-03-26 06:08:41,132 Train: ../../Datasets_adhoc/CSCS_corpus-GUC/all_participants.conllu +2021-03-26 06:08:41,132 Dev: None +2021-03-26 06:08:41,133 Test: None +2021-03-26 06:08:41,417 Reading data from ../../Datasets_adhoc/UD_MADAR +2021-03-26 06:08:41,417 Train: ../../Datasets_adhoc/UD_MADAR/ajp_madar-ud-test-edit.conllu +2021-03-26 06:08:41,418 Dev: None +2021-03-26 06:08:41,418 Test: None +2021-03-26 06:08:41,462 Reading data from ../../Datasets_adhoc/dialectal_arabic_resources-master +2021-03-26 06:08:41,462 Train: ../../Datasets_adhoc/dialectal_arabic_resources-master/seg_plus_pos_lev.txt +2021-03-26 06:08:41,462 Dev: None +2021-03-26 06:08:41,462 Test: None +2021-03-26 06:08:41,610 Reading data from ../../Datasets_adhoc/dialectal_arabic_resources-master +2021-03-26 06:08:41,611 Train: ../../Datasets_adhoc/dialectal_arabic_resources-master/seg_plus_pos_egy.txt +2021-03-26 06:08:41,611 Dev: None +2021-03-26 06:08:41,611 Test: None +2021-03-26 06:08:41,782 Reading data from ../../Datasets_adhoc/dialectal_arabic_resources-master +2021-03-26 06:08:41,783 Train: ../../Datasets_adhoc/dialectal_arabic_resources-master/seg_plus_pos_glf.txt +2021-03-26 06:08:41,783 Dev: None +2021-03-26 06:08:41,783 Test: None +2021-03-26 06:08:41,954 Reading data from ../../Datasets_adhoc/dialectal_arabic_resources-master +2021-03-26 06:08:41,955 Train: ../../Datasets_adhoc/dialectal_arabic_resources-master/seg_plus_pos_mgr.txt +2021-03-26 06:08:41,955 Dev: None +2021-03-26 06:08:41,956 Test: None +2021-03-26 06:08:42,103 Filtering long sentences +2021-03-26 06:08:42,144 MultiCorpus: 1573 train + 176 dev + 195 test sentences + - ColumnCorpus Corpus: 934 train + 104 dev + 115 test sentences + - ColumnCorpus Corpus: 81 train + 9 dev + 10 test sentences + - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences + - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences +2021-03-26 06:08:42,545 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:08:42,546 Model: "SequenceTagger( + (embeddings): StackedEmbeddings( + (list_embedding_0): WordEmbeddings('ar') + (list_embedding_1): FlairEmbeddings( + (lm): LanguageModel( + (drop): Dropout(p=0.1, inplace=False) + (encoder): Embedding(7125, 100) + (rnn): LSTM(100, 2048) + (decoder): Linear(in_features=2048, out_features=7125, bias=True) + ) + ) + (list_embedding_2): FlairEmbeddings( + (lm): LanguageModel( + (drop): Dropout(p=0.1, inplace=False) + (encoder): Embedding(7125, 100) + (rnn): LSTM(100, 2048) + (decoder): Linear(in_features=2048, out_features=7125, bias=True) + ) + ) + ) + (word_dropout): WordDropout(p=0.05) + (locked_dropout): LockedDropout(p=0.5) + (embedding2nn): Linear(in_features=4396, out_features=4396, bias=True) + (rnn): LSTM(4396, 256, batch_first=True, bidirectional=True) + (linear): Linear(in_features=512, out_features=206, bias=True) + (beta): 1.0 + (weights): None + (weight_tensor) None +)" +2021-03-26 06:08:42,546 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:08:42,546 Corpus: "MultiCorpus: 1573 train + 176 dev + 195 test sentences + - ColumnCorpus Corpus: 934 train + 104 dev + 115 test sentences + - ColumnCorpus Corpus: 81 train + 9 dev + 10 test sentences + - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences + - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences" +2021-03-26 06:08:42,547 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:08:42,547 Parameters: +2021-03-26 06:08:42,547 - learning_rate: "0.5" +2021-03-26 06:08:42,548 - mini_batch_size: "32" +2021-03-26 06:08:42,548 - patience: "3" +2021-03-26 06:08:42,548 - anneal_factor: "0.5" +2021-03-26 06:08:42,548 - max_epochs: "150" +2021-03-26 06:08:42,549 - shuffle: "True" +2021-03-26 06:08:42,549 - train_with_dev: "False" +2021-03-26 06:08:42,549 - batch_growth_annealing: "False" +2021-03-26 06:08:42,549 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:08:42,550 Model training base path: "/home/tmp/megahedm/models/multipos/multipos_UDMADAR_4Diale-LEV_EGY_GLF_MGR__fasttext_flairbwfw__32__0.5_202103260608" +2021-03-26 06:08:42,550 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:08:42,550 Device: cuda:0 +2021-03-26 06:08:42,551 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:08:42,551 Embeddings storage mode: cpu +2021-03-26 06:08:42,552 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:08:45,412 epoch 1 - iter 5/50 - loss 82.38076324 - samples/sec: 56.00 - lr: 0.500000 +2021-03-26 06:08:47,767 epoch 1 - iter 10/50 - loss 72.45342216 - samples/sec: 67.97 - lr: 0.500000 +2021-03-26 06:08:50,426 epoch 1 - iter 15/50 - loss 69.72581075 - samples/sec: 60.21 - lr: 0.500000 +2021-03-26 06:08:52,798 epoch 1 - iter 20/50 - loss 65.73783169 - samples/sec: 67.50 - lr: 0.500000 +2021-03-26 06:08:55,293 epoch 1 - iter 25/50 - loss 62.81619507 - samples/sec: 64.18 - lr: 0.500000 +2021-03-26 06:08:57,724 epoch 1 - iter 30/50 - loss 59.64540342 - samples/sec: 65.87 - lr: 0.500000 +2021-03-26 06:09:00,117 epoch 1 - iter 35/50 - loss 56.33768899 - samples/sec: 66.90 - lr: 0.500000 +2021-03-26 06:09:02,584 epoch 1 - iter 40/50 - loss 53.98569193 - samples/sec: 64.92 - lr: 0.500000 +2021-03-26 06:09:05,056 epoch 1 - iter 45/50 - loss 51.83909531 - samples/sec: 64.76 - lr: 0.500000 +2021-03-26 06:09:07,437 epoch 1 - iter 50/50 - loss 49.78158417 - samples/sec: 67.27 - lr: 0.500000 +2021-03-26 06:09:07,438 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:09:07,438 EPOCH 1 done: loss 49.7816 - lr 0.5000000 +2021-03-26 06:09:08,778 DEV : loss 32.66966247558594 - score 0.5248 +2021-03-26 06:09:08,803 BAD EPOCHS (no improvement): 0 +2021-03-26 06:09:18,311 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:09:20,362 epoch 2 - iter 5/50 - loss 29.21817436 - samples/sec: 78.13 - lr: 0.500000 +2021-03-26 06:09:22,218 epoch 2 - iter 10/50 - loss 27.90745525 - samples/sec: 86.29 - lr: 0.500000 +2021-03-26 06:09:24,250 epoch 2 - iter 15/50 - loss 27.92045682 - samples/sec: 78.85 - lr: 0.500000 +2021-03-26 06:09:26,263 epoch 2 - iter 20/50 - loss 27.60578327 - samples/sec: 79.54 - lr: 0.500000 +2021-03-26 06:09:28,349 epoch 2 - iter 25/50 - loss 27.37625938 - samples/sec: 76.80 - lr: 0.500000 +2021-03-26 06:09:30,286 epoch 2 - iter 30/50 - loss 26.46395251 - samples/sec: 82.74 - lr: 0.500000 +2021-03-26 06:09:32,155 epoch 2 - iter 35/50 - loss 25.86999474 - samples/sec: 85.71 - lr: 0.500000 +2021-03-26 06:09:33,980 epoch 2 - iter 40/50 - loss 25.79278846 - samples/sec: 87.74 - lr: 0.500000 +2021-03-26 06:09:36,027 epoch 2 - iter 45/50 - loss 25.15292172 - samples/sec: 78.25 - lr: 0.500000 +2021-03-26 06:09:37,966 epoch 2 - iter 50/50 - loss 24.61890293 - samples/sec: 82.63 - lr: 0.500000 +2021-03-26 06:09:37,966 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:09:37,966 EPOCH 2 done: loss 24.6189 - lr 0.5000000 +2021-03-26 06:09:38,823 DEV : loss 18.001943588256836 - score 0.6871 +2021-03-26 06:09:38,852 BAD EPOCHS (no improvement): 0 +2021-03-26 06:09:48,858 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:09:50,846 epoch 3 - iter 5/50 - loss 20.17376251 - samples/sec: 80.59 - lr: 0.500000 +2021-03-26 06:09:52,846 epoch 3 - iter 10/50 - loss 19.30381107 - samples/sec: 80.10 - lr: 0.500000 +2021-03-26 06:09:54,888 epoch 3 - iter 15/50 - loss 18.62234275 - samples/sec: 78.44 - lr: 0.500000 +2021-03-26 06:09:56,880 epoch 3 - iter 20/50 - loss 18.49893241 - samples/sec: 80.53 - lr: 0.500000 +2021-03-26 06:09:58,848 epoch 3 - iter 25/50 - loss 18.25910244 - samples/sec: 81.38 - lr: 0.500000 +2021-03-26 06:10:00,846 epoch 3 - iter 30/50 - loss 17.74629262 - samples/sec: 80.16 - lr: 0.500000 +2021-03-26 06:10:02,721 epoch 3 - iter 35/50 - loss 17.59613062 - samples/sec: 85.41 - lr: 0.500000 +2021-03-26 06:10:04,734 epoch 3 - iter 40/50 - loss 17.44145768 - samples/sec: 79.58 - lr: 0.500000 +2021-03-26 06:10:06,725 epoch 3 - iter 45/50 - loss 17.46167198 - samples/sec: 80.44 - lr: 0.500000 +2021-03-26 06:10:08,382 epoch 3 - iter 50/50 - loss 17.12887486 - samples/sec: 96.65 - lr: 0.500000 +2021-03-26 06:10:08,383 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:10:08,383 EPOCH 3 done: loss 17.1289 - lr 0.5000000 +2021-03-26 06:10:09,199 DEV : loss 15.706961631774902 - score 0.732 +2021-03-26 06:10:09,225 BAD EPOCHS (no improvement): 0 +2021-03-26 06:10:18,978 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:10:21,066 epoch 4 - iter 5/50 - loss 13.97613335 - samples/sec: 76.77 - lr: 0.500000 +2021-03-26 06:10:23,081 epoch 4 - iter 10/50 - loss 14.95872822 - samples/sec: 79.48 - lr: 0.500000 +2021-03-26 06:10:25,143 epoch 4 - iter 15/50 - loss 14.66935984 - samples/sec: 77.66 - lr: 0.500000 +2021-03-26 06:10:27,077 epoch 4 - iter 20/50 - loss 14.39494796 - samples/sec: 82.77 - lr: 0.500000 +2021-03-26 06:10:29,005 epoch 4 - iter 25/50 - loss 14.47061756 - samples/sec: 83.12 - lr: 0.500000 +2021-03-26 06:10:30,820 epoch 4 - iter 30/50 - loss 14.52058875 - samples/sec: 88.27 - lr: 0.500000 +2021-03-26 06:10:32,733 epoch 4 - iter 35/50 - loss 14.39601618 - samples/sec: 83.71 - lr: 0.500000 +2021-03-26 06:10:34,640 epoch 4 - iter 40/50 - loss 13.98630881 - samples/sec: 83.98 - lr: 0.500000 +2021-03-26 06:10:36,501 epoch 4 - iter 45/50 - loss 14.07221792 - samples/sec: 86.04 - lr: 0.500000 +2021-03-26 06:10:38,324 epoch 4 - iter 50/50 - loss 14.00682487 - samples/sec: 87.87 - lr: 0.500000 +2021-03-26 06:10:38,325 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:10:38,325 EPOCH 4 done: loss 14.0068 - lr 0.5000000 +2021-03-26 06:10:39,130 DEV : loss 12.180521965026855 - score 0.7796 +2021-03-26 06:10:39,155 BAD EPOCHS (no improvement): 0 +2021-03-26 06:10:48,923 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:10:51,016 epoch 5 - iter 5/50 - loss 11.56535034 - samples/sec: 76.55 - lr: 0.500000 +2021-03-26 06:10:53,638 epoch 5 - iter 10/50 - loss 11.53233051 - samples/sec: 61.08 - lr: 0.500000 +2021-03-26 06:10:55,804 epoch 5 - iter 15/50 - loss 12.04551531 - samples/sec: 73.94 - lr: 0.500000 +2021-03-26 06:10:57,689 epoch 5 - iter 20/50 - loss 12.20313921 - samples/sec: 84.96 - lr: 0.500000 +2021-03-26 06:10:59,583 epoch 5 - iter 25/50 - loss 12.66698563 - samples/sec: 84.56 - lr: 0.500000 +2021-03-26 06:11:01,684 epoch 5 - iter 30/50 - loss 12.56654412 - samples/sec: 76.27 - lr: 0.500000 +2021-03-26 06:11:03,679 epoch 5 - iter 35/50 - loss 12.49077797 - samples/sec: 80.27 - lr: 0.500000 +2021-03-26 06:11:05,908 epoch 5 - iter 40/50 - loss 12.50401361 - samples/sec: 71.87 - lr: 0.500000 +2021-03-26 06:11:07,803 epoch 5 - iter 45/50 - loss 12.35786915 - samples/sec: 84.52 - lr: 0.500000 +2021-03-26 06:11:09,566 epoch 5 - iter 50/50 - loss 12.31663839 - samples/sec: 90.85 - lr: 0.500000 +2021-03-26 06:11:09,567 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:11:09,567 EPOCH 5 done: loss 12.3166 - lr 0.5000000 +2021-03-26 06:11:10,380 DEV : loss 9.9617280960083 - score 0.8285 +2021-03-26 06:11:10,406 BAD EPOCHS (no improvement): 0 +2021-03-26 06:11:19,847 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:11:21,763 epoch 6 - iter 5/50 - loss 9.79197845 - samples/sec: 83.62 - lr: 0.500000 +2021-03-26 06:11:23,801 epoch 6 - iter 10/50 - loss 9.94693480 - samples/sec: 78.55 - lr: 0.500000 +2021-03-26 06:11:25,926 epoch 6 - iter 15/50 - loss 10.89867058 - samples/sec: 75.38 - lr: 0.500000 +2021-03-26 06:11:27,780 epoch 6 - iter 20/50 - loss 11.04457297 - samples/sec: 86.41 - lr: 0.500000 +2021-03-26 06:11:29,718 epoch 6 - iter 25/50 - loss 10.92623383 - samples/sec: 82.63 - lr: 0.500000 +2021-03-26 06:11:31,983 epoch 6 - iter 30/50 - loss 11.11003348 - samples/sec: 70.68 - lr: 0.500000 +2021-03-26 06:11:33,943 epoch 6 - iter 35/50 - loss 11.11525487 - samples/sec: 81.71 - lr: 0.500000 +2021-03-26 06:11:35,953 epoch 6 - iter 40/50 - loss 11.06258175 - samples/sec: 79.69 - lr: 0.500000 +2021-03-26 06:11:37,890 epoch 6 - iter 45/50 - loss 10.88173576 - samples/sec: 82.69 - lr: 0.500000 +2021-03-26 06:11:39,859 epoch 6 - iter 50/50 - loss 10.87174243 - samples/sec: 81.33 - lr: 0.500000 +2021-03-26 06:11:39,860 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:11:39,861 EPOCH 6 done: loss 10.8717 - lr 0.5000000 +2021-03-26 06:11:40,690 DEV : loss 9.194941520690918 - score 0.8365 +2021-03-26 06:11:40,718 BAD EPOCHS (no improvement): 0 +2021-03-26 06:11:50,455 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:11:52,589 epoch 7 - iter 5/50 - loss 9.42767963 - samples/sec: 75.11 - lr: 0.500000 +2021-03-26 06:11:54,539 epoch 7 - iter 10/50 - loss 10.54737329 - samples/sec: 82.16 - lr: 0.500000 +2021-03-26 06:11:56,590 epoch 7 - iter 15/50 - loss 10.36283264 - samples/sec: 78.09 - lr: 0.500000 +2021-03-26 06:11:58,549 epoch 7 - iter 20/50 - loss 9.77013283 - samples/sec: 81.77 - lr: 0.500000 +2021-03-26 06:12:00,711 epoch 7 - iter 25/50 - loss 9.82616146 - samples/sec: 74.04 - lr: 0.500000 +2021-03-26 06:12:02,534 epoch 7 - iter 30/50 - loss 9.88486067 - samples/sec: 87.88 - lr: 0.500000 +2021-03-26 06:12:04,348 epoch 7 - iter 35/50 - loss 9.92839804 - samples/sec: 88.30 - lr: 0.500000 +2021-03-26 06:12:06,383 epoch 7 - iter 40/50 - loss 9.83319703 - samples/sec: 78.71 - lr: 0.500000 +2021-03-26 06:12:08,432 epoch 7 - iter 45/50 - loss 9.77937012 - samples/sec: 78.18 - lr: 0.500000 +2021-03-26 06:12:10,308 epoch 7 - iter 50/50 - loss 9.99064981 - samples/sec: 85.37 - lr: 0.500000 +2021-03-26 06:12:10,309 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:12:10,310 EPOCH 7 done: loss 9.9906 - lr 0.5000000 +2021-03-26 06:12:11,157 DEV : loss 9.499085426330566 - score 0.8304 +2021-03-26 06:12:11,175 BAD EPOCHS (no improvement): 1 +2021-03-26 06:12:11,176 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:12:13,192 epoch 8 - iter 5/50 - loss 9.51839771 - samples/sec: 79.42 - lr: 0.500000 +2021-03-26 06:12:15,302 epoch 8 - iter 10/50 - loss 9.78024168 - samples/sec: 75.92 - lr: 0.500000 +2021-03-26 06:12:17,159 epoch 8 - iter 15/50 - loss 9.80983674 - samples/sec: 86.26 - lr: 0.500000 +2021-03-26 06:12:19,321 epoch 8 - iter 20/50 - loss 9.67828443 - samples/sec: 74.08 - lr: 0.500000 +2021-03-26 06:12:21,384 epoch 8 - iter 25/50 - loss 9.43407003 - samples/sec: 77.64 - lr: 0.500000 +2021-03-26 06:12:23,454 epoch 8 - iter 30/50 - loss 9.60205196 - samples/sec: 77.35 - lr: 0.500000 +2021-03-26 06:12:25,465 epoch 8 - iter 35/50 - loss 9.51624957 - samples/sec: 79.61 - lr: 0.500000 +2021-03-26 06:12:27,402 epoch 8 - iter 40/50 - loss 9.43358691 - samples/sec: 82.70 - lr: 0.500000 +2021-03-26 06:12:29,527 epoch 8 - iter 45/50 - loss 9.54377569 - samples/sec: 75.34 - lr: 0.500000 +2021-03-26 06:12:31,460 epoch 8 - iter 50/50 - loss 9.43047031 - samples/sec: 82.89 - lr: 0.500000 +2021-03-26 06:12:31,461 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:12:31,461 EPOCH 8 done: loss 9.4305 - lr 0.5000000 +2021-03-26 06:12:32,288 DEV : loss 8.57904052734375 - score 0.8468 +2021-03-26 06:12:32,315 BAD EPOCHS (no improvement): 0 +2021-03-26 06:12:42,103 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:12:44,199 epoch 9 - iter 5/50 - loss 7.69350290 - samples/sec: 76.48 - lr: 0.500000 +2021-03-26 06:12:46,324 epoch 9 - iter 10/50 - loss 7.68171325 - samples/sec: 75.38 - lr: 0.500000 +2021-03-26 06:12:48,318 epoch 9 - iter 15/50 - loss 8.52010759 - samples/sec: 80.31 - lr: 0.500000 +2021-03-26 06:12:50,344 epoch 9 - iter 20/50 - loss 8.30756495 - samples/sec: 79.07 - lr: 0.500000 +2021-03-26 06:12:52,326 epoch 9 - iter 25/50 - loss 8.40493088 - samples/sec: 80.82 - lr: 0.500000 +2021-03-26 06:12:54,388 epoch 9 - iter 30/50 - loss 8.46827960 - samples/sec: 77.68 - lr: 0.500000 +2021-03-26 06:12:56,481 epoch 9 - iter 35/50 - loss 8.47599471 - samples/sec: 76.50 - lr: 0.500000 +2021-03-26 06:12:58,432 epoch 9 - iter 40/50 - loss 8.47426549 - samples/sec: 82.10 - lr: 0.500000 +2021-03-26 06:13:00,478 epoch 9 - iter 45/50 - loss 8.47557190 - samples/sec: 78.27 - lr: 0.500000 +2021-03-26 06:13:02,428 epoch 9 - iter 50/50 - loss 8.44233681 - samples/sec: 82.13 - lr: 0.500000 +2021-03-26 06:13:02,429 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:13:02,429 EPOCH 9 done: loss 8.4423 - lr 0.5000000 +2021-03-26 06:13:03,234 DEV : loss 7.8837103843688965 - score 0.8662 +2021-03-26 06:13:03,260 BAD EPOCHS (no improvement): 0 +2021-03-26 06:13:12,899 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:13:15,004 epoch 10 - iter 5/50 - loss 7.26523943 - samples/sec: 76.13 - lr: 0.500000 +2021-03-26 06:13:16,811 epoch 10 - iter 10/50 - loss 7.22758307 - samples/sec: 88.64 - lr: 0.500000 +2021-03-26 06:13:18,871 epoch 10 - iter 15/50 - loss 7.90553220 - samples/sec: 77.74 - lr: 0.500000 +2021-03-26 06:13:20,825 epoch 10 - iter 20/50 - loss 8.07978196 - samples/sec: 81.93 - lr: 0.500000 +2021-03-26 06:13:22,788 epoch 10 - iter 25/50 - loss 8.07027122 - samples/sec: 81.62 - lr: 0.500000 +2021-03-26 06:13:24,679 epoch 10 - iter 30/50 - loss 7.96194596 - samples/sec: 84.71 - lr: 0.500000 +2021-03-26 06:13:26,850 epoch 10 - iter 35/50 - loss 8.07621027 - samples/sec: 73.80 - lr: 0.500000 +2021-03-26 06:13:28,847 epoch 10 - iter 40/50 - loss 8.07766933 - samples/sec: 80.18 - lr: 0.500000 +2021-03-26 06:13:30,754 epoch 10 - iter 45/50 - loss 8.08958091 - samples/sec: 83.95 - lr: 0.500000 +2021-03-26 06:13:32,690 epoch 10 - iter 50/50 - loss 8.25567496 - samples/sec: 82.80 - lr: 0.500000 +2021-03-26 06:13:32,690 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:13:32,691 EPOCH 10 done: loss 8.2557 - lr 0.5000000 +2021-03-26 06:13:33,492 DEV : loss 7.9223737716674805 - score 0.8646 +2021-03-26 06:13:33,517 BAD EPOCHS (no improvement): 1 +2021-03-26 06:13:33,518 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:13:35,584 epoch 11 - iter 5/50 - loss 6.95426130 - samples/sec: 77.54 - lr: 0.500000 +2021-03-26 06:13:37,640 epoch 11 - iter 10/50 - loss 7.45029273 - samples/sec: 77.89 - lr: 0.500000 +2021-03-26 06:13:39,666 epoch 11 - iter 15/50 - loss 7.58958505 - samples/sec: 79.07 - lr: 0.500000 +2021-03-26 06:13:41,612 epoch 11 - iter 20/50 - loss 7.46826644 - samples/sec: 82.29 - lr: 0.500000 +2021-03-26 06:13:43,535 epoch 11 - iter 25/50 - loss 7.58064199 - samples/sec: 83.26 - lr: 0.500000 +2021-03-26 06:13:45,470 epoch 11 - iter 30/50 - loss 7.82060990 - samples/sec: 82.79 - lr: 0.500000 +2021-03-26 06:13:47,670 epoch 11 - iter 35/50 - loss 7.85852415 - samples/sec: 72.74 - lr: 0.500000 +2021-03-26 06:13:49,684 epoch 11 - iter 40/50 - loss 7.85756209 - samples/sec: 79.57 - lr: 0.500000 +2021-03-26 06:13:51,764 epoch 11 - iter 45/50 - loss 7.93629023 - samples/sec: 76.99 - lr: 0.500000 +2021-03-26 06:13:53,692 epoch 11 - iter 50/50 - loss 7.82427173 - samples/sec: 83.10 - lr: 0.500000 +2021-03-26 06:13:53,692 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:13:53,693 EPOCH 11 done: loss 7.8243 - lr 0.5000000 +2021-03-26 06:13:54,500 DEV : loss 8.221084594726562 - score 0.8649 +2021-03-26 06:13:54,526 BAD EPOCHS (no improvement): 2 +2021-03-26 06:13:54,527 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:13:56,488 epoch 12 - iter 5/50 - loss 6.72069206 - samples/sec: 81.66 - lr: 0.500000 +2021-03-26 06:13:58,466 epoch 12 - iter 10/50 - loss 6.91607099 - samples/sec: 81.02 - lr: 0.500000 +2021-03-26 06:14:00,469 epoch 12 - iter 15/50 - loss 7.14914818 - samples/sec: 79.93 - lr: 0.500000 +2021-03-26 06:14:02,477 epoch 12 - iter 20/50 - loss 7.04800429 - samples/sec: 79.77 - lr: 0.500000 +2021-03-26 06:14:04,583 epoch 12 - iter 25/50 - loss 6.98862305 - samples/sec: 76.06 - lr: 0.500000 +2021-03-26 06:14:06,535 epoch 12 - iter 30/50 - loss 7.23589487 - samples/sec: 82.02 - lr: 0.500000 +2021-03-26 06:14:08,669 epoch 12 - iter 35/50 - loss 7.31580941 - samples/sec: 75.06 - lr: 0.500000 +2021-03-26 06:14:10,699 epoch 12 - iter 40/50 - loss 7.40670675 - samples/sec: 78.87 - lr: 0.500000 +2021-03-26 06:14:12,860 epoch 12 - iter 45/50 - loss 7.41732713 - samples/sec: 74.12 - lr: 0.500000 +2021-03-26 06:14:14,847 epoch 12 - iter 50/50 - loss 7.43627445 - samples/sec: 80.58 - lr: 0.500000 +2021-03-26 06:14:14,848 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:14:14,849 EPOCH 12 done: loss 7.4363 - lr 0.5000000 +2021-03-26 06:14:15,689 DEV : loss 7.278804779052734 - score 0.8755 +2021-03-26 06:14:15,719 BAD EPOCHS (no improvement): 0 +2021-03-26 06:14:25,575 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:14:27,591 epoch 13 - iter 5/50 - loss 6.20711269 - samples/sec: 79.44 - lr: 0.500000 +2021-03-26 06:14:29,702 epoch 13 - iter 10/50 - loss 6.75995908 - samples/sec: 75.89 - lr: 0.500000 +2021-03-26 06:14:31,766 epoch 13 - iter 15/50 - loss 6.80636247 - samples/sec: 77.59 - lr: 0.500000 +2021-03-26 06:14:33,680 epoch 13 - iter 20/50 - loss 6.94637887 - samples/sec: 83.64 - lr: 0.500000 +2021-03-26 06:14:35,674 epoch 13 - iter 25/50 - loss 6.90052332 - samples/sec: 80.31 - lr: 0.500000 +2021-03-26 06:14:37,568 epoch 13 - iter 30/50 - loss 6.98368498 - samples/sec: 84.58 - lr: 0.500000 +2021-03-26 06:14:39,824 epoch 13 - iter 35/50 - loss 6.98485334 - samples/sec: 71.01 - lr: 0.500000 +2021-03-26 06:14:41,879 epoch 13 - iter 40/50 - loss 7.06695175 - samples/sec: 77.93 - lr: 0.500000 +2021-03-26 06:14:43,867 epoch 13 - iter 45/50 - loss 6.98606850 - samples/sec: 80.57 - lr: 0.500000 +2021-03-26 06:14:45,701 epoch 13 - iter 50/50 - loss 7.08044469 - samples/sec: 87.32 - lr: 0.500000 +2021-03-26 06:14:45,701 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:14:45,702 EPOCH 13 done: loss 7.0804 - lr 0.5000000 +2021-03-26 06:14:46,504 DEV : loss 7.4897284507751465 - score 0.8719 +2021-03-26 06:14:46,529 BAD EPOCHS (no improvement): 1 +2021-03-26 06:14:46,530 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:14:48,405 epoch 14 - iter 5/50 - loss 7.57633114 - samples/sec: 85.46 - lr: 0.500000 +2021-03-26 06:14:50,632 epoch 14 - iter 10/50 - loss 7.42076797 - samples/sec: 71.90 - lr: 0.500000 +2021-03-26 06:14:52,687 epoch 14 - iter 15/50 - loss 7.21374400 - samples/sec: 77.95 - lr: 0.500000 +2021-03-26 06:14:54,498 epoch 14 - iter 20/50 - loss 7.15166688 - samples/sec: 88.41 - lr: 0.500000 +2021-03-26 06:14:56,346 epoch 14 - iter 25/50 - loss 6.90149778 - samples/sec: 86.67 - lr: 0.500000 +2021-03-26 06:14:58,407 epoch 14 - iter 30/50 - loss 6.92999091 - samples/sec: 77.72 - lr: 0.500000 +2021-03-26 06:15:00,390 epoch 14 - iter 35/50 - loss 6.82046195 - samples/sec: 80.78 - lr: 0.500000 +2021-03-26 06:15:02,279 epoch 14 - iter 40/50 - loss 6.83094221 - samples/sec: 84.77 - lr: 0.500000 +2021-03-26 06:15:04,448 epoch 14 - iter 45/50 - loss 6.72926092 - samples/sec: 73.84 - lr: 0.500000 +2021-03-26 06:15:06,312 epoch 14 - iter 50/50 - loss 6.71817581 - samples/sec: 85.93 - lr: 0.500000 +2021-03-26 06:15:06,312 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:15:06,313 EPOCH 14 done: loss 6.7182 - lr 0.5000000 +2021-03-26 06:15:07,135 DEV : loss 7.169527053833008 - score 0.8864 +2021-03-26 06:15:07,167 BAD EPOCHS (no improvement): 0 +2021-03-26 06:15:16,787 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:15:18,732 epoch 15 - iter 5/50 - loss 5.80208635 - samples/sec: 82.38 - lr: 0.500000 +2021-03-26 06:15:20,703 epoch 15 - iter 10/50 - loss 5.96044936 - samples/sec: 81.26 - lr: 0.500000 +2021-03-26 06:15:22,749 epoch 15 - iter 15/50 - loss 6.08839035 - samples/sec: 78.27 - lr: 0.500000 +2021-03-26 06:15:24,806 epoch 15 - iter 20/50 - loss 5.84570023 - samples/sec: 77.86 - lr: 0.500000 +2021-03-26 06:15:26,917 epoch 15 - iter 25/50 - loss 5.87174901 - samples/sec: 75.89 - lr: 0.500000 +2021-03-26 06:15:28,997 epoch 15 - iter 30/50 - loss 5.93718410 - samples/sec: 76.97 - lr: 0.500000 +2021-03-26 06:15:30,898 epoch 15 - iter 35/50 - loss 5.89442012 - samples/sec: 84.27 - lr: 0.500000 +2021-03-26 06:15:33,130 epoch 15 - iter 40/50 - loss 6.00830467 - samples/sec: 71.72 - lr: 0.500000 +2021-03-26 06:15:35,087 epoch 15 - iter 45/50 - loss 6.02102583 - samples/sec: 81.85 - lr: 0.500000 +2021-03-26 06:15:37,009 epoch 15 - iter 50/50 - loss 6.10250228 - samples/sec: 83.35 - lr: 0.500000 +2021-03-26 06:15:37,010 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:15:37,010 EPOCH 15 done: loss 6.1025 - lr 0.5000000 +2021-03-26 06:15:37,834 DEV : loss 7.358739852905273 - score 0.8742 +2021-03-26 06:15:37,860 BAD EPOCHS (no improvement): 1 +2021-03-26 06:15:37,861 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:15:39,834 epoch 16 - iter 5/50 - loss 6.96504974 - samples/sec: 81.15 - lr: 0.500000 +2021-03-26 06:15:41,884 epoch 16 - iter 10/50 - loss 6.91026793 - samples/sec: 78.16 - lr: 0.500000 +2021-03-26 06:15:44,050 epoch 16 - iter 15/50 - loss 6.68998731 - samples/sec: 73.95 - lr: 0.500000 +2021-03-26 06:15:45,978 epoch 16 - iter 20/50 - loss 6.60481675 - samples/sec: 83.05 - lr: 0.500000 +2021-03-26 06:15:47,988 epoch 16 - iter 25/50 - loss 6.48665789 - samples/sec: 79.68 - lr: 0.500000 +2021-03-26 06:15:50,019 epoch 16 - iter 30/50 - loss 6.29562639 - samples/sec: 78.88 - lr: 0.500000 +2021-03-26 06:15:51,866 epoch 16 - iter 35/50 - loss 6.23206079 - samples/sec: 86.71 - lr: 0.500000 +2021-03-26 06:15:53,828 epoch 16 - iter 40/50 - loss 6.17947595 - samples/sec: 81.64 - lr: 0.500000 +2021-03-26 06:15:55,798 epoch 16 - iter 45/50 - loss 6.15443833 - samples/sec: 81.29 - lr: 0.500000 +2021-03-26 06:15:57,465 epoch 16 - iter 50/50 - loss 6.11608263 - samples/sec: 96.17 - lr: 0.500000 +2021-03-26 06:15:57,465 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:15:57,466 EPOCH 16 done: loss 6.1161 - lr 0.5000000 +2021-03-26 06:15:58,289 DEV : loss 7.640191555023193 - score 0.875 +2021-03-26 06:15:58,318 BAD EPOCHS (no improvement): 2 +2021-03-26 06:15:58,319 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:16:00,374 epoch 17 - iter 5/50 - loss 5.76472216 - samples/sec: 77.91 - lr: 0.500000 +2021-03-26 06:16:02,165 epoch 17 - iter 10/50 - loss 5.60648484 - samples/sec: 89.45 - lr: 0.500000 +2021-03-26 06:16:04,243 epoch 17 - iter 15/50 - loss 5.77117564 - samples/sec: 77.05 - lr: 0.500000 +2021-03-26 06:16:06,119 epoch 17 - iter 20/50 - loss 5.80412705 - samples/sec: 85.35 - lr: 0.500000 +2021-03-26 06:16:08,090 epoch 17 - iter 25/50 - loss 5.77925716 - samples/sec: 81.25 - lr: 0.500000 +2021-03-26 06:16:10,089 epoch 17 - iter 30/50 - loss 5.70927278 - samples/sec: 80.13 - lr: 0.500000 +2021-03-26 06:16:12,107 epoch 17 - iter 35/50 - loss 5.70613817 - samples/sec: 79.39 - lr: 0.500000 +2021-03-26 06:16:14,074 epoch 17 - iter 40/50 - loss 5.77921520 - samples/sec: 81.41 - lr: 0.500000 +2021-03-26 06:16:16,031 epoch 17 - iter 45/50 - loss 5.70467767 - samples/sec: 81.84 - lr: 0.500000 +2021-03-26 06:16:18,138 epoch 17 - iter 50/50 - loss 5.87843167 - samples/sec: 76.00 - lr: 0.500000 +2021-03-26 06:16:18,139 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:16:18,140 EPOCH 17 done: loss 5.8784 - lr 0.5000000 +2021-03-26 06:16:19,033 DEV : loss 7.120694637298584 - score 0.8817 +2021-03-26 06:16:19,061 BAD EPOCHS (no improvement): 3 +2021-03-26 06:16:19,062 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:16:21,362 epoch 18 - iter 5/50 - loss 5.80058298 - samples/sec: 69.63 - lr: 0.500000 +2021-03-26 06:16:23,438 epoch 18 - iter 10/50 - loss 6.00460644 - samples/sec: 77.15 - lr: 0.500000 +2021-03-26 06:16:25,358 epoch 18 - iter 15/50 - loss 5.90615212 - samples/sec: 83.43 - lr: 0.500000 +2021-03-26 06:16:27,357 epoch 18 - iter 20/50 - loss 5.72014546 - samples/sec: 80.12 - lr: 0.500000 +2021-03-26 06:16:29,346 epoch 18 - iter 25/50 - loss 5.80794409 - samples/sec: 80.52 - lr: 0.500000 +2021-03-26 06:16:31,315 epoch 18 - iter 30/50 - loss 5.80667328 - samples/sec: 81.37 - lr: 0.500000 +2021-03-26 06:16:33,298 epoch 18 - iter 35/50 - loss 5.76956917 - samples/sec: 80.78 - lr: 0.500000 +2021-03-26 06:16:35,522 epoch 18 - iter 40/50 - loss 5.68624325 - samples/sec: 72.06 - lr: 0.500000 +2021-03-26 06:16:37,455 epoch 18 - iter 45/50 - loss 5.70470218 - samples/sec: 82.87 - lr: 0.500000 +2021-03-26 06:16:39,290 epoch 18 - iter 50/50 - loss 5.82466750 - samples/sec: 87.27 - lr: 0.500000 +2021-03-26 06:16:39,292 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:16:39,292 EPOCH 18 done: loss 5.8247 - lr 0.5000000 +2021-03-26 06:16:40,114 DEV : loss 7.067171096801758 - score 0.8886 +2021-03-26 06:16:40,141 BAD EPOCHS (no improvement): 0 +2021-03-26 06:16:49,832 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:16:51,943 epoch 19 - iter 5/50 - loss 5.32156200 - samples/sec: 75.90 - lr: 0.500000 +2021-03-26 06:16:53,868 epoch 19 - iter 10/50 - loss 4.92078037 - samples/sec: 83.24 - lr: 0.500000 +2021-03-26 06:16:55,873 epoch 19 - iter 15/50 - loss 5.07983332 - samples/sec: 79.84 - lr: 0.500000 +2021-03-26 06:16:57,998 epoch 19 - iter 20/50 - loss 5.24572079 - samples/sec: 75.36 - lr: 0.500000 +2021-03-26 06:17:00,367 epoch 19 - iter 25/50 - loss 5.34888216 - samples/sec: 67.61 - lr: 0.500000 +2021-03-26 06:17:02,223 epoch 19 - iter 30/50 - loss 5.33492244 - samples/sec: 86.28 - lr: 0.500000 +2021-03-26 06:17:04,089 epoch 19 - iter 35/50 - loss 5.38144553 - samples/sec: 85.85 - lr: 0.500000 +2021-03-26 06:17:06,062 epoch 19 - iter 40/50 - loss 5.36061793 - samples/sec: 81.15 - lr: 0.500000 +2021-03-26 06:17:08,112 epoch 19 - iter 45/50 - loss 5.48110933 - samples/sec: 78.14 - lr: 0.500000 +2021-03-26 06:17:09,926 epoch 19 - iter 50/50 - loss 5.53508369 - samples/sec: 88.30 - lr: 0.500000 +2021-03-26 06:17:09,927 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:17:09,927 EPOCH 19 done: loss 5.5351 - lr 0.5000000 +2021-03-26 06:17:10,741 DEV : loss 7.041009902954102 - score 0.8852 +2021-03-26 06:17:10,766 BAD EPOCHS (no improvement): 1 +2021-03-26 06:17:10,767 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:17:12,781 epoch 20 - iter 5/50 - loss 5.35584059 - samples/sec: 79.52 - lr: 0.500000 +2021-03-26 06:17:14,751 epoch 20 - iter 10/50 - loss 4.90886288 - samples/sec: 81.29 - lr: 0.500000 +2021-03-26 06:17:16,668 epoch 20 - iter 15/50 - loss 4.89809780 - samples/sec: 83.53 - lr: 0.500000 +2021-03-26 06:17:18,533 epoch 20 - iter 20/50 - loss 5.13916326 - samples/sec: 85.85 - lr: 0.500000 +2021-03-26 06:17:20,464 epoch 20 - iter 25/50 - loss 5.10323879 - samples/sec: 82.96 - lr: 0.500000 +2021-03-26 06:17:22,463 epoch 20 - iter 30/50 - loss 5.10014350 - samples/sec: 80.09 - lr: 0.500000 +2021-03-26 06:17:24,533 epoch 20 - iter 35/50 - loss 5.20402196 - samples/sec: 77.37 - lr: 0.500000 +2021-03-26 06:17:26,636 epoch 20 - iter 40/50 - loss 5.37771060 - samples/sec: 76.18 - lr: 0.500000 +2021-03-26 06:17:28,694 epoch 20 - iter 45/50 - loss 5.33990641 - samples/sec: 77.84 - lr: 0.500000 +2021-03-26 06:17:30,776 epoch 20 - iter 50/50 - loss 5.52597651 - samples/sec: 76.91 - lr: 0.500000 +2021-03-26 06:17:30,777 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:17:30,777 EPOCH 20 done: loss 5.5260 - lr 0.5000000 +2021-03-26 06:17:31,645 DEV : loss 7.155633926391602 - score 0.8897 +2021-03-26 06:17:31,672 BAD EPOCHS (no improvement): 0 +2021-03-26 06:17:41,442 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:17:43,438 epoch 21 - iter 5/50 - loss 4.66815495 - samples/sec: 80.24 - lr: 0.500000 +2021-03-26 06:17:45,394 epoch 21 - iter 10/50 - loss 5.02459919 - samples/sec: 81.88 - lr: 0.500000 +2021-03-26 06:17:47,435 epoch 21 - iter 15/50 - loss 5.26974597 - samples/sec: 78.50 - lr: 0.500000 +2021-03-26 06:17:49,541 epoch 21 - iter 20/50 - loss 5.35654365 - samples/sec: 76.02 - lr: 0.500000 +2021-03-26 06:17:51,710 epoch 21 - iter 25/50 - loss 5.32445769 - samples/sec: 73.84 - lr: 0.500000 +2021-03-26 06:17:53,601 epoch 21 - iter 30/50 - loss 5.29939301 - samples/sec: 84.68 - lr: 0.500000 +2021-03-26 06:17:55,610 epoch 21 - iter 35/50 - loss 5.35172689 - samples/sec: 79.74 - lr: 0.500000 +2021-03-26 06:17:57,641 epoch 21 - iter 40/50 - loss 5.39133795 - samples/sec: 78.84 - lr: 0.500000 +2021-03-26 06:17:59,650 epoch 21 - iter 45/50 - loss 5.44440748 - samples/sec: 79.74 - lr: 0.500000 +2021-03-26 06:18:01,498 epoch 21 - iter 50/50 - loss 5.35541382 - samples/sec: 86.73 - lr: 0.500000 +2021-03-26 06:18:01,499 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:18:01,499 EPOCH 21 done: loss 5.3554 - lr 0.5000000 +2021-03-26 06:18:02,315 DEV : loss 7.129164695739746 - score 0.8845 +2021-03-26 06:18:02,341 BAD EPOCHS (no improvement): 1 +2021-03-26 06:18:02,342 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:18:04,458 epoch 22 - iter 5/50 - loss 4.22467690 - samples/sec: 75.67 - lr: 0.500000 +2021-03-26 06:18:06,354 epoch 22 - iter 10/50 - loss 4.66893404 - samples/sec: 84.46 - lr: 0.500000 +2021-03-26 06:18:08,340 epoch 22 - iter 15/50 - loss 4.67403258 - samples/sec: 80.68 - lr: 0.500000 +2021-03-26 06:18:10,377 epoch 22 - iter 20/50 - loss 4.74228729 - samples/sec: 78.59 - lr: 0.500000 +2021-03-26 06:18:12,309 epoch 22 - iter 25/50 - loss 4.84663138 - samples/sec: 82.90 - lr: 0.500000 +2021-03-26 06:18:14,283 epoch 22 - iter 30/50 - loss 4.83845309 - samples/sec: 81.16 - lr: 0.500000 +2021-03-26 06:18:16,352 epoch 22 - iter 35/50 - loss 4.90364776 - samples/sec: 77.40 - lr: 0.500000 +2021-03-26 06:18:18,535 epoch 22 - iter 40/50 - loss 4.99079722 - samples/sec: 73.39 - lr: 0.500000 +2021-03-26 06:18:20,558 epoch 22 - iter 45/50 - loss 5.00144645 - samples/sec: 79.18 - lr: 0.500000 +2021-03-26 06:18:22,538 epoch 22 - iter 50/50 - loss 5.00026049 - samples/sec: 80.89 - lr: 0.500000 +2021-03-26 06:18:22,539 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:18:22,539 EPOCH 22 done: loss 5.0003 - lr 0.5000000 +2021-03-26 06:18:23,358 DEV : loss 7.209229946136475 - score 0.892 +2021-03-26 06:18:23,376 BAD EPOCHS (no improvement): 0 +2021-03-26 06:18:33,094 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:18:35,157 epoch 23 - iter 5/50 - loss 4.42421436 - samples/sec: 77.67 - lr: 0.500000 +2021-03-26 06:18:37,257 epoch 23 - iter 10/50 - loss 4.58858795 - samples/sec: 76.24 - lr: 0.500000 +2021-03-26 06:18:39,192 epoch 23 - iter 15/50 - loss 4.68085804 - samples/sec: 82.79 - lr: 0.500000 +2021-03-26 06:18:41,191 epoch 23 - iter 20/50 - loss 4.48246378 - samples/sec: 80.15 - lr: 0.500000 +2021-03-26 06:18:43,197 epoch 23 - iter 25/50 - loss 4.60831195 - samples/sec: 79.84 - lr: 0.500000 +2021-03-26 06:18:45,338 epoch 23 - iter 30/50 - loss 4.53582498 - samples/sec: 74.82 - lr: 0.500000 +2021-03-26 06:18:47,158 epoch 23 - iter 35/50 - loss 4.60754177 - samples/sec: 87.95 - lr: 0.500000 +2021-03-26 06:18:49,133 epoch 23 - iter 40/50 - loss 4.75192796 - samples/sec: 81.14 - lr: 0.500000 +2021-03-26 06:18:51,214 epoch 23 - iter 45/50 - loss 4.73687989 - samples/sec: 76.95 - lr: 0.500000 +2021-03-26 06:18:53,076 epoch 23 - iter 50/50 - loss 4.76302664 - samples/sec: 85.98 - lr: 0.500000 +2021-03-26 06:18:53,077 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:18:53,077 EPOCH 23 done: loss 4.7630 - lr 0.5000000 +2021-03-26 06:18:53,928 DEV : loss 7.368309020996094 - score 0.8877 +2021-03-26 06:18:53,958 BAD EPOCHS (no improvement): 1 +2021-03-26 06:18:53,959 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:18:55,819 epoch 24 - iter 5/50 - loss 4.53367014 - samples/sec: 86.10 - lr: 0.500000 +2021-03-26 06:18:57,857 epoch 24 - iter 10/50 - loss 4.77556491 - samples/sec: 78.59 - lr: 0.500000 +2021-03-26 06:18:59,726 epoch 24 - iter 15/50 - loss 4.68768574 - samples/sec: 85.68 - lr: 0.500000 +2021-03-26 06:19:01,703 epoch 24 - iter 20/50 - loss 4.69659667 - samples/sec: 81.03 - lr: 0.500000 +2021-03-26 06:19:03,714 epoch 24 - iter 25/50 - loss 4.78619684 - samples/sec: 79.62 - lr: 0.500000 +2021-03-26 06:19:05,530 epoch 24 - iter 30/50 - loss 4.71550113 - samples/sec: 88.19 - lr: 0.500000 +2021-03-26 06:19:07,538 epoch 24 - iter 35/50 - loss 4.63759924 - samples/sec: 79.79 - lr: 0.500000 +2021-03-26 06:19:09,731 epoch 24 - iter 40/50 - loss 4.67768221 - samples/sec: 73.00 - lr: 0.500000 +2021-03-26 06:19:11,687 epoch 24 - iter 45/50 - loss 4.77335002 - samples/sec: 81.89 - lr: 0.500000 +2021-03-26 06:19:13,645 epoch 24 - iter 50/50 - loss 4.70509425 - samples/sec: 81.77 - lr: 0.500000 +2021-03-26 06:19:13,646 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:19:13,646 EPOCH 24 done: loss 4.7051 - lr 0.5000000 +2021-03-26 06:19:14,450 DEV : loss 7.073756217956543 - score 0.8827 +2021-03-26 06:19:14,471 BAD EPOCHS (no improvement): 2 +2021-03-26 06:19:14,471 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:19:16,465 epoch 25 - iter 5/50 - loss 5.02869673 - samples/sec: 80.30 - lr: 0.500000 +2021-03-26 06:19:18,430 epoch 25 - iter 10/50 - loss 4.77017746 - samples/sec: 81.52 - lr: 0.500000 +2021-03-26 06:19:20,331 epoch 25 - iter 15/50 - loss 4.87391117 - samples/sec: 84.29 - lr: 0.500000 +2021-03-26 06:19:22,391 epoch 25 - iter 20/50 - loss 4.88199726 - samples/sec: 77.76 - lr: 0.500000 +2021-03-26 06:19:24,358 epoch 25 - iter 25/50 - loss 4.92657302 - samples/sec: 81.38 - lr: 0.500000 +2021-03-26 06:19:26,455 epoch 25 - iter 30/50 - loss 4.89457783 - samples/sec: 76.42 - lr: 0.500000 +2021-03-26 06:19:28,517 epoch 25 - iter 35/50 - loss 4.78275283 - samples/sec: 77.64 - lr: 0.500000 +2021-03-26 06:19:30,658 epoch 25 - iter 40/50 - loss 4.82162649 - samples/sec: 74.81 - lr: 0.500000 +2021-03-26 06:19:32,627 epoch 25 - iter 45/50 - loss 4.90526964 - samples/sec: 81.31 - lr: 0.500000 +2021-03-26 06:19:34,235 epoch 25 - iter 50/50 - loss 4.78762905 - samples/sec: 99.59 - lr: 0.500000 +2021-03-26 06:19:34,236 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:19:34,236 EPOCH 25 done: loss 4.7876 - lr 0.5000000 +2021-03-26 06:19:35,034 DEV : loss 7.848437786102295 - score 0.8751 +2021-03-26 06:19:35,059 BAD EPOCHS (no improvement): 3 +2021-03-26 06:19:35,060 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:19:37,013 epoch 26 - iter 5/50 - loss 4.39827414 - samples/sec: 82.02 - lr: 0.500000 +2021-03-26 06:19:38,978 epoch 26 - iter 10/50 - loss 4.59327972 - samples/sec: 81.55 - lr: 0.500000 +2021-03-26 06:19:40,939 epoch 26 - iter 15/50 - loss 4.79339280 - samples/sec: 81.68 - lr: 0.500000 +2021-03-26 06:19:42,973 epoch 26 - iter 20/50 - loss 4.70258976 - samples/sec: 78.72 - lr: 0.500000 +2021-03-26 06:19:45,190 epoch 26 - iter 25/50 - loss 4.63147681 - samples/sec: 72.22 - lr: 0.500000 +2021-03-26 06:19:47,322 epoch 26 - iter 30/50 - loss 4.65599515 - samples/sec: 75.09 - lr: 0.500000 +2021-03-26 06:19:49,139 epoch 26 - iter 35/50 - loss 4.64802615 - samples/sec: 88.17 - lr: 0.500000 +2021-03-26 06:19:50,954 epoch 26 - iter 40/50 - loss 4.52358636 - samples/sec: 88.23 - lr: 0.500000 +2021-03-26 06:19:53,038 epoch 26 - iter 45/50 - loss 4.53593655 - samples/sec: 76.85 - lr: 0.500000 +2021-03-26 06:19:54,966 epoch 26 - iter 50/50 - loss 4.52930590 - samples/sec: 83.07 - lr: 0.500000 +2021-03-26 06:19:54,966 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:19:54,966 EPOCH 26 done: loss 4.5293 - lr 0.5000000 +2021-03-26 06:19:55,779 DEV : loss 7.53148078918457 - score 0.8884 +2021-03-26 06:19:55,804 BAD EPOCHS (no improvement): 4 +2021-03-26 06:19:55,805 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:19:57,707 epoch 27 - iter 5/50 - loss 3.82686830 - samples/sec: 84.19 - lr: 0.250000 +2021-03-26 06:19:59,679 epoch 27 - iter 10/50 - loss 3.90787556 - samples/sec: 81.26 - lr: 0.250000 +2021-03-26 06:20:01,702 epoch 27 - iter 15/50 - loss 3.89724426 - samples/sec: 79.15 - lr: 0.250000 +2021-03-26 06:20:03,651 epoch 27 - iter 20/50 - loss 4.00114959 - samples/sec: 82.24 - lr: 0.250000 +2021-03-26 06:20:05,654 epoch 27 - iter 25/50 - loss 3.96433598 - samples/sec: 79.97 - lr: 0.250000 +2021-03-26 06:20:07,642 epoch 27 - iter 30/50 - loss 4.10539275 - samples/sec: 80.54 - lr: 0.250000 +2021-03-26 06:20:09,525 epoch 27 - iter 35/50 - loss 4.07188780 - samples/sec: 85.05 - lr: 0.250000 +2021-03-26 06:20:11,589 epoch 27 - iter 40/50 - loss 4.01781631 - samples/sec: 77.60 - lr: 0.250000 +2021-03-26 06:20:13,728 epoch 27 - iter 45/50 - loss 3.93384314 - samples/sec: 74.89 - lr: 0.250000 +2021-03-26 06:20:15,676 epoch 27 - iter 50/50 - loss 3.94948688 - samples/sec: 82.18 - lr: 0.250000 +2021-03-26 06:20:15,677 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:20:15,677 EPOCH 27 done: loss 3.9495 - lr 0.2500000 +2021-03-26 06:20:16,497 DEV : loss 6.849296569824219 - score 0.8994 +2021-03-26 06:20:16,524 BAD EPOCHS (no improvement): 0 +2021-03-26 06:20:26,002 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:20:29,219 epoch 28 - iter 5/50 - loss 3.81995850 - samples/sec: 49.78 - lr: 0.250000 +2021-03-26 06:20:31,150 epoch 28 - iter 10/50 - loss 4.01414056 - samples/sec: 82.96 - lr: 0.250000 +2021-03-26 06:20:33,235 epoch 28 - iter 15/50 - loss 4.01229688 - samples/sec: 76.80 - lr: 0.250000 +2021-03-26 06:20:35,220 epoch 28 - iter 20/50 - loss 3.84462682 - samples/sec: 80.66 - lr: 0.250000 +2021-03-26 06:20:37,200 epoch 28 - iter 25/50 - loss 3.89083829 - samples/sec: 80.96 - lr: 0.250000 +2021-03-26 06:20:39,270 epoch 28 - iter 30/50 - loss 3.85976992 - samples/sec: 77.36 - lr: 0.250000 +2021-03-26 06:20:41,560 epoch 28 - iter 35/50 - loss 3.85642435 - samples/sec: 69.94 - lr: 0.250000 +2021-03-26 06:20:43,739 epoch 28 - iter 40/50 - loss 3.84121451 - samples/sec: 73.48 - lr: 0.250000 +2021-03-26 06:20:45,542 epoch 28 - iter 45/50 - loss 3.83978069 - samples/sec: 88.87 - lr: 0.250000 +2021-03-26 06:20:47,391 epoch 28 - iter 50/50 - loss 3.77387645 - samples/sec: 86.65 - lr: 0.250000 +2021-03-26 06:20:47,391 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:20:47,392 EPOCH 28 done: loss 3.7739 - lr 0.2500000 +2021-03-26 06:20:48,184 DEV : loss 6.964501857757568 - score 0.9041 +2021-03-26 06:20:48,209 BAD EPOCHS (no improvement): 0 +2021-03-26 06:20:57,673 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:21:00,127 epoch 29 - iter 5/50 - loss 4.25758047 - samples/sec: 65.28 - lr: 0.250000 +2021-03-26 06:21:01,940 epoch 29 - iter 10/50 - loss 4.10903742 - samples/sec: 88.34 - lr: 0.250000 +2021-03-26 06:21:03,888 epoch 29 - iter 15/50 - loss 4.05873850 - samples/sec: 82.23 - lr: 0.250000 +2021-03-26 06:21:05,746 epoch 29 - iter 20/50 - loss 4.01524862 - samples/sec: 86.22 - lr: 0.250000 +2021-03-26 06:21:07,887 epoch 29 - iter 25/50 - loss 3.92797098 - samples/sec: 74.81 - lr: 0.250000 +2021-03-26 06:21:09,833 epoch 29 - iter 30/50 - loss 3.90643357 - samples/sec: 82.32 - lr: 0.250000 +2021-03-26 06:21:11,857 epoch 29 - iter 35/50 - loss 3.89231799 - samples/sec: 79.14 - lr: 0.250000 +2021-03-26 06:21:13,967 epoch 29 - iter 40/50 - loss 3.86935532 - samples/sec: 75.93 - lr: 0.250000 +2021-03-26 06:21:15,892 epoch 29 - iter 45/50 - loss 3.83151262 - samples/sec: 83.17 - lr: 0.250000 +2021-03-26 06:21:17,638 epoch 29 - iter 50/50 - loss 3.83023209 - samples/sec: 91.72 - lr: 0.250000 +2021-03-26 06:21:17,639 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:21:17,640 EPOCH 29 done: loss 3.8302 - lr 0.2500000 +2021-03-26 06:21:18,435 DEV : loss 6.683016777038574 - score 0.9014 +2021-03-26 06:21:18,460 BAD EPOCHS (no improvement): 1 +2021-03-26 06:21:18,461 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:21:20,384 epoch 30 - iter 5/50 - loss 3.60974512 - samples/sec: 83.31 - lr: 0.250000 +2021-03-26 06:21:22,259 epoch 30 - iter 10/50 - loss 3.61541648 - samples/sec: 85.42 - lr: 0.250000 +2021-03-26 06:21:24,185 epoch 30 - iter 15/50 - loss 3.51423572 - samples/sec: 83.19 - lr: 0.250000 +2021-03-26 06:21:26,347 epoch 30 - iter 20/50 - loss 3.53319275 - samples/sec: 74.10 - lr: 0.250000 +2021-03-26 06:21:28,304 epoch 30 - iter 25/50 - loss 3.55055931 - samples/sec: 81.87 - lr: 0.250000 +2021-03-26 06:21:30,267 epoch 30 - iter 30/50 - loss 3.54994337 - samples/sec: 81.60 - lr: 0.250000 +2021-03-26 06:21:32,073 epoch 30 - iter 35/50 - loss 3.45288790 - samples/sec: 88.66 - lr: 0.250000 +2021-03-26 06:21:33,966 epoch 30 - iter 40/50 - loss 3.45501449 - samples/sec: 84.60 - lr: 0.250000 +2021-03-26 06:21:36,166 epoch 30 - iter 45/50 - loss 3.48634175 - samples/sec: 72.78 - lr: 0.250000 +2021-03-26 06:21:38,099 epoch 30 - iter 50/50 - loss 3.59221848 - samples/sec: 82.87 - lr: 0.250000 +2021-03-26 06:21:38,100 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:21:38,100 EPOCH 30 done: loss 3.5922 - lr 0.2500000 +2021-03-26 06:21:38,892 DEV : loss 6.731375694274902 - score 0.9015 +2021-03-26 06:21:38,917 BAD EPOCHS (no improvement): 2 +2021-03-26 06:21:38,918 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:21:40,960 epoch 31 - iter 5/50 - loss 4.21938186 - samples/sec: 78.45 - lr: 0.250000 +2021-03-26 06:21:42,822 epoch 31 - iter 10/50 - loss 3.82266951 - samples/sec: 86.03 - lr: 0.250000 +2021-03-26 06:21:44,824 epoch 31 - iter 15/50 - loss 3.58426552 - samples/sec: 80.01 - lr: 0.250000 +2021-03-26 06:21:46,855 epoch 31 - iter 20/50 - loss 3.58201323 - samples/sec: 78.85 - lr: 0.250000 +2021-03-26 06:21:48,698 epoch 31 - iter 25/50 - loss 3.57560792 - samples/sec: 86.89 - lr: 0.250000 +2021-03-26 06:21:50,670 epoch 31 - iter 30/50 - loss 3.42569039 - samples/sec: 81.23 - lr: 0.250000 +2021-03-26 06:21:52,689 epoch 31 - iter 35/50 - loss 3.49048048 - samples/sec: 79.30 - lr: 0.250000 +2021-03-26 06:21:54,700 epoch 31 - iter 40/50 - loss 3.50268674 - samples/sec: 79.63 - lr: 0.250000 +2021-03-26 06:21:56,692 epoch 31 - iter 45/50 - loss 3.46837054 - samples/sec: 80.38 - lr: 0.250000 +2021-03-26 06:21:58,607 epoch 31 - iter 50/50 - loss 3.57005912 - samples/sec: 83.62 - lr: 0.250000 +2021-03-26 06:21:58,608 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:21:58,608 EPOCH 31 done: loss 3.5701 - lr 0.2500000 +2021-03-26 06:21:59,398 DEV : loss 6.836461067199707 - score 0.9019 +2021-03-26 06:21:59,424 BAD EPOCHS (no improvement): 3 +2021-03-26 06:21:59,425 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:22:01,484 epoch 32 - iter 5/50 - loss 2.85547342 - samples/sec: 77.75 - lr: 0.250000 +2021-03-26 06:22:03,524 epoch 32 - iter 10/50 - loss 3.03122098 - samples/sec: 78.51 - lr: 0.250000 +2021-03-26 06:22:05,581 epoch 32 - iter 15/50 - loss 3.22923800 - samples/sec: 77.86 - lr: 0.250000 +2021-03-26 06:22:07,481 epoch 32 - iter 20/50 - loss 3.31757702 - samples/sec: 84.29 - lr: 0.250000 +2021-03-26 06:22:09,504 epoch 32 - iter 25/50 - loss 3.27683636 - samples/sec: 79.16 - lr: 0.250000 +2021-03-26 06:22:11,483 epoch 32 - iter 30/50 - loss 3.24276355 - samples/sec: 80.96 - lr: 0.250000 +2021-03-26 06:22:13,444 epoch 32 - iter 35/50 - loss 3.23634225 - samples/sec: 81.68 - lr: 0.250000 +2021-03-26 06:22:15,387 epoch 32 - iter 40/50 - loss 3.20656458 - samples/sec: 82.46 - lr: 0.250000 +2021-03-26 06:22:17,206 epoch 32 - iter 45/50 - loss 3.18769871 - samples/sec: 88.05 - lr: 0.250000 +2021-03-26 06:22:19,018 epoch 32 - iter 50/50 - loss 3.25717090 - samples/sec: 88.34 - lr: 0.250000 +2021-03-26 06:22:19,019 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:22:19,020 EPOCH 32 done: loss 3.2572 - lr 0.2500000 +2021-03-26 06:22:19,845 DEV : loss 7.020652770996094 - score 0.8992 +2021-03-26 06:22:19,869 BAD EPOCHS (no improvement): 4 +2021-03-26 06:22:19,869 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:22:21,939 epoch 33 - iter 5/50 - loss 3.13701382 - samples/sec: 77.39 - lr: 0.125000 +2021-03-26 06:22:23,942 epoch 33 - iter 10/50 - loss 2.86191452 - samples/sec: 79.96 - lr: 0.125000 +2021-03-26 06:22:25,922 epoch 33 - iter 15/50 - loss 3.00424566 - samples/sec: 80.87 - lr: 0.125000 +2021-03-26 06:22:27,923 epoch 33 - iter 20/50 - loss 3.03444778 - samples/sec: 80.06 - lr: 0.125000 +2021-03-26 06:22:29,911 epoch 33 - iter 25/50 - loss 3.12197756 - samples/sec: 80.55 - lr: 0.125000 +2021-03-26 06:22:31,898 epoch 33 - iter 30/50 - loss 3.03927702 - samples/sec: 80.61 - lr: 0.125000 +2021-03-26 06:22:33,980 epoch 33 - iter 35/50 - loss 3.08446097 - samples/sec: 76.89 - lr: 0.125000 +2021-03-26 06:22:36,147 epoch 33 - iter 40/50 - loss 3.05413010 - samples/sec: 73.91 - lr: 0.125000 +2021-03-26 06:22:38,285 epoch 33 - iter 45/50 - loss 3.09317961 - samples/sec: 74.93 - lr: 0.125000 +2021-03-26 06:22:40,047 epoch 33 - iter 50/50 - loss 3.13532100 - samples/sec: 90.97 - lr: 0.125000 +2021-03-26 06:22:40,048 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:22:40,048 EPOCH 33 done: loss 3.1353 - lr 0.1250000 +2021-03-26 06:22:40,857 DEV : loss 6.7104997634887695 - score 0.9059 +2021-03-26 06:22:40,875 BAD EPOCHS (no improvement): 0 +2021-03-26 06:22:50,423 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:22:52,500 epoch 34 - iter 5/50 - loss 2.97224755 - samples/sec: 77.16 - lr: 0.125000 +2021-03-26 06:22:54,279 epoch 34 - iter 10/50 - loss 2.79403913 - samples/sec: 90.02 - lr: 0.125000 +2021-03-26 06:22:56,702 epoch 34 - iter 15/50 - loss 2.96103163 - samples/sec: 66.08 - lr: 0.125000 +2021-03-26 06:22:58,558 epoch 34 - iter 20/50 - loss 3.00038437 - samples/sec: 86.35 - lr: 0.125000 +2021-03-26 06:23:00,439 epoch 34 - iter 25/50 - loss 3.00155699 - samples/sec: 85.17 - lr: 0.125000 +2021-03-26 06:23:02,343 epoch 34 - iter 30/50 - loss 3.01357543 - samples/sec: 84.10 - lr: 0.125000 +2021-03-26 06:23:04,427 epoch 34 - iter 35/50 - loss 3.05171795 - samples/sec: 76.86 - lr: 0.125000 +2021-03-26 06:23:06,363 epoch 34 - iter 40/50 - loss 3.04145568 - samples/sec: 82.73 - lr: 0.125000 +2021-03-26 06:23:08,553 epoch 34 - iter 45/50 - loss 3.06644186 - samples/sec: 73.15 - lr: 0.125000 +2021-03-26 06:23:10,411 epoch 34 - iter 50/50 - loss 3.18904578 - samples/sec: 86.19 - lr: 0.125000 +2021-03-26 06:23:10,412 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:23:10,412 EPOCH 34 done: loss 3.1890 - lr 0.1250000 +2021-03-26 06:23:11,228 DEV : loss 6.743964195251465 - score 0.9039 +2021-03-26 06:23:11,254 BAD EPOCHS (no improvement): 1 +2021-03-26 06:23:11,255 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:23:13,323 epoch 35 - iter 5/50 - loss 3.39564099 - samples/sec: 77.45 - lr: 0.125000 +2021-03-26 06:23:15,123 epoch 35 - iter 10/50 - loss 3.06941562 - samples/sec: 89.01 - lr: 0.125000 +2021-03-26 06:23:17,156 epoch 35 - iter 15/50 - loss 3.02297851 - samples/sec: 78.77 - lr: 0.125000 +2021-03-26 06:23:19,140 epoch 35 - iter 20/50 - loss 3.13186671 - samples/sec: 80.74 - lr: 0.125000 +2021-03-26 06:23:21,073 epoch 35 - iter 25/50 - loss 3.11243931 - samples/sec: 82.83 - lr: 0.125000 +2021-03-26 06:23:23,244 epoch 35 - iter 30/50 - loss 3.08870709 - samples/sec: 73.78 - lr: 0.125000 +2021-03-26 06:23:25,385 epoch 35 - iter 35/50 - loss 3.09268135 - samples/sec: 74.78 - lr: 0.125000 +2021-03-26 06:23:27,392 epoch 35 - iter 40/50 - loss 3.08425107 - samples/sec: 79.80 - lr: 0.125000 +2021-03-26 06:23:29,379 epoch 35 - iter 45/50 - loss 3.03528121 - samples/sec: 80.58 - lr: 0.125000 +2021-03-26 06:23:31,335 epoch 35 - iter 50/50 - loss 3.02395340 - samples/sec: 81.91 - lr: 0.125000 +2021-03-26 06:23:31,336 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:23:31,336 EPOCH 35 done: loss 3.0240 - lr 0.1250000 +2021-03-26 06:23:32,181 DEV : loss 6.659381866455078 - score 0.9053 +2021-03-26 06:23:32,209 BAD EPOCHS (no improvement): 2 +2021-03-26 06:23:32,210 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:23:34,186 epoch 36 - iter 5/50 - loss 2.70919876 - samples/sec: 81.04 - lr: 0.125000 +2021-03-26 06:23:36,188 epoch 36 - iter 10/50 - loss 2.72161946 - samples/sec: 80.02 - lr: 0.125000 +2021-03-26 06:23:38,140 epoch 36 - iter 15/50 - loss 2.75374413 - samples/sec: 82.04 - lr: 0.125000 +2021-03-26 06:23:40,098 epoch 36 - iter 20/50 - loss 2.82946196 - samples/sec: 81.84 - lr: 0.125000 +2021-03-26 06:23:42,028 epoch 36 - iter 25/50 - loss 2.80491947 - samples/sec: 83.00 - lr: 0.125000 +2021-03-26 06:23:43,997 epoch 36 - iter 30/50 - loss 2.75406922 - samples/sec: 81.31 - lr: 0.125000 +2021-03-26 06:23:46,008 epoch 36 - iter 35/50 - loss 2.81473930 - samples/sec: 79.66 - lr: 0.125000 +2021-03-26 06:23:47,999 epoch 36 - iter 40/50 - loss 2.88393461 - samples/sec: 80.43 - lr: 0.125000 +2021-03-26 06:23:50,307 epoch 36 - iter 45/50 - loss 2.86291863 - samples/sec: 69.38 - lr: 0.125000 +2021-03-26 06:23:52,128 epoch 36 - iter 50/50 - loss 2.95692794 - samples/sec: 87.99 - lr: 0.125000 +2021-03-26 06:23:52,129 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:23:52,129 EPOCH 36 done: loss 2.9569 - lr 0.1250000 +2021-03-26 06:23:52,924 DEV : loss 6.722310543060303 - score 0.9073 +2021-03-26 06:23:52,950 BAD EPOCHS (no improvement): 0 +2021-03-26 06:24:02,551 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:24:04,482 epoch 37 - iter 5/50 - loss 2.98329856 - samples/sec: 82.97 - lr: 0.125000 +2021-03-26 06:24:06,335 epoch 37 - iter 10/50 - loss 2.84934751 - samples/sec: 86.45 - lr: 0.125000 +2021-03-26 06:24:08,383 epoch 37 - iter 15/50 - loss 2.94346907 - samples/sec: 78.21 - lr: 0.125000 +2021-03-26 06:24:10,496 epoch 37 - iter 20/50 - loss 2.94718229 - samples/sec: 75.78 - lr: 0.125000 +2021-03-26 06:24:12,662 epoch 37 - iter 25/50 - loss 2.97769873 - samples/sec: 73.99 - lr: 0.125000 +2021-03-26 06:24:14,789 epoch 37 - iter 30/50 - loss 3.06307853 - samples/sec: 75.31 - lr: 0.125000 +2021-03-26 06:24:16,705 epoch 37 - iter 35/50 - loss 3.06100252 - samples/sec: 83.55 - lr: 0.125000 +2021-03-26 06:24:18,806 epoch 37 - iter 40/50 - loss 3.07404596 - samples/sec: 76.26 - lr: 0.125000 +2021-03-26 06:24:20,702 epoch 37 - iter 45/50 - loss 3.04176707 - samples/sec: 84.47 - lr: 0.125000 +2021-03-26 06:24:22,559 epoch 37 - iter 50/50 - loss 3.05250724 - samples/sec: 86.26 - lr: 0.125000 +2021-03-26 06:24:22,560 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:24:22,560 EPOCH 37 done: loss 3.0525 - lr 0.1250000 +2021-03-26 06:24:23,430 DEV : loss 6.977239608764648 - score 0.903 +2021-03-26 06:24:23,456 BAD EPOCHS (no improvement): 1 +2021-03-26 06:24:23,457 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:24:25,331 epoch 38 - iter 5/50 - loss 2.45115697 - samples/sec: 85.48 - lr: 0.125000 +2021-03-26 06:24:27,448 epoch 38 - iter 10/50 - loss 2.73839563 - samples/sec: 75.63 - lr: 0.125000 +2021-03-26 06:24:29,559 epoch 38 - iter 15/50 - loss 2.77460554 - samples/sec: 75.86 - lr: 0.125000 +2021-03-26 06:24:31,510 epoch 38 - iter 20/50 - loss 2.69243065 - samples/sec: 82.08 - lr: 0.125000 +2021-03-26 06:24:33,407 epoch 38 - iter 25/50 - loss 2.70140969 - samples/sec: 84.43 - lr: 0.125000 +2021-03-26 06:24:35,323 epoch 38 - iter 30/50 - loss 2.66803439 - samples/sec: 83.58 - lr: 0.125000 +2021-03-26 06:24:37,261 epoch 38 - iter 35/50 - loss 2.63252071 - samples/sec: 82.66 - lr: 0.125000 +2021-03-26 06:24:39,443 epoch 38 - iter 40/50 - loss 2.69146514 - samples/sec: 73.44 - lr: 0.125000 +2021-03-26 06:24:41,734 epoch 38 - iter 45/50 - loss 2.65392441 - samples/sec: 69.89 - lr: 0.125000 +2021-03-26 06:24:43,774 epoch 38 - iter 50/50 - loss 2.71148792 - samples/sec: 78.54 - lr: 0.125000 +2021-03-26 06:24:43,774 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:24:43,774 EPOCH 38 done: loss 2.7115 - lr 0.1250000 +2021-03-26 06:24:44,591 DEV : loss 6.88974666595459 - score 0.9069 +2021-03-26 06:24:44,620 BAD EPOCHS (no improvement): 2 +2021-03-26 06:24:44,620 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:24:46,729 epoch 39 - iter 5/50 - loss 2.89031749 - samples/sec: 75.95 - lr: 0.125000 +2021-03-26 06:24:48,776 epoch 39 - iter 10/50 - loss 2.72867922 - samples/sec: 78.26 - lr: 0.125000 +2021-03-26 06:24:50,761 epoch 39 - iter 15/50 - loss 2.58544503 - samples/sec: 80.68 - lr: 0.125000 +2021-03-26 06:24:52,832 epoch 39 - iter 20/50 - loss 2.64498028 - samples/sec: 77.36 - lr: 0.125000 +2021-03-26 06:24:54,870 epoch 39 - iter 25/50 - loss 2.71735142 - samples/sec: 78.61 - lr: 0.125000 +2021-03-26 06:24:56,906 epoch 39 - iter 30/50 - loss 2.69654935 - samples/sec: 78.66 - lr: 0.125000 +2021-03-26 06:24:58,840 epoch 39 - iter 35/50 - loss 2.77980161 - samples/sec: 82.82 - lr: 0.125000 +2021-03-26 06:25:00,885 epoch 39 - iter 40/50 - loss 2.74732492 - samples/sec: 78.30 - lr: 0.125000 +2021-03-26 06:25:02,936 epoch 39 - iter 45/50 - loss 2.75970213 - samples/sec: 78.09 - lr: 0.125000 +2021-03-26 06:25:04,820 epoch 39 - iter 50/50 - loss 2.73851163 - samples/sec: 85.00 - lr: 0.125000 +2021-03-26 06:25:04,821 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:25:04,821 EPOCH 39 done: loss 2.7385 - lr 0.1250000 +2021-03-26 06:25:05,634 DEV : loss 6.86600399017334 - score 0.9061 +2021-03-26 06:25:05,661 BAD EPOCHS (no improvement): 3 +2021-03-26 06:25:05,661 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:25:07,492 epoch 40 - iter 5/50 - loss 2.93947897 - samples/sec: 87.51 - lr: 0.125000 +2021-03-26 06:25:09,591 epoch 40 - iter 10/50 - loss 2.59734519 - samples/sec: 76.30 - lr: 0.125000 +2021-03-26 06:25:11,603 epoch 40 - iter 15/50 - loss 2.59909554 - samples/sec: 79.60 - lr: 0.125000 +2021-03-26 06:25:13,531 epoch 40 - iter 20/50 - loss 2.56581937 - samples/sec: 83.06 - lr: 0.125000 +2021-03-26 06:25:15,644 epoch 40 - iter 25/50 - loss 2.63403885 - samples/sec: 75.80 - lr: 0.125000 +2021-03-26 06:25:17,734 epoch 40 - iter 30/50 - loss 2.66231444 - samples/sec: 76.59 - lr: 0.125000 +2021-03-26 06:25:19,742 epoch 40 - iter 35/50 - loss 2.60746949 - samples/sec: 79.77 - lr: 0.125000 +2021-03-26 06:25:21,846 epoch 40 - iter 40/50 - loss 2.65097475 - samples/sec: 76.10 - lr: 0.125000 +2021-03-26 06:25:23,899 epoch 40 - iter 45/50 - loss 2.70838504 - samples/sec: 78.06 - lr: 0.125000 +2021-03-26 06:25:26,172 epoch 40 - iter 50/50 - loss 2.72248378 - samples/sec: 70.43 - lr: 0.125000 +2021-03-26 06:25:26,173 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:25:26,173 EPOCH 40 done: loss 2.7225 - lr 0.1250000 +2021-03-26 06:25:26,974 DEV : loss 7.042795181274414 - score 0.9061 +2021-03-26 06:25:26,998 BAD EPOCHS (no improvement): 4 +2021-03-26 06:25:26,999 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:25:28,891 epoch 41 - iter 5/50 - loss 3.03020132 - samples/sec: 84.66 - lr: 0.062500 +2021-03-26 06:25:30,996 epoch 41 - iter 10/50 - loss 2.92019740 - samples/sec: 76.07 - lr: 0.062500 +2021-03-26 06:25:33,009 epoch 41 - iter 15/50 - loss 2.80438588 - samples/sec: 79.56 - lr: 0.062500 +2021-03-26 06:25:34,912 epoch 41 - iter 20/50 - loss 2.79609125 - samples/sec: 84.20 - lr: 0.062500 +2021-03-26 06:25:36,831 epoch 41 - iter 25/50 - loss 2.66950620 - samples/sec: 83.48 - lr: 0.062500 +2021-03-26 06:25:38,766 epoch 41 - iter 30/50 - loss 2.64369078 - samples/sec: 82.75 - lr: 0.062500 +2021-03-26 06:25:40,669 epoch 41 - iter 35/50 - loss 2.71000017 - samples/sec: 84.16 - lr: 0.062500 +2021-03-26 06:25:42,605 epoch 41 - iter 40/50 - loss 2.67410662 - samples/sec: 82.70 - lr: 0.062500 +2021-03-26 06:25:44,568 epoch 41 - iter 45/50 - loss 2.72351690 - samples/sec: 81.61 - lr: 0.062500 +2021-03-26 06:25:46,331 epoch 41 - iter 50/50 - loss 2.68049579 - samples/sec: 90.80 - lr: 0.062500 +2021-03-26 06:25:46,332 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:25:46,333 EPOCH 41 done: loss 2.6805 - lr 0.0625000 +2021-03-26 06:25:47,172 DEV : loss 6.836457252502441 - score 0.9069 +2021-03-26 06:25:47,195 BAD EPOCHS (no improvement): 1 +2021-03-26 06:25:47,195 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:25:49,081 epoch 42 - iter 5/50 - loss 2.83079109 - samples/sec: 84.93 - lr: 0.062500 +2021-03-26 06:25:51,170 epoch 42 - iter 10/50 - loss 2.60190370 - samples/sec: 76.68 - lr: 0.062500 +2021-03-26 06:25:53,425 epoch 42 - iter 15/50 - loss 2.63134141 - samples/sec: 70.98 - lr: 0.062500 +2021-03-26 06:25:55,509 epoch 42 - iter 20/50 - loss 2.64654483 - samples/sec: 76.84 - lr: 0.062500 +2021-03-26 06:25:57,489 epoch 42 - iter 25/50 - loss 2.63535637 - samples/sec: 80.90 - lr: 0.062500 +2021-03-26 06:25:59,512 epoch 42 - iter 30/50 - loss 2.61309492 - samples/sec: 79.13 - lr: 0.062500 +2021-03-26 06:26:01,568 epoch 42 - iter 35/50 - loss 2.62725921 - samples/sec: 77.92 - lr: 0.062500 +2021-03-26 06:26:03,567 epoch 42 - iter 40/50 - loss 2.62116627 - samples/sec: 80.07 - lr: 0.062500 +2021-03-26 06:26:05,470 epoch 42 - iter 45/50 - loss 2.61511114 - samples/sec: 84.20 - lr: 0.062500 +2021-03-26 06:26:07,470 epoch 42 - iter 50/50 - loss 2.65905147 - samples/sec: 80.07 - lr: 0.062500 +2021-03-26 06:26:07,471 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:26:07,471 EPOCH 42 done: loss 2.6591 - lr 0.0625000 +2021-03-26 06:26:08,289 DEV : loss 6.842776298522949 - score 0.9073 +2021-03-26 06:26:08,315 BAD EPOCHS (no improvement): 2 +2021-03-26 06:26:08,316 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:26:10,383 epoch 43 - iter 5/50 - loss 2.99454298 - samples/sec: 77.48 - lr: 0.062500 +2021-03-26 06:26:12,295 epoch 43 - iter 10/50 - loss 2.78095438 - samples/sec: 83.76 - lr: 0.062500 +2021-03-26 06:26:14,251 epoch 43 - iter 15/50 - loss 2.72735570 - samples/sec: 81.87 - lr: 0.062500 +2021-03-26 06:26:16,157 epoch 43 - iter 20/50 - loss 2.76881919 - samples/sec: 84.02 - lr: 0.062500 +2021-03-26 06:26:18,023 epoch 43 - iter 25/50 - loss 2.70888021 - samples/sec: 85.83 - lr: 0.062500 +2021-03-26 06:26:20,126 epoch 43 - iter 30/50 - loss 2.77247720 - samples/sec: 76.15 - lr: 0.062500 +2021-03-26 06:26:21,995 epoch 43 - iter 35/50 - loss 2.79106678 - samples/sec: 85.74 - lr: 0.062500 +2021-03-26 06:26:23,894 epoch 43 - iter 40/50 - loss 2.72925793 - samples/sec: 84.33 - lr: 0.062500 +2021-03-26 06:26:25,728 epoch 43 - iter 45/50 - loss 2.62710329 - samples/sec: 87.37 - lr: 0.062500 +2021-03-26 06:26:27,771 epoch 43 - iter 50/50 - loss 2.69412341 - samples/sec: 78.35 - lr: 0.062500 +2021-03-26 06:26:27,772 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:26:27,772 EPOCH 43 done: loss 2.6941 - lr 0.0625000 +2021-03-26 06:26:28,606 DEV : loss 6.853123664855957 - score 0.9077 +2021-03-26 06:26:28,632 BAD EPOCHS (no improvement): 0 +2021-03-26 06:26:38,399 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:26:40,417 epoch 44 - iter 5/50 - loss 2.92129889 - samples/sec: 79.39 - lr: 0.062500 +2021-03-26 06:26:42,392 epoch 44 - iter 10/50 - loss 2.71936698 - samples/sec: 81.11 - lr: 0.062500 +2021-03-26 06:26:44,365 epoch 44 - iter 15/50 - loss 2.64585795 - samples/sec: 81.18 - lr: 0.062500 +2021-03-26 06:26:46,328 epoch 44 - iter 20/50 - loss 2.64387424 - samples/sec: 81.57 - lr: 0.062500 +2021-03-26 06:26:48,311 epoch 44 - iter 25/50 - loss 2.71903181 - samples/sec: 80.74 - lr: 0.062500 +2021-03-26 06:26:50,314 epoch 44 - iter 30/50 - loss 2.72144562 - samples/sec: 80.00 - lr: 0.062500 +2021-03-26 06:26:52,203 epoch 44 - iter 35/50 - loss 2.68308628 - samples/sec: 84.77 - lr: 0.062500 +2021-03-26 06:26:53,970 epoch 44 - iter 40/50 - loss 2.65461990 - samples/sec: 90.67 - lr: 0.062500 +2021-03-26 06:26:55,801 epoch 44 - iter 45/50 - loss 2.61197187 - samples/sec: 87.43 - lr: 0.062500 +2021-03-26 06:26:57,800 epoch 44 - iter 50/50 - loss 2.62100311 - samples/sec: 80.13 - lr: 0.062500 +2021-03-26 06:26:57,801 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:26:57,801 EPOCH 44 done: loss 2.6210 - lr 0.0625000 +2021-03-26 06:26:58,632 DEV : loss 6.854997158050537 - score 0.9049 +2021-03-26 06:26:58,658 BAD EPOCHS (no improvement): 1 +2021-03-26 06:26:58,659 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:27:00,764 epoch 45 - iter 5/50 - loss 3.28177967 - samples/sec: 76.10 - lr: 0.062500 +2021-03-26 06:27:02,821 epoch 45 - iter 10/50 - loss 3.06228626 - samples/sec: 77.88 - lr: 0.062500 +2021-03-26 06:27:04,672 epoch 45 - iter 15/50 - loss 2.91840841 - samples/sec: 86.60 - lr: 0.062500 +2021-03-26 06:27:06,564 epoch 45 - iter 20/50 - loss 2.84936029 - samples/sec: 84.61 - lr: 0.062500 +2021-03-26 06:27:08,635 epoch 45 - iter 25/50 - loss 2.74072369 - samples/sec: 77.34 - lr: 0.062500 +2021-03-26 06:27:10,756 epoch 45 - iter 30/50 - loss 2.75789352 - samples/sec: 75.48 - lr: 0.062500 +2021-03-26 06:27:12,729 epoch 45 - iter 35/50 - loss 2.69706963 - samples/sec: 81.19 - lr: 0.062500 +2021-03-26 06:27:14,716 epoch 45 - iter 40/50 - loss 2.65504144 - samples/sec: 80.57 - lr: 0.062500 +2021-03-26 06:27:16,848 epoch 45 - iter 45/50 - loss 2.69478651 - samples/sec: 75.14 - lr: 0.062500 +2021-03-26 06:27:18,660 epoch 45 - iter 50/50 - loss 2.65418336 - samples/sec: 88.40 - lr: 0.062500 +2021-03-26 06:27:18,661 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:27:18,661 EPOCH 45 done: loss 2.6542 - lr 0.0625000 +2021-03-26 06:27:19,458 DEV : loss 6.984309196472168 - score 0.9061 +2021-03-26 06:27:19,477 BAD EPOCHS (no improvement): 2 +2021-03-26 06:27:19,478 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:27:21,492 epoch 46 - iter 5/50 - loss 2.30996280 - samples/sec: 79.50 - lr: 0.062500 +2021-03-26 06:27:23,499 epoch 46 - iter 10/50 - loss 2.30306711 - samples/sec: 79.82 - lr: 0.062500 +2021-03-26 06:27:25,486 epoch 46 - iter 15/50 - loss 2.26478287 - samples/sec: 80.63 - lr: 0.062500 +2021-03-26 06:27:27,582 epoch 46 - iter 20/50 - loss 2.30942514 - samples/sec: 76.41 - lr: 0.062500 +2021-03-26 06:27:29,516 epoch 46 - iter 25/50 - loss 2.29877130 - samples/sec: 82.80 - lr: 0.062500 +2021-03-26 06:27:31,389 epoch 46 - iter 30/50 - loss 2.34397881 - samples/sec: 85.53 - lr: 0.062500 +2021-03-26 06:27:33,547 epoch 46 - iter 35/50 - loss 2.41658501 - samples/sec: 74.21 - lr: 0.062500 +2021-03-26 06:27:35,528 epoch 46 - iter 40/50 - loss 2.38164246 - samples/sec: 80.82 - lr: 0.062500 +2021-03-26 06:27:37,533 epoch 46 - iter 45/50 - loss 2.40390159 - samples/sec: 79.93 - lr: 0.062500 +2021-03-26 06:27:39,300 epoch 46 - iter 50/50 - loss 2.46796207 - samples/sec: 90.61 - lr: 0.062500 +2021-03-26 06:27:39,301 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:27:39,301 EPOCH 46 done: loss 2.4680 - lr 0.0625000 +2021-03-26 06:27:40,109 DEV : loss 6.913209438323975 - score 0.9063 +2021-03-26 06:27:40,135 BAD EPOCHS (no improvement): 3 +2021-03-26 06:27:40,136 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:27:42,137 epoch 47 - iter 5/50 - loss 2.18956802 - samples/sec: 80.03 - lr: 0.062500 +2021-03-26 06:27:44,077 epoch 47 - iter 10/50 - loss 2.32679708 - samples/sec: 82.55 - lr: 0.062500 +2021-03-26 06:27:46,002 epoch 47 - iter 15/50 - loss 2.30111314 - samples/sec: 83.20 - lr: 0.062500 +2021-03-26 06:27:47,814 epoch 47 - iter 20/50 - loss 2.36776996 - samples/sec: 88.42 - lr: 0.062500 +2021-03-26 06:27:49,736 epoch 47 - iter 25/50 - loss 2.41560647 - samples/sec: 83.32 - lr: 0.062500 +2021-03-26 06:27:51,867 epoch 47 - iter 30/50 - loss 2.51549866 - samples/sec: 75.16 - lr: 0.062500 +2021-03-26 06:27:53,813 epoch 47 - iter 35/50 - loss 2.52135927 - samples/sec: 82.32 - lr: 0.062500 +2021-03-26 06:27:56,368 epoch 47 - iter 40/50 - loss 2.52845867 - samples/sec: 62.66 - lr: 0.062500 +2021-03-26 06:27:58,419 epoch 47 - iter 45/50 - loss 2.54538176 - samples/sec: 78.09 - lr: 0.062500 +2021-03-26 06:28:00,367 epoch 47 - iter 50/50 - loss 2.54789333 - samples/sec: 82.20 - lr: 0.062500 +2021-03-26 06:28:00,368 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:28:00,368 EPOCH 47 done: loss 2.5479 - lr 0.0625000 +2021-03-26 06:28:01,161 DEV : loss 6.987135887145996 - score 0.9045 +2021-03-26 06:28:01,180 BAD EPOCHS (no improvement): 4 +2021-03-26 06:28:01,180 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:28:03,192 epoch 48 - iter 5/50 - loss 2.81061130 - samples/sec: 79.65 - lr: 0.031250 +2021-03-26 06:28:05,368 epoch 48 - iter 10/50 - loss 2.62682199 - samples/sec: 73.58 - lr: 0.031250 +2021-03-26 06:28:07,466 epoch 48 - iter 15/50 - loss 2.61593165 - samples/sec: 76.32 - lr: 0.031250 +2021-03-26 06:28:09,554 epoch 48 - iter 20/50 - loss 2.65244003 - samples/sec: 76.75 - lr: 0.031250 +2021-03-26 06:28:11,657 epoch 48 - iter 25/50 - loss 2.63186843 - samples/sec: 76.12 - lr: 0.031250 +2021-03-26 06:28:13,596 epoch 48 - iter 30/50 - loss 2.64713775 - samples/sec: 82.63 - lr: 0.031250 +2021-03-26 06:28:15,574 epoch 48 - iter 35/50 - loss 2.62877602 - samples/sec: 80.94 - lr: 0.031250 +2021-03-26 06:28:17,448 epoch 48 - iter 40/50 - loss 2.60681930 - samples/sec: 85.49 - lr: 0.031250 +2021-03-26 06:28:19,385 epoch 48 - iter 45/50 - loss 2.60738970 - samples/sec: 82.65 - lr: 0.031250 +2021-03-26 06:28:21,336 epoch 48 - iter 50/50 - loss 2.59641813 - samples/sec: 82.08 - lr: 0.031250 +2021-03-26 06:28:21,337 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:28:21,337 EPOCH 48 done: loss 2.5964 - lr 0.0312500 +2021-03-26 06:28:22,159 DEV : loss 6.932497978210449 - score 0.9067 +2021-03-26 06:28:22,178 BAD EPOCHS (no improvement): 1 +2021-03-26 06:28:22,178 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:28:24,056 epoch 49 - iter 5/50 - loss 1.88422883 - samples/sec: 85.30 - lr: 0.031250 +2021-03-26 06:28:26,127 epoch 49 - iter 10/50 - loss 2.10428255 - samples/sec: 77.33 - lr: 0.031250 +2021-03-26 06:28:28,343 epoch 49 - iter 15/50 - loss 2.11673807 - samples/sec: 72.28 - lr: 0.031250 +2021-03-26 06:28:30,435 epoch 49 - iter 20/50 - loss 2.26653606 - samples/sec: 76.52 - lr: 0.031250 +2021-03-26 06:28:32,392 epoch 49 - iter 25/50 - loss 2.33341152 - samples/sec: 81.86 - lr: 0.031250 +2021-03-26 06:28:34,420 epoch 49 - iter 30/50 - loss 2.31826424 - samples/sec: 78.94 - lr: 0.031250 +2021-03-26 06:28:36,367 epoch 49 - iter 35/50 - loss 2.28970445 - samples/sec: 82.25 - lr: 0.031250 +2021-03-26 06:28:38,483 epoch 49 - iter 40/50 - loss 2.29986542 - samples/sec: 75.66 - lr: 0.031250 +2021-03-26 06:28:40,619 epoch 49 - iter 45/50 - loss 2.30139024 - samples/sec: 74.98 - lr: 0.031250 +2021-03-26 06:28:42,612 epoch 49 - iter 50/50 - loss 2.28942521 - samples/sec: 80.38 - lr: 0.031250 +2021-03-26 06:28:42,613 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:28:42,613 EPOCH 49 done: loss 2.2894 - lr 0.0312500 +2021-03-26 06:28:43,404 DEV : loss 6.93155574798584 - score 0.9073 +2021-03-26 06:28:43,429 BAD EPOCHS (no improvement): 2 +2021-03-26 06:28:43,430 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:28:45,259 epoch 50 - iter 5/50 - loss 2.34129095 - samples/sec: 87.59 - lr: 0.031250 +2021-03-26 06:28:47,334 epoch 50 - iter 10/50 - loss 2.65424306 - samples/sec: 77.19 - lr: 0.031250 +2021-03-26 06:28:49,442 epoch 50 - iter 15/50 - loss 2.70122841 - samples/sec: 75.98 - lr: 0.031250 +2021-03-26 06:28:51,798 epoch 50 - iter 20/50 - loss 2.61473920 - samples/sec: 67.98 - lr: 0.031250 +2021-03-26 06:28:53,857 epoch 50 - iter 25/50 - loss 2.58245116 - samples/sec: 77.82 - lr: 0.031250 +2021-03-26 06:28:55,886 epoch 50 - iter 30/50 - loss 2.49980253 - samples/sec: 78.94 - lr: 0.031250 +2021-03-26 06:28:57,844 epoch 50 - iter 35/50 - loss 2.42350192 - samples/sec: 81.98 - lr: 0.031250 +2021-03-26 06:28:59,900 epoch 50 - iter 40/50 - loss 2.43286690 - samples/sec: 77.87 - lr: 0.031250 +2021-03-26 06:29:01,807 epoch 50 - iter 45/50 - loss 2.45776600 - samples/sec: 83.97 - lr: 0.031250 +2021-03-26 06:29:03,680 epoch 50 - iter 50/50 - loss 2.47108955 - samples/sec: 85.54 - lr: 0.031250 +2021-03-26 06:29:03,680 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:29:03,681 EPOCH 50 done: loss 2.4711 - lr 0.0312500 +2021-03-26 06:29:04,483 DEV : loss 6.923022270202637 - score 0.9079 +2021-03-26 06:29:04,513 BAD EPOCHS (no improvement): 0 +2021-03-26 06:29:14,331 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:29:16,637 epoch 51 - iter 5/50 - loss 2.19648669 - samples/sec: 69.48 - lr: 0.031250 +2021-03-26 06:29:18,647 epoch 51 - iter 10/50 - loss 2.40439309 - samples/sec: 79.64 - lr: 0.031250 +2021-03-26 06:29:20,792 epoch 51 - iter 15/50 - loss 2.41015805 - samples/sec: 74.67 - lr: 0.031250 +2021-03-26 06:29:22,920 epoch 51 - iter 20/50 - loss 2.38679830 - samples/sec: 75.24 - lr: 0.031250 +2021-03-26 06:29:24,740 epoch 51 - iter 25/50 - loss 2.36478817 - samples/sec: 88.00 - lr: 0.031250 +2021-03-26 06:29:26,747 epoch 51 - iter 30/50 - loss 2.38509798 - samples/sec: 79.80 - lr: 0.031250 +2021-03-26 06:29:28,748 epoch 51 - iter 35/50 - loss 2.38499307 - samples/sec: 80.05 - lr: 0.031250 +2021-03-26 06:29:30,781 epoch 51 - iter 40/50 - loss 2.38311165 - samples/sec: 78.76 - lr: 0.031250 +2021-03-26 06:29:32,956 epoch 51 - iter 45/50 - loss 2.38785090 - samples/sec: 73.65 - lr: 0.031250 +2021-03-26 06:29:34,543 epoch 51 - iter 50/50 - loss 2.33555826 - samples/sec: 100.92 - lr: 0.031250 +2021-03-26 06:29:34,544 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:29:34,544 EPOCH 51 done: loss 2.3356 - lr 0.0312500 +2021-03-26 06:29:35,465 DEV : loss 6.930795192718506 - score 0.9085 +2021-03-26 06:29:35,489 BAD EPOCHS (no improvement): 0 +2021-03-26 06:29:45,316 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:29:47,331 epoch 52 - iter 5/50 - loss 2.31803503 - samples/sec: 79.53 - lr: 0.031250 +2021-03-26 06:29:49,380 epoch 52 - iter 10/50 - loss 2.48303349 - samples/sec: 78.17 - lr: 0.031250 +2021-03-26 06:29:51,368 epoch 52 - iter 15/50 - loss 2.52508163 - samples/sec: 80.55 - lr: 0.031250 +2021-03-26 06:29:53,344 epoch 52 - iter 20/50 - loss 2.48334989 - samples/sec: 81.04 - lr: 0.031250 +2021-03-26 06:29:55,432 epoch 52 - iter 25/50 - loss 2.55548737 - samples/sec: 76.71 - lr: 0.031250 +2021-03-26 06:29:57,291 epoch 52 - iter 30/50 - loss 2.52494575 - samples/sec: 86.15 - lr: 0.031250 +2021-03-26 06:29:59,287 epoch 52 - iter 35/50 - loss 2.56608360 - samples/sec: 80.23 - lr: 0.031250 +2021-03-26 06:30:01,387 epoch 52 - iter 40/50 - loss 2.55950036 - samples/sec: 76.25 - lr: 0.031250 +2021-03-26 06:30:03,315 epoch 52 - iter 45/50 - loss 2.50635450 - samples/sec: 83.06 - lr: 0.031250 +2021-03-26 06:30:05,174 epoch 52 - iter 50/50 - loss 2.49100937 - samples/sec: 86.16 - lr: 0.031250 +2021-03-26 06:30:05,174 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:30:05,174 EPOCH 52 done: loss 2.4910 - lr 0.0312500 +2021-03-26 06:30:06,005 DEV : loss 6.951305389404297 - score 0.9065 +2021-03-26 06:30:06,030 BAD EPOCHS (no improvement): 1 +2021-03-26 06:30:06,031 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:30:07,924 epoch 53 - iter 5/50 - loss 2.52924616 - samples/sec: 84.61 - lr: 0.031250 +2021-03-26 06:30:10,031 epoch 53 - iter 10/50 - loss 2.44083222 - samples/sec: 76.18 - lr: 0.031250 +2021-03-26 06:30:12,320 epoch 53 - iter 15/50 - loss 2.48448149 - samples/sec: 69.97 - lr: 0.031250 +2021-03-26 06:30:14,239 epoch 53 - iter 20/50 - loss 2.39575719 - samples/sec: 83.42 - lr: 0.031250 +2021-03-26 06:30:16,356 epoch 53 - iter 25/50 - loss 2.44080576 - samples/sec: 75.66 - lr: 0.031250 +2021-03-26 06:30:18,319 epoch 53 - iter 30/50 - loss 2.45035690 - samples/sec: 81.60 - lr: 0.031250 +2021-03-26 06:30:20,224 epoch 53 - iter 35/50 - loss 2.47454193 - samples/sec: 84.05 - lr: 0.031250 +2021-03-26 06:30:22,368 epoch 53 - iter 40/50 - loss 2.51555236 - samples/sec: 74.72 - lr: 0.031250 +2021-03-26 06:30:24,355 epoch 53 - iter 45/50 - loss 2.46132856 - samples/sec: 80.60 - lr: 0.031250 +2021-03-26 06:30:25,991 epoch 53 - iter 50/50 - loss 2.44961992 - samples/sec: 97.93 - lr: 0.031250 +2021-03-26 06:30:25,992 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:30:25,992 EPOCH 53 done: loss 2.4496 - lr 0.0312500 +2021-03-26 06:30:26,800 DEV : loss 6.995821952819824 - score 0.9069 +2021-03-26 06:30:26,826 BAD EPOCHS (no improvement): 2 +2021-03-26 06:30:26,827 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:30:28,642 epoch 54 - iter 5/50 - loss 1.79619451 - samples/sec: 88.23 - lr: 0.031250 +2021-03-26 06:30:30,643 epoch 54 - iter 10/50 - loss 2.11190095 - samples/sec: 80.05 - lr: 0.031250 +2021-03-26 06:30:32,587 epoch 54 - iter 15/50 - loss 2.20107423 - samples/sec: 82.38 - lr: 0.031250 +2021-03-26 06:30:34,720 epoch 54 - iter 20/50 - loss 2.28467238 - samples/sec: 75.08 - lr: 0.031250 +2021-03-26 06:30:36,636 epoch 54 - iter 25/50 - loss 2.27544066 - samples/sec: 83.63 - lr: 0.031250 +2021-03-26 06:30:38,920 epoch 54 - iter 30/50 - loss 2.28766701 - samples/sec: 70.11 - lr: 0.031250 +2021-03-26 06:30:40,846 epoch 54 - iter 35/50 - loss 2.34247447 - samples/sec: 83.17 - lr: 0.031250 +2021-03-26 06:30:43,019 epoch 54 - iter 40/50 - loss 2.37213463 - samples/sec: 73.68 - lr: 0.031250 +2021-03-26 06:30:45,034 epoch 54 - iter 45/50 - loss 2.38454656 - samples/sec: 79.47 - lr: 0.031250 +2021-03-26 06:30:46,956 epoch 54 - iter 50/50 - loss 2.32289241 - samples/sec: 83.39 - lr: 0.031250 +2021-03-26 06:30:46,957 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:30:46,958 EPOCH 54 done: loss 2.3229 - lr 0.0312500 +2021-03-26 06:30:47,808 DEV : loss 6.978719711303711 - score 0.9081 +2021-03-26 06:30:47,833 BAD EPOCHS (no improvement): 3 +2021-03-26 06:30:47,834 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:30:49,840 epoch 55 - iter 5/50 - loss 2.17873693 - samples/sec: 79.88 - lr: 0.031250 +2021-03-26 06:30:51,816 epoch 55 - iter 10/50 - loss 2.29883653 - samples/sec: 81.08 - lr: 0.031250 +2021-03-26 06:30:53,847 epoch 55 - iter 15/50 - loss 2.39913157 - samples/sec: 78.84 - lr: 0.031250 +2021-03-26 06:30:55,812 epoch 55 - iter 20/50 - loss 2.48835702 - samples/sec: 81.48 - lr: 0.031250 +2021-03-26 06:30:57,836 epoch 55 - iter 25/50 - loss 2.56765687 - samples/sec: 79.18 - lr: 0.031250 +2021-03-26 06:30:59,832 epoch 55 - iter 30/50 - loss 2.46267944 - samples/sec: 80.21 - lr: 0.031250 +2021-03-26 06:31:01,858 epoch 55 - iter 35/50 - loss 2.45796065 - samples/sec: 79.03 - lr: 0.031250 +2021-03-26 06:31:03,845 epoch 55 - iter 40/50 - loss 2.41670229 - samples/sec: 80.64 - lr: 0.031250 +2021-03-26 06:31:05,833 epoch 55 - iter 45/50 - loss 2.39211874 - samples/sec: 80.56 - lr: 0.031250 +2021-03-26 06:31:07,734 epoch 55 - iter 50/50 - loss 2.39450463 - samples/sec: 84.25 - lr: 0.031250 +2021-03-26 06:31:07,735 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:31:07,735 EPOCH 55 done: loss 2.3945 - lr 0.0312500 +2021-03-26 06:31:08,516 DEV : loss 6.9821391105651855 - score 0.9085 +2021-03-26 06:31:08,542 BAD EPOCHS (no improvement): 4 +2021-03-26 06:31:08,543 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:31:10,595 epoch 56 - iter 5/50 - loss 2.45446777 - samples/sec: 78.07 - lr: 0.015625 +2021-03-26 06:31:12,510 epoch 56 - iter 10/50 - loss 2.22199507 - samples/sec: 83.59 - lr: 0.015625 +2021-03-26 06:31:14,363 epoch 56 - iter 15/50 - loss 2.10502376 - samples/sec: 86.51 - lr: 0.015625 +2021-03-26 06:31:16,485 epoch 56 - iter 20/50 - loss 2.24321173 - samples/sec: 75.47 - lr: 0.015625 +2021-03-26 06:31:18,768 epoch 56 - iter 25/50 - loss 2.23664840 - samples/sec: 70.16 - lr: 0.015625 +2021-03-26 06:31:20,821 epoch 56 - iter 30/50 - loss 2.30482445 - samples/sec: 77.98 - lr: 0.015625 +2021-03-26 06:31:22,870 epoch 56 - iter 35/50 - loss 2.29642651 - samples/sec: 78.22 - lr: 0.015625 +2021-03-26 06:31:25,184 epoch 56 - iter 40/50 - loss 2.26805057 - samples/sec: 69.19 - lr: 0.015625 +2021-03-26 06:31:27,337 epoch 56 - iter 45/50 - loss 2.22978039 - samples/sec: 74.40 - lr: 0.015625 +2021-03-26 06:31:29,212 epoch 56 - iter 50/50 - loss 2.24846376 - samples/sec: 85.42 - lr: 0.015625 +2021-03-26 06:31:29,213 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:31:29,213 EPOCH 56 done: loss 2.2485 - lr 0.0156250 +2021-03-26 06:31:30,036 DEV : loss 6.974067687988281 - score 0.9085 +2021-03-26 06:31:30,062 BAD EPOCHS (no improvement): 1 +2021-03-26 06:31:30,063 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:31:32,073 epoch 57 - iter 5/50 - loss 2.64384212 - samples/sec: 79.65 - lr: 0.015625 +2021-03-26 06:31:34,058 epoch 57 - iter 10/50 - loss 2.46076876 - samples/sec: 80.66 - lr: 0.015625 +2021-03-26 06:31:35,971 epoch 57 - iter 15/50 - loss 2.51936740 - samples/sec: 83.73 - lr: 0.015625 +2021-03-26 06:31:37,840 epoch 57 - iter 20/50 - loss 2.41057147 - samples/sec: 85.70 - lr: 0.015625 +2021-03-26 06:31:39,880 epoch 57 - iter 25/50 - loss 2.46316812 - samples/sec: 78.53 - lr: 0.015625 +2021-03-26 06:31:41,926 epoch 57 - iter 30/50 - loss 2.50473585 - samples/sec: 78.26 - lr: 0.015625 +2021-03-26 06:31:43,993 epoch 57 - iter 35/50 - loss 2.50068537 - samples/sec: 77.53 - lr: 0.015625 +2021-03-26 06:31:46,047 epoch 57 - iter 40/50 - loss 2.52348277 - samples/sec: 77.97 - lr: 0.015625 +2021-03-26 06:31:47,885 epoch 57 - iter 45/50 - loss 2.52473041 - samples/sec: 87.19 - lr: 0.015625 +2021-03-26 06:31:49,749 epoch 57 - iter 50/50 - loss 2.51703615 - samples/sec: 85.91 - lr: 0.015625 +2021-03-26 06:31:49,750 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:31:49,750 EPOCH 57 done: loss 2.5170 - lr 0.0156250 +2021-03-26 06:31:50,525 DEV : loss 6.9953436851501465 - score 0.9085 +2021-03-26 06:31:50,551 BAD EPOCHS (no improvement): 2 +2021-03-26 06:31:50,552 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:31:52,377 epoch 58 - iter 5/50 - loss 1.99235198 - samples/sec: 87.79 - lr: 0.015625 +2021-03-26 06:31:54,379 epoch 58 - iter 10/50 - loss 2.16711535 - samples/sec: 79.98 - lr: 0.015625 +2021-03-26 06:31:56,416 epoch 58 - iter 15/50 - loss 2.26408356 - samples/sec: 78.63 - lr: 0.015625 +2021-03-26 06:31:58,382 epoch 58 - iter 20/50 - loss 2.20503994 - samples/sec: 81.45 - lr: 0.015625 +2021-03-26 06:32:00,325 epoch 58 - iter 25/50 - loss 2.27958061 - samples/sec: 82.63 - lr: 0.015625 +2021-03-26 06:32:02,186 epoch 58 - iter 30/50 - loss 2.28277320 - samples/sec: 86.03 - lr: 0.015625 +2021-03-26 06:32:04,143 epoch 58 - iter 35/50 - loss 2.27501016 - samples/sec: 82.02 - lr: 0.015625 +2021-03-26 06:32:06,054 epoch 58 - iter 40/50 - loss 2.27238119 - samples/sec: 83.82 - lr: 0.015625 +2021-03-26 06:32:08,064 epoch 58 - iter 45/50 - loss 2.24532957 - samples/sec: 79.72 - lr: 0.015625 +2021-03-26 06:32:09,825 epoch 58 - iter 50/50 - loss 2.30386667 - samples/sec: 90.94 - lr: 0.015625 +2021-03-26 06:32:09,825 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:32:09,826 EPOCH 58 done: loss 2.3039 - lr 0.0156250 +2021-03-26 06:32:10,603 DEV : loss 6.978977203369141 - score 0.9085 +2021-03-26 06:32:10,628 BAD EPOCHS (no improvement): 3 +2021-03-26 06:32:10,629 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:32:12,571 epoch 59 - iter 5/50 - loss 2.65150757 - samples/sec: 82.47 - lr: 0.015625 +2021-03-26 06:32:14,555 epoch 59 - iter 10/50 - loss 2.28247037 - samples/sec: 80.68 - lr: 0.015625 +2021-03-26 06:32:16,560 epoch 59 - iter 15/50 - loss 2.24065713 - samples/sec: 79.88 - lr: 0.015625 +2021-03-26 06:32:18,386 epoch 59 - iter 20/50 - loss 2.24658821 - samples/sec: 87.77 - lr: 0.015625 +2021-03-26 06:32:20,261 epoch 59 - iter 25/50 - loss 2.21917336 - samples/sec: 85.44 - lr: 0.015625 +2021-03-26 06:32:22,271 epoch 59 - iter 30/50 - loss 2.27102419 - samples/sec: 79.68 - lr: 0.015625 +2021-03-26 06:32:24,218 epoch 59 - iter 35/50 - loss 2.29636217 - samples/sec: 82.24 - lr: 0.015625 +2021-03-26 06:32:26,292 epoch 59 - iter 40/50 - loss 2.35502929 - samples/sec: 77.23 - lr: 0.015625 +2021-03-26 06:32:28,437 epoch 59 - iter 45/50 - loss 2.34205148 - samples/sec: 74.70 - lr: 0.015625 +2021-03-26 06:32:30,252 epoch 59 - iter 50/50 - loss 2.32947894 - samples/sec: 88.26 - lr: 0.015625 +2021-03-26 06:32:30,254 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:32:30,254 EPOCH 59 done: loss 2.3295 - lr 0.0156250 +2021-03-26 06:32:31,029 DEV : loss 6.991868495941162 - score 0.9077 +2021-03-26 06:32:31,055 BAD EPOCHS (no improvement): 4 +2021-03-26 06:32:31,055 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:32:32,924 epoch 60 - iter 5/50 - loss 2.06255875 - samples/sec: 85.70 - lr: 0.007812 +2021-03-26 06:32:35,056 epoch 60 - iter 10/50 - loss 2.46557306 - samples/sec: 75.15 - lr: 0.007812 +2021-03-26 06:32:37,085 epoch 60 - iter 15/50 - loss 2.41977864 - samples/sec: 78.95 - lr: 0.007812 +2021-03-26 06:32:39,435 epoch 60 - iter 20/50 - loss 2.33271241 - samples/sec: 68.18 - lr: 0.007812 +2021-03-26 06:32:41,548 epoch 60 - iter 25/50 - loss 2.40608020 - samples/sec: 75.81 - lr: 0.007812 +2021-03-26 06:32:43,612 epoch 60 - iter 30/50 - loss 2.41698582 - samples/sec: 77.60 - lr: 0.007812 +2021-03-26 06:32:45,488 epoch 60 - iter 35/50 - loss 2.41312816 - samples/sec: 85.38 - lr: 0.007812 +2021-03-26 06:32:47,628 epoch 60 - iter 40/50 - loss 2.44367003 - samples/sec: 74.83 - lr: 0.007812 +2021-03-26 06:32:49,645 epoch 60 - iter 45/50 - loss 2.46628921 - samples/sec: 79.39 - lr: 0.007812 +2021-03-26 06:32:51,504 epoch 60 - iter 50/50 - loss 2.49696096 - samples/sec: 86.20 - lr: 0.007812 +2021-03-26 06:32:51,505 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:32:51,505 EPOCH 60 done: loss 2.4970 - lr 0.0078125 +2021-03-26 06:32:52,343 DEV : loss 6.983875274658203 - score 0.9085 +2021-03-26 06:32:52,368 BAD EPOCHS (no improvement): 1 +2021-03-26 06:32:52,369 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:32:54,449 epoch 61 - iter 5/50 - loss 2.79651160 - samples/sec: 76.99 - lr: 0.007812 +2021-03-26 06:32:56,568 epoch 61 - iter 10/50 - loss 2.62580528 - samples/sec: 75.57 - lr: 0.007812 +2021-03-26 06:32:58,704 epoch 61 - iter 15/50 - loss 2.37854838 - samples/sec: 74.97 - lr: 0.007812 +2021-03-26 06:33:00,696 epoch 61 - iter 20/50 - loss 2.32473083 - samples/sec: 80.40 - lr: 0.007812 +2021-03-26 06:33:02,726 epoch 61 - iter 25/50 - loss 2.32159848 - samples/sec: 78.89 - lr: 0.007812 +2021-03-26 06:33:04,813 epoch 61 - iter 30/50 - loss 2.37875148 - samples/sec: 76.72 - lr: 0.007812 +2021-03-26 06:33:06,687 epoch 61 - iter 35/50 - loss 2.38597493 - samples/sec: 85.53 - lr: 0.007812 +2021-03-26 06:33:08,626 epoch 61 - iter 40/50 - loss 2.39198539 - samples/sec: 82.63 - lr: 0.007812 +2021-03-26 06:33:10,685 epoch 61 - iter 45/50 - loss 2.36188733 - samples/sec: 77.77 - lr: 0.007812 +2021-03-26 06:33:12,661 epoch 61 - iter 50/50 - loss 2.39228940 - samples/sec: 81.06 - lr: 0.007812 +2021-03-26 06:33:12,662 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:33:12,662 EPOCH 61 done: loss 2.3923 - lr 0.0078125 +2021-03-26 06:33:13,474 DEV : loss 6.977319240570068 - score 0.9081 +2021-03-26 06:33:13,498 BAD EPOCHS (no improvement): 2 +2021-03-26 06:33:13,498 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:33:15,563 epoch 62 - iter 5/50 - loss 2.31099248 - samples/sec: 77.58 - lr: 0.007812 +2021-03-26 06:33:17,406 epoch 62 - iter 10/50 - loss 2.62232625 - samples/sec: 86.91 - lr: 0.007812 +2021-03-26 06:33:19,558 epoch 62 - iter 15/50 - loss 2.64010038 - samples/sec: 74.41 - lr: 0.007812 +2021-03-26 06:33:21,592 epoch 62 - iter 20/50 - loss 2.55420481 - samples/sec: 78.75 - lr: 0.007812 +2021-03-26 06:33:23,697 epoch 62 - iter 25/50 - loss 2.54057485 - samples/sec: 76.06 - lr: 0.007812 +2021-03-26 06:33:25,633 epoch 62 - iter 30/50 - loss 2.48879879 - samples/sec: 82.80 - lr: 0.007812 +2021-03-26 06:33:27,457 epoch 62 - iter 35/50 - loss 2.39246745 - samples/sec: 87.83 - lr: 0.007812 +2021-03-26 06:33:29,382 epoch 62 - iter 40/50 - loss 2.38044807 - samples/sec: 83.24 - lr: 0.007812 +2021-03-26 06:33:31,243 epoch 62 - iter 45/50 - loss 2.39029401 - samples/sec: 86.12 - lr: 0.007812 +2021-03-26 06:33:33,048 epoch 62 - iter 50/50 - loss 2.34383795 - samples/sec: 88.71 - lr: 0.007812 +2021-03-26 06:33:33,050 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:33:33,050 EPOCH 62 done: loss 2.3438 - lr 0.0078125 +2021-03-26 06:33:33,867 DEV : loss 6.974912166595459 - score 0.9081 +2021-03-26 06:33:33,886 BAD EPOCHS (no improvement): 3 +2021-03-26 06:33:33,887 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:33:35,800 epoch 63 - iter 5/50 - loss 2.44417329 - samples/sec: 83.72 - lr: 0.007812 +2021-03-26 06:33:37,832 epoch 63 - iter 10/50 - loss 2.33602285 - samples/sec: 78.83 - lr: 0.007812 +2021-03-26 06:33:39,903 epoch 63 - iter 15/50 - loss 2.30568608 - samples/sec: 77.32 - lr: 0.007812 +2021-03-26 06:33:41,871 epoch 63 - iter 20/50 - loss 2.34029179 - samples/sec: 81.38 - lr: 0.007812 +2021-03-26 06:33:43,840 epoch 63 - iter 25/50 - loss 2.40513277 - samples/sec: 81.33 - lr: 0.007812 +2021-03-26 06:33:45,749 epoch 63 - iter 30/50 - loss 2.46392405 - samples/sec: 83.88 - lr: 0.007812 +2021-03-26 06:33:47,690 epoch 63 - iter 35/50 - loss 2.45260457 - samples/sec: 82.53 - lr: 0.007812 +2021-03-26 06:33:49,832 epoch 63 - iter 40/50 - loss 2.47064377 - samples/sec: 74.75 - lr: 0.007812 +2021-03-26 06:33:51,744 epoch 63 - iter 45/50 - loss 2.47501290 - samples/sec: 83.76 - lr: 0.007812 +2021-03-26 06:33:53,476 epoch 63 - iter 50/50 - loss 2.43155539 - samples/sec: 92.54 - lr: 0.007812 +2021-03-26 06:33:53,477 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:33:53,477 EPOCH 63 done: loss 2.4316 - lr 0.0078125 +2021-03-26 06:33:54,263 DEV : loss 6.969865322113037 - score 0.9069 +2021-03-26 06:33:54,281 BAD EPOCHS (no improvement): 4 +2021-03-26 06:33:54,281 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:33:56,270 epoch 64 - iter 5/50 - loss 2.39939275 - samples/sec: 80.54 - lr: 0.003906 +2021-03-26 06:33:58,286 epoch 64 - iter 10/50 - loss 2.33425676 - samples/sec: 79.44 - lr: 0.003906 +2021-03-26 06:34:00,204 epoch 64 - iter 15/50 - loss 2.25310161 - samples/sec: 83.49 - lr: 0.003906 +2021-03-26 06:34:02,188 epoch 64 - iter 20/50 - loss 2.31001709 - samples/sec: 80.76 - lr: 0.003906 +2021-03-26 06:34:04,360 epoch 64 - iter 25/50 - loss 2.33261591 - samples/sec: 73.76 - lr: 0.003906 +2021-03-26 06:34:06,230 epoch 64 - iter 30/50 - loss 2.31331635 - samples/sec: 85.62 - lr: 0.003906 +2021-03-26 06:34:08,207 epoch 64 - iter 35/50 - loss 2.21878336 - samples/sec: 81.01 - lr: 0.003906 +2021-03-26 06:34:10,376 epoch 64 - iter 40/50 - loss 2.24540700 - samples/sec: 73.88 - lr: 0.003906 +2021-03-26 06:34:12,411 epoch 64 - iter 45/50 - loss 2.26711696 - samples/sec: 78.71 - lr: 0.003906 +2021-03-26 06:34:14,178 epoch 64 - iter 50/50 - loss 2.25436100 - samples/sec: 90.64 - lr: 0.003906 +2021-03-26 06:34:14,179 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:34:14,179 EPOCH 64 done: loss 2.2544 - lr 0.0039062 +2021-03-26 06:34:14,985 DEV : loss 6.970977783203125 - score 0.9069 +2021-03-26 06:34:15,008 BAD EPOCHS (no improvement): 1 +2021-03-26 06:34:15,009 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:34:17,126 epoch 65 - iter 5/50 - loss 2.40217991 - samples/sec: 75.67 - lr: 0.003906 +2021-03-26 06:34:19,195 epoch 65 - iter 10/50 - loss 2.63361117 - samples/sec: 77.41 - lr: 0.003906 +2021-03-26 06:34:21,188 epoch 65 - iter 15/50 - loss 2.50125097 - samples/sec: 80.35 - lr: 0.003906 +2021-03-26 06:34:23,109 epoch 65 - iter 20/50 - loss 2.52610157 - samples/sec: 83.38 - lr: 0.003906 +2021-03-26 06:34:25,070 epoch 65 - iter 25/50 - loss 2.49457950 - samples/sec: 81.69 - lr: 0.003906 +2021-03-26 06:34:26,939 epoch 65 - iter 30/50 - loss 2.41889030 - samples/sec: 85.65 - lr: 0.003906 +2021-03-26 06:34:28,862 epoch 65 - iter 35/50 - loss 2.39602560 - samples/sec: 83.31 - lr: 0.003906 +2021-03-26 06:34:31,097 epoch 65 - iter 40/50 - loss 2.44050950 - samples/sec: 71.67 - lr: 0.003906 +2021-03-26 06:34:33,128 epoch 65 - iter 45/50 - loss 2.48830043 - samples/sec: 78.83 - lr: 0.003906 +2021-03-26 06:34:35,244 epoch 65 - iter 50/50 - loss 2.49684168 - samples/sec: 75.70 - lr: 0.003906 +2021-03-26 06:34:35,245 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:34:35,245 EPOCH 65 done: loss 2.4968 - lr 0.0039062 +2021-03-26 06:34:36,057 DEV : loss 6.972274303436279 - score 0.9073 +2021-03-26 06:34:36,078 BAD EPOCHS (no improvement): 2 +2021-03-26 06:34:36,079 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:34:38,081 epoch 66 - iter 5/50 - loss 2.64790382 - samples/sec: 79.98 - lr: 0.003906 +2021-03-26 06:34:40,053 epoch 66 - iter 10/50 - loss 2.31422681 - samples/sec: 81.27 - lr: 0.003906 +2021-03-26 06:34:42,108 epoch 66 - iter 15/50 - loss 2.18160001 - samples/sec: 77.93 - lr: 0.003906 +2021-03-26 06:34:44,010 epoch 66 - iter 20/50 - loss 2.20333160 - samples/sec: 84.24 - lr: 0.003906 +2021-03-26 06:34:46,138 epoch 66 - iter 25/50 - loss 2.22043241 - samples/sec: 75.27 - lr: 0.003906 +2021-03-26 06:34:48,024 epoch 66 - iter 30/50 - loss 2.23266133 - samples/sec: 84.94 - lr: 0.003906 +2021-03-26 06:34:50,202 epoch 66 - iter 35/50 - loss 2.22326636 - samples/sec: 73.52 - lr: 0.003906 +2021-03-26 06:34:52,060 epoch 66 - iter 40/50 - loss 2.22595744 - samples/sec: 86.24 - lr: 0.003906 +2021-03-26 06:34:54,013 epoch 66 - iter 45/50 - loss 2.27869542 - samples/sec: 82.05 - lr: 0.003906 +2021-03-26 06:34:55,845 epoch 66 - iter 50/50 - loss 2.30708797 - samples/sec: 87.42 - lr: 0.003906 +2021-03-26 06:34:55,846 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:34:55,847 EPOCH 66 done: loss 2.3071 - lr 0.0039062 +2021-03-26 06:34:56,675 DEV : loss 6.974557399749756 - score 0.9077 +2021-03-26 06:34:56,714 BAD EPOCHS (no improvement): 3 +2021-03-26 06:34:56,715 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:34:58,841 epoch 67 - iter 5/50 - loss 2.96658783 - samples/sec: 75.32 - lr: 0.003906 +2021-03-26 06:35:00,812 epoch 67 - iter 10/50 - loss 2.71965330 - samples/sec: 81.26 - lr: 0.003906 +2021-03-26 06:35:02,970 epoch 67 - iter 15/50 - loss 2.48100974 - samples/sec: 74.21 - lr: 0.003906 +2021-03-26 06:35:04,864 epoch 67 - iter 20/50 - loss 2.41596460 - samples/sec: 84.56 - lr: 0.003906 +2021-03-26 06:35:06,998 epoch 67 - iter 25/50 - loss 2.46593875 - samples/sec: 75.08 - lr: 0.003906 +2021-03-26 06:35:08,909 epoch 67 - iter 30/50 - loss 2.43296328 - samples/sec: 83.79 - lr: 0.003906 +2021-03-26 06:35:10,769 epoch 67 - iter 35/50 - loss 2.38436657 - samples/sec: 86.10 - lr: 0.003906 +2021-03-26 06:35:12,947 epoch 67 - iter 40/50 - loss 2.38160687 - samples/sec: 73.52 - lr: 0.003906 +2021-03-26 06:35:14,827 epoch 67 - iter 45/50 - loss 2.41164478 - samples/sec: 85.23 - lr: 0.003906 +2021-03-26 06:35:16,677 epoch 67 - iter 50/50 - loss 2.36796960 - samples/sec: 86.61 - lr: 0.003906 +2021-03-26 06:35:16,677 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:35:16,678 EPOCH 67 done: loss 2.3680 - lr 0.0039062 +2021-03-26 06:35:17,495 DEV : loss 6.978951930999756 - score 0.9081 +2021-03-26 06:35:17,520 BAD EPOCHS (no improvement): 4 +2021-03-26 06:35:17,520 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:35:19,624 epoch 68 - iter 5/50 - loss 2.00108428 - samples/sec: 76.15 - lr: 0.001953 +2021-03-26 06:35:21,586 epoch 68 - iter 10/50 - loss 2.05748752 - samples/sec: 81.61 - lr: 0.001953 +2021-03-26 06:35:23,543 epoch 68 - iter 15/50 - loss 2.20883069 - samples/sec: 81.81 - lr: 0.001953 +2021-03-26 06:35:25,506 epoch 68 - iter 20/50 - loss 2.31636751 - samples/sec: 81.62 - lr: 0.001953 +2021-03-26 06:35:27,585 epoch 68 - iter 25/50 - loss 2.26701699 - samples/sec: 77.00 - lr: 0.001953 +2021-03-26 06:35:29,632 epoch 68 - iter 30/50 - loss 2.25838938 - samples/sec: 78.25 - lr: 0.001953 +2021-03-26 06:35:31,617 epoch 68 - iter 35/50 - loss 2.29719248 - samples/sec: 80.74 - lr: 0.001953 +2021-03-26 06:35:33,673 epoch 68 - iter 40/50 - loss 2.31215978 - samples/sec: 77.88 - lr: 0.001953 +2021-03-26 06:35:35,496 epoch 68 - iter 45/50 - loss 2.28480958 - samples/sec: 87.82 - lr: 0.001953 +2021-03-26 06:35:37,365 epoch 68 - iter 50/50 - loss 2.36084390 - samples/sec: 85.70 - lr: 0.001953 +2021-03-26 06:35:37,366 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:35:37,366 EPOCH 68 done: loss 2.3608 - lr 0.0019531 +2021-03-26 06:35:38,197 DEV : loss 6.982030391693115 - score 0.9085 +2021-03-26 06:35:38,225 BAD EPOCHS (no improvement): 1 +2021-03-26 06:35:38,226 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:35:40,234 epoch 69 - iter 5/50 - loss 2.34777708 - samples/sec: 79.76 - lr: 0.001953 +2021-03-26 06:35:42,268 epoch 69 - iter 10/50 - loss 2.57082732 - samples/sec: 78.72 - lr: 0.001953 +2021-03-26 06:35:44,194 epoch 69 - iter 15/50 - loss 2.47520032 - samples/sec: 83.17 - lr: 0.001953 +2021-03-26 06:35:46,157 epoch 69 - iter 20/50 - loss 2.37645018 - samples/sec: 81.61 - lr: 0.001953 +2021-03-26 06:35:48,206 epoch 69 - iter 25/50 - loss 2.29699348 - samples/sec: 78.14 - lr: 0.001953 +2021-03-26 06:35:50,224 epoch 69 - iter 30/50 - loss 2.33158594 - samples/sec: 79.37 - lr: 0.001953 +2021-03-26 06:35:52,198 epoch 69 - iter 35/50 - loss 2.37844305 - samples/sec: 81.14 - lr: 0.001953 +2021-03-26 06:35:54,057 epoch 69 - iter 40/50 - loss 2.34696364 - samples/sec: 86.16 - lr: 0.001953 +2021-03-26 06:35:56,198 epoch 69 - iter 45/50 - loss 2.36180254 - samples/sec: 74.80 - lr: 0.001953 +2021-03-26 06:35:58,097 epoch 69 - iter 50/50 - loss 2.36717457 - samples/sec: 84.33 - lr: 0.001953 +2021-03-26 06:35:58,098 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:35:58,098 EPOCH 69 done: loss 2.3672 - lr 0.0019531 +2021-03-26 06:35:58,905 DEV : loss 6.981610298156738 - score 0.9085 +2021-03-26 06:35:58,931 BAD EPOCHS (no improvement): 2 +2021-03-26 06:35:58,932 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:36:00,950 epoch 70 - iter 5/50 - loss 2.39901481 - samples/sec: 79.38 - lr: 0.001953 +2021-03-26 06:36:02,836 epoch 70 - iter 10/50 - loss 2.35888321 - samples/sec: 84.88 - lr: 0.001953 +2021-03-26 06:36:04,860 epoch 70 - iter 15/50 - loss 2.43844715 - samples/sec: 79.15 - lr: 0.001953 +2021-03-26 06:36:06,786 epoch 70 - iter 20/50 - loss 2.33588195 - samples/sec: 83.12 - lr: 0.001953 +2021-03-26 06:36:08,782 epoch 70 - iter 25/50 - loss 2.25162241 - samples/sec: 80.30 - lr: 0.001953 +2021-03-26 06:36:10,721 epoch 70 - iter 30/50 - loss 2.21778770 - samples/sec: 82.58 - lr: 0.001953 +2021-03-26 06:36:12,888 epoch 70 - iter 35/50 - loss 2.20874165 - samples/sec: 73.90 - lr: 0.001953 +2021-03-26 06:36:14,860 epoch 70 - iter 40/50 - loss 2.23795464 - samples/sec: 81.20 - lr: 0.001953 +2021-03-26 06:36:16,889 epoch 70 - iter 45/50 - loss 2.28399056 - samples/sec: 78.91 - lr: 0.001953 +2021-03-26 06:36:18,675 epoch 70 - iter 50/50 - loss 2.28482699 - samples/sec: 89.68 - lr: 0.001953 +2021-03-26 06:36:18,676 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:36:18,676 EPOCH 70 done: loss 2.2848 - lr 0.0019531 +2021-03-26 06:36:19,499 DEV : loss 6.978541851043701 - score 0.9085 +2021-03-26 06:36:19,526 BAD EPOCHS (no improvement): 3 +2021-03-26 06:36:19,526 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:36:21,448 epoch 71 - iter 5/50 - loss 2.11014709 - samples/sec: 83.38 - lr: 0.001953 +2021-03-26 06:36:23,579 epoch 71 - iter 10/50 - loss 2.29674499 - samples/sec: 75.16 - lr: 0.001953 +2021-03-26 06:36:25,884 epoch 71 - iter 15/50 - loss 2.46122870 - samples/sec: 69.46 - lr: 0.001953 +2021-03-26 06:36:27,768 epoch 71 - iter 20/50 - loss 2.38656909 - samples/sec: 85.01 - lr: 0.001953 +2021-03-26 06:36:29,708 epoch 71 - iter 25/50 - loss 2.40159395 - samples/sec: 82.54 - lr: 0.001953 +2021-03-26 06:36:31,656 epoch 71 - iter 30/50 - loss 2.42725286 - samples/sec: 82.26 - lr: 0.001953 +2021-03-26 06:36:33,384 epoch 71 - iter 35/50 - loss 2.40592557 - samples/sec: 92.71 - lr: 0.001953 +2021-03-26 06:36:35,346 epoch 71 - iter 40/50 - loss 2.38390470 - samples/sec: 81.63 - lr: 0.001953 +2021-03-26 06:36:37,375 epoch 71 - iter 45/50 - loss 2.38397009 - samples/sec: 78.91 - lr: 0.001953 +2021-03-26 06:36:39,274 epoch 71 - iter 50/50 - loss 2.38416300 - samples/sec: 84.33 - lr: 0.001953 +2021-03-26 06:36:39,275 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:36:39,275 EPOCH 71 done: loss 2.3842 - lr 0.0019531 +2021-03-26 06:36:40,111 DEV : loss 6.977227687835693 - score 0.9081 +2021-03-26 06:36:40,138 BAD EPOCHS (no improvement): 4 +2021-03-26 06:36:40,138 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:36:42,121 epoch 72 - iter 5/50 - loss 2.51898463 - samples/sec: 80.79 - lr: 0.000977 +2021-03-26 06:36:44,016 epoch 72 - iter 10/50 - loss 2.57655363 - samples/sec: 84.49 - lr: 0.000977 +2021-03-26 06:36:45,969 epoch 72 - iter 15/50 - loss 2.44915295 - samples/sec: 82.02 - lr: 0.000977 +2021-03-26 06:36:47,892 epoch 72 - iter 20/50 - loss 2.42828819 - samples/sec: 83.27 - lr: 0.000977 +2021-03-26 06:36:49,708 epoch 72 - iter 25/50 - loss 2.40627950 - samples/sec: 88.18 - lr: 0.000977 +2021-03-26 06:36:51,922 epoch 72 - iter 30/50 - loss 2.45578847 - samples/sec: 72.32 - lr: 0.000977 +2021-03-26 06:36:54,237 epoch 72 - iter 35/50 - loss 2.44811320 - samples/sec: 69.20 - lr: 0.000977 +2021-03-26 06:36:56,060 epoch 72 - iter 40/50 - loss 2.42384168 - samples/sec: 87.83 - lr: 0.000977 +2021-03-26 06:36:58,065 epoch 72 - iter 45/50 - loss 2.46919827 - samples/sec: 79.86 - lr: 0.000977 +2021-03-26 06:36:59,973 epoch 72 - iter 50/50 - loss 2.42740743 - samples/sec: 83.94 - lr: 0.000977 +2021-03-26 06:36:59,974 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:36:59,974 EPOCH 72 done: loss 2.4274 - lr 0.0009766 +2021-03-26 06:37:00,798 DEV : loss 6.975755214691162 - score 0.9077 +2021-03-26 06:37:00,824 BAD EPOCHS (no improvement): 1 +2021-03-26 06:37:00,825 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:37:02,909 epoch 73 - iter 5/50 - loss 2.65864558 - samples/sec: 76.82 - lr: 0.000977 +2021-03-26 06:37:04,841 epoch 73 - iter 10/50 - loss 2.38106189 - samples/sec: 82.91 - lr: 0.000977 +2021-03-26 06:37:06,808 epoch 73 - iter 15/50 - loss 2.48718756 - samples/sec: 81.41 - lr: 0.000977 +2021-03-26 06:37:08,729 epoch 73 - iter 20/50 - loss 2.44999477 - samples/sec: 83.40 - lr: 0.000977 +2021-03-26 06:37:10,749 epoch 73 - iter 25/50 - loss 2.38722777 - samples/sec: 79.28 - lr: 0.000977 +2021-03-26 06:37:12,831 epoch 73 - iter 30/50 - loss 2.47670416 - samples/sec: 76.93 - lr: 0.000977 +2021-03-26 06:37:14,732 epoch 73 - iter 35/50 - loss 2.47073682 - samples/sec: 84.24 - lr: 0.000977 +2021-03-26 06:37:16,732 epoch 73 - iter 40/50 - loss 2.47696608 - samples/sec: 80.08 - lr: 0.000977 +2021-03-26 06:37:18,700 epoch 73 - iter 45/50 - loss 2.46970473 - samples/sec: 81.40 - lr: 0.000977 +2021-03-26 06:37:20,574 epoch 73 - iter 50/50 - loss 2.43881308 - samples/sec: 85.43 - lr: 0.000977 +2021-03-26 06:37:20,575 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:37:20,575 EPOCH 73 done: loss 2.4388 - lr 0.0009766 +2021-03-26 06:37:21,431 DEV : loss 6.9776506423950195 - score 0.9077 +2021-03-26 06:37:21,457 BAD EPOCHS (no improvement): 2 +2021-03-26 06:37:21,458 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:37:23,596 epoch 74 - iter 5/50 - loss 2.47849181 - samples/sec: 74.89 - lr: 0.000977 +2021-03-26 06:37:25,583 epoch 74 - iter 10/50 - loss 2.28436197 - samples/sec: 80.65 - lr: 0.000977 +2021-03-26 06:37:27,686 epoch 74 - iter 15/50 - loss 2.22879147 - samples/sec: 76.14 - lr: 0.000977 +2021-03-26 06:37:29,797 epoch 74 - iter 20/50 - loss 2.20650732 - samples/sec: 75.85 - lr: 0.000977 +2021-03-26 06:37:31,671 epoch 74 - iter 25/50 - loss 2.30046971 - samples/sec: 85.46 - lr: 0.000977 +2021-03-26 06:37:33,840 epoch 74 - iter 30/50 - loss 2.26771376 - samples/sec: 73.84 - lr: 0.000977 +2021-03-26 06:37:36,030 epoch 74 - iter 35/50 - loss 2.31302182 - samples/sec: 73.12 - lr: 0.000977 +2021-03-26 06:37:38,171 epoch 74 - iter 40/50 - loss 2.27059739 - samples/sec: 74.85 - lr: 0.000977 +2021-03-26 06:37:40,057 epoch 74 - iter 45/50 - loss 2.24902633 - samples/sec: 84.93 - lr: 0.000977 +2021-03-26 06:37:41,941 epoch 74 - iter 50/50 - loss 2.22677505 - samples/sec: 85.03 - lr: 0.000977 +2021-03-26 06:37:41,942 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:37:41,942 EPOCH 74 done: loss 2.2268 - lr 0.0009766 +2021-03-26 06:37:42,762 DEV : loss 6.978555202484131 - score 0.9085 +2021-03-26 06:37:42,784 BAD EPOCHS (no improvement): 3 +2021-03-26 06:37:42,785 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:37:44,741 epoch 75 - iter 5/50 - loss 2.61107960 - samples/sec: 81.92 - lr: 0.000977 +2021-03-26 06:37:46,814 epoch 75 - iter 10/50 - loss 2.43473128 - samples/sec: 77.27 - lr: 0.000977 +2021-03-26 06:37:48,770 epoch 75 - iter 15/50 - loss 2.42162393 - samples/sec: 81.89 - lr: 0.000977 +2021-03-26 06:37:51,045 epoch 75 - iter 20/50 - loss 2.41610397 - samples/sec: 70.39 - lr: 0.000977 +2021-03-26 06:37:53,177 epoch 75 - iter 25/50 - loss 2.37377398 - samples/sec: 75.12 - lr: 0.000977 +2021-03-26 06:37:55,129 epoch 75 - iter 30/50 - loss 2.37280536 - samples/sec: 82.05 - lr: 0.000977 +2021-03-26 06:37:57,302 epoch 75 - iter 35/50 - loss 2.35321695 - samples/sec: 73.71 - lr: 0.000977 +2021-03-26 06:37:59,206 epoch 75 - iter 40/50 - loss 2.37838320 - samples/sec: 84.10 - lr: 0.000977 +2021-03-26 06:38:01,143 epoch 75 - iter 45/50 - loss 2.36447463 - samples/sec: 82.70 - lr: 0.000977 +2021-03-26 06:38:02,955 epoch 75 - iter 50/50 - loss 2.33910566 - samples/sec: 88.43 - lr: 0.000977 +2021-03-26 06:38:02,956 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:38:02,956 EPOCH 75 done: loss 2.3391 - lr 0.0009766 +2021-03-26 06:38:03,760 DEV : loss 6.977889060974121 - score 0.9081 +2021-03-26 06:38:03,779 BAD EPOCHS (no improvement): 4 +2021-03-26 06:38:03,780 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:38:05,974 epoch 76 - iter 5/50 - loss 2.18173971 - samples/sec: 72.99 - lr: 0.000488 +2021-03-26 06:38:08,091 epoch 76 - iter 10/50 - loss 2.49060502 - samples/sec: 75.63 - lr: 0.000488 +2021-03-26 06:38:10,140 epoch 76 - iter 15/50 - loss 2.63107382 - samples/sec: 78.19 - lr: 0.000488 +2021-03-26 06:38:12,034 epoch 76 - iter 20/50 - loss 2.49251807 - samples/sec: 84.56 - lr: 0.000488 +2021-03-26 06:38:14,076 epoch 76 - iter 25/50 - loss 2.46709409 - samples/sec: 78.41 - lr: 0.000488 +2021-03-26 06:38:16,021 epoch 76 - iter 30/50 - loss 2.41153224 - samples/sec: 82.33 - lr: 0.000488 +2021-03-26 06:38:18,042 epoch 76 - iter 35/50 - loss 2.44753593 - samples/sec: 79.30 - lr: 0.000488 +2021-03-26 06:38:20,055 epoch 76 - iter 40/50 - loss 2.40904220 - samples/sec: 79.55 - lr: 0.000488 +2021-03-26 06:38:22,198 epoch 76 - iter 45/50 - loss 2.36160962 - samples/sec: 74.75 - lr: 0.000488 +2021-03-26 06:38:23,977 epoch 76 - iter 50/50 - loss 2.33519658 - samples/sec: 90.02 - lr: 0.000488 +2021-03-26 06:38:23,978 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:38:23,979 EPOCH 76 done: loss 2.3352 - lr 0.0004883 +2021-03-26 06:38:24,807 DEV : loss 6.977874755859375 - score 0.9085 +2021-03-26 06:38:24,832 BAD EPOCHS (no improvement): 1 +2021-03-26 06:38:24,833 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:38:26,749 epoch 77 - iter 5/50 - loss 2.34179015 - samples/sec: 83.62 - lr: 0.000488 +2021-03-26 06:38:28,651 epoch 77 - iter 10/50 - loss 2.40300212 - samples/sec: 84.21 - lr: 0.000488 +2021-03-26 06:38:30,673 epoch 77 - iter 15/50 - loss 2.56726098 - samples/sec: 79.18 - lr: 0.000488 +2021-03-26 06:38:32,700 epoch 77 - iter 20/50 - loss 2.45115488 - samples/sec: 78.98 - lr: 0.000488 +2021-03-26 06:38:34,697 epoch 77 - iter 25/50 - loss 2.38101733 - samples/sec: 80.25 - lr: 0.000488 +2021-03-26 06:38:36,544 epoch 77 - iter 30/50 - loss 2.32366014 - samples/sec: 86.75 - lr: 0.000488 +2021-03-26 06:38:38,512 epoch 77 - iter 35/50 - loss 2.29412194 - samples/sec: 81.38 - lr: 0.000488 +2021-03-26 06:38:40,441 epoch 77 - iter 40/50 - loss 2.28109241 - samples/sec: 83.00 - lr: 0.000488 +2021-03-26 06:38:42,316 epoch 77 - iter 45/50 - loss 2.24014996 - samples/sec: 85.40 - lr: 0.000488 +2021-03-26 06:38:44,230 epoch 77 - iter 50/50 - loss 2.22467905 - samples/sec: 83.71 - lr: 0.000488 +2021-03-26 06:38:44,231 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:38:44,231 EPOCH 77 done: loss 2.2247 - lr 0.0004883 +2021-03-26 06:38:45,039 DEV : loss 6.978282928466797 - score 0.9085 +2021-03-26 06:38:45,065 BAD EPOCHS (no improvement): 2 +2021-03-26 06:38:45,066 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:38:47,132 epoch 78 - iter 5/50 - loss 2.36803710 - samples/sec: 77.53 - lr: 0.000488 +2021-03-26 06:38:49,245 epoch 78 - iter 10/50 - loss 2.36767075 - samples/sec: 75.80 - lr: 0.000488 +2021-03-26 06:38:51,321 epoch 78 - iter 15/50 - loss 2.36822032 - samples/sec: 77.12 - lr: 0.000488 +2021-03-26 06:38:53,205 epoch 78 - iter 20/50 - loss 2.43670136 - samples/sec: 85.02 - lr: 0.000488 +2021-03-26 06:38:56,392 epoch 78 - iter 25/50 - loss 2.54821656 - samples/sec: 50.23 - lr: 0.000488 +2021-03-26 06:38:58,274 epoch 78 - iter 30/50 - loss 2.50956246 - samples/sec: 85.14 - lr: 0.000488 +2021-03-26 06:39:00,269 epoch 78 - iter 35/50 - loss 2.50315506 - samples/sec: 80.26 - lr: 0.000488 +2021-03-26 06:39:02,195 epoch 78 - iter 40/50 - loss 2.48417808 - samples/sec: 83.11 - lr: 0.000488 +2021-03-26 06:39:04,387 epoch 78 - iter 45/50 - loss 2.48333181 - samples/sec: 73.10 - lr: 0.000488 +2021-03-26 06:39:06,265 epoch 78 - iter 50/50 - loss 2.51921896 - samples/sec: 85.24 - lr: 0.000488 +2021-03-26 06:39:06,266 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:39:06,266 EPOCH 78 done: loss 2.5192 - lr 0.0004883 +2021-03-26 06:39:07,058 DEV : loss 6.978475093841553 - score 0.9089 +2021-03-26 06:39:07,077 BAD EPOCHS (no improvement): 0 +2021-03-26 06:39:16,637 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:39:18,659 epoch 79 - iter 5/50 - loss 2.17209723 - samples/sec: 79.21 - lr: 0.000488 +2021-03-26 06:39:20,697 epoch 79 - iter 10/50 - loss 2.41715573 - samples/sec: 78.56 - lr: 0.000488 +2021-03-26 06:39:22,792 epoch 79 - iter 15/50 - loss 2.41948086 - samples/sec: 76.43 - lr: 0.000488 +2021-03-26 06:39:24,817 epoch 79 - iter 20/50 - loss 2.34051616 - samples/sec: 79.12 - lr: 0.000488 +2021-03-26 06:39:26,806 epoch 79 - iter 25/50 - loss 2.37092519 - samples/sec: 80.54 - lr: 0.000488 +2021-03-26 06:39:28,681 epoch 79 - iter 30/50 - loss 2.37991382 - samples/sec: 85.39 - lr: 0.000488 +2021-03-26 06:39:30,604 epoch 79 - iter 35/50 - loss 2.33880022 - samples/sec: 83.30 - lr: 0.000488 +2021-03-26 06:39:32,575 epoch 79 - iter 40/50 - loss 2.31781658 - samples/sec: 81.27 - lr: 0.000488 +2021-03-26 06:39:34,484 epoch 79 - iter 45/50 - loss 2.31230679 - samples/sec: 83.87 - lr: 0.000488 +2021-03-26 06:39:36,253 epoch 79 - iter 50/50 - loss 2.32847360 - samples/sec: 90.55 - lr: 0.000488 +2021-03-26 06:39:36,253 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:39:36,254 EPOCH 79 done: loss 2.3285 - lr 0.0004883 +2021-03-26 06:39:37,043 DEV : loss 6.978264808654785 - score 0.9085 +2021-03-26 06:39:37,068 BAD EPOCHS (no improvement): 1 +2021-03-26 06:39:37,069 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:39:38,917 epoch 80 - iter 5/50 - loss 2.10100920 - samples/sec: 86.66 - lr: 0.000488 +2021-03-26 06:39:40,905 epoch 80 - iter 10/50 - loss 2.34779080 - samples/sec: 80.60 - lr: 0.000488 +2021-03-26 06:39:42,961 epoch 80 - iter 15/50 - loss 2.31027282 - samples/sec: 77.91 - lr: 0.000488 +2021-03-26 06:39:45,191 epoch 80 - iter 20/50 - loss 2.34272481 - samples/sec: 71.79 - lr: 0.000488 +2021-03-26 06:39:47,142 epoch 80 - iter 25/50 - loss 2.42053922 - samples/sec: 82.11 - lr: 0.000488 +2021-03-26 06:39:48,938 epoch 80 - iter 30/50 - loss 2.38158702 - samples/sec: 89.16 - lr: 0.000488 +2021-03-26 06:39:50,971 epoch 80 - iter 35/50 - loss 2.35023596 - samples/sec: 78.75 - lr: 0.000488 +2021-03-26 06:39:52,937 epoch 80 - iter 40/50 - loss 2.35125192 - samples/sec: 81.50 - lr: 0.000488 +2021-03-26 06:39:54,859 epoch 80 - iter 45/50 - loss 2.32870909 - samples/sec: 83.29 - lr: 0.000488 +2021-03-26 06:39:56,680 epoch 80 - iter 50/50 - loss 2.33931771 - samples/sec: 88.00 - lr: 0.000488 +2021-03-26 06:39:56,681 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:39:56,681 EPOCH 80 done: loss 2.3393 - lr 0.0004883 +2021-03-26 06:39:57,472 DEV : loss 6.979015350341797 - score 0.9085 +2021-03-26 06:39:57,492 BAD EPOCHS (no improvement): 2 +2021-03-26 06:39:57,493 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:39:59,386 epoch 81 - iter 5/50 - loss 2.24958613 - samples/sec: 84.62 - lr: 0.000488 +2021-03-26 06:40:01,445 epoch 81 - iter 10/50 - loss 2.38275572 - samples/sec: 77.79 - lr: 0.000488 +2021-03-26 06:40:03,325 epoch 81 - iter 15/50 - loss 2.29164292 - samples/sec: 85.19 - lr: 0.000488 +2021-03-26 06:40:05,495 epoch 81 - iter 20/50 - loss 2.23068131 - samples/sec: 73.83 - lr: 0.000488 +2021-03-26 06:40:07,548 epoch 81 - iter 25/50 - loss 2.28351954 - samples/sec: 78.03 - lr: 0.000488 +2021-03-26 06:40:09,528 epoch 81 - iter 30/50 - loss 2.28917814 - samples/sec: 80.88 - lr: 0.000488 +2021-03-26 06:40:11,488 epoch 81 - iter 35/50 - loss 2.27807065 - samples/sec: 81.74 - lr: 0.000488 +2021-03-26 06:40:13,472 epoch 81 - iter 40/50 - loss 2.29849924 - samples/sec: 80.76 - lr: 0.000488 +2021-03-26 06:40:15,586 epoch 81 - iter 45/50 - loss 2.32857788 - samples/sec: 75.76 - lr: 0.000488 +2021-03-26 06:40:17,260 epoch 81 - iter 50/50 - loss 2.28520381 - samples/sec: 95.68 - lr: 0.000488 +2021-03-26 06:40:17,261 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:40:17,261 EPOCH 81 done: loss 2.2852 - lr 0.0004883 +2021-03-26 06:40:18,038 DEV : loss 6.977649211883545 - score 0.9085 +2021-03-26 06:40:18,063 BAD EPOCHS (no improvement): 3 +2021-03-26 06:40:18,064 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:40:20,200 epoch 82 - iter 5/50 - loss 2.10976236 - samples/sec: 74.98 - lr: 0.000488 +2021-03-26 06:40:22,140 epoch 82 - iter 10/50 - loss 2.34995564 - samples/sec: 82.61 - lr: 0.000488 +2021-03-26 06:40:24,016 epoch 82 - iter 15/50 - loss 2.16507681 - samples/sec: 85.34 - lr: 0.000488 +2021-03-26 06:40:25,816 epoch 82 - iter 20/50 - loss 2.16873180 - samples/sec: 89.04 - lr: 0.000488 +2021-03-26 06:40:27,697 epoch 82 - iter 25/50 - loss 2.16649104 - samples/sec: 85.15 - lr: 0.000488 +2021-03-26 06:40:29,542 epoch 82 - iter 30/50 - loss 2.19325154 - samples/sec: 86.85 - lr: 0.000488 +2021-03-26 06:40:31,484 epoch 82 - iter 35/50 - loss 2.21187508 - samples/sec: 82.44 - lr: 0.000488 +2021-03-26 06:40:33,431 epoch 82 - iter 40/50 - loss 2.25956178 - samples/sec: 82.27 - lr: 0.000488 +2021-03-26 06:40:35,325 epoch 82 - iter 45/50 - loss 2.28050901 - samples/sec: 84.53 - lr: 0.000488 +2021-03-26 06:40:37,077 epoch 82 - iter 50/50 - loss 2.32061182 - samples/sec: 91.44 - lr: 0.000488 +2021-03-26 06:40:37,078 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:40:37,078 EPOCH 82 done: loss 2.3206 - lr 0.0004883 +2021-03-26 06:40:37,850 DEV : loss 6.976663112640381 - score 0.9081 +2021-03-26 06:40:37,871 BAD EPOCHS (no improvement): 4 +2021-03-26 06:40:37,871 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:40:39,804 epoch 83 - iter 5/50 - loss 2.51074567 - samples/sec: 82.87 - lr: 0.000244 +2021-03-26 06:40:41,789 epoch 83 - iter 10/50 - loss 2.25548234 - samples/sec: 80.69 - lr: 0.000244 +2021-03-26 06:40:43,924 epoch 83 - iter 15/50 - loss 2.50467954 - samples/sec: 75.03 - lr: 0.000244 +2021-03-26 06:40:45,824 epoch 83 - iter 20/50 - loss 2.39716090 - samples/sec: 84.29 - lr: 0.000244 +2021-03-26 06:40:47,599 epoch 83 - iter 25/50 - loss 2.38792141 - samples/sec: 90.23 - lr: 0.000244 +2021-03-26 06:40:49,486 epoch 83 - iter 30/50 - loss 2.36569843 - samples/sec: 84.89 - lr: 0.000244 +2021-03-26 06:40:51,382 epoch 83 - iter 35/50 - loss 2.37671602 - samples/sec: 84.45 - lr: 0.000244 +2021-03-26 06:40:53,214 epoch 83 - iter 40/50 - loss 2.37701699 - samples/sec: 87.46 - lr: 0.000244 +2021-03-26 06:40:55,165 epoch 83 - iter 45/50 - loss 2.35641030 - samples/sec: 82.05 - lr: 0.000244 +2021-03-26 06:40:57,019 epoch 83 - iter 50/50 - loss 2.40662164 - samples/sec: 86.38 - lr: 0.000244 +2021-03-26 06:40:57,020 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:40:57,020 EPOCH 83 done: loss 2.4066 - lr 0.0002441 +2021-03-26 06:40:57,859 DEV : loss 6.976783752441406 - score 0.9081 +2021-03-26 06:40:57,882 BAD EPOCHS (no improvement): 1 +2021-03-26 06:40:57,883 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:40:59,880 epoch 84 - iter 5/50 - loss 2.11736424 - samples/sec: 80.18 - lr: 0.000244 +2021-03-26 06:41:01,725 epoch 84 - iter 10/50 - loss 2.27809557 - samples/sec: 86.82 - lr: 0.000244 +2021-03-26 06:41:03,986 epoch 84 - iter 15/50 - loss 2.30093797 - samples/sec: 70.82 - lr: 0.000244 +2021-03-26 06:41:06,068 epoch 84 - iter 20/50 - loss 2.30980914 - samples/sec: 76.93 - lr: 0.000244 +2021-03-26 06:41:07,898 epoch 84 - iter 25/50 - loss 2.37576721 - samples/sec: 87.50 - lr: 0.000244 +2021-03-26 06:41:09,867 epoch 84 - iter 30/50 - loss 2.36038343 - samples/sec: 81.32 - lr: 0.000244 +2021-03-26 06:41:11,715 epoch 84 - iter 35/50 - loss 2.37259437 - samples/sec: 86.69 - lr: 0.000244 +2021-03-26 06:41:13,635 epoch 84 - iter 40/50 - loss 2.32505912 - samples/sec: 83.41 - lr: 0.000244 +2021-03-26 06:41:15,493 epoch 84 - iter 45/50 - loss 2.35598942 - samples/sec: 86.21 - lr: 0.000244 +2021-03-26 06:41:17,527 epoch 84 - iter 50/50 - loss 2.36311971 - samples/sec: 78.72 - lr: 0.000244 +2021-03-26 06:41:17,528 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:41:17,528 EPOCH 84 done: loss 2.3631 - lr 0.0002441 +2021-03-26 06:41:18,385 DEV : loss 6.976834297180176 - score 0.9081 +2021-03-26 06:41:18,419 BAD EPOCHS (no improvement): 2 +2021-03-26 06:41:18,420 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:41:20,424 epoch 85 - iter 5/50 - loss 2.38491588 - samples/sec: 79.94 - lr: 0.000244 +2021-03-26 06:41:22,145 epoch 85 - iter 10/50 - loss 2.31585956 - samples/sec: 93.08 - lr: 0.000244 +2021-03-26 06:41:24,066 epoch 85 - iter 15/50 - loss 2.32960722 - samples/sec: 83.37 - lr: 0.000244 +2021-03-26 06:41:25,888 epoch 85 - iter 20/50 - loss 2.32371256 - samples/sec: 87.90 - lr: 0.000244 +2021-03-26 06:41:27,846 epoch 85 - iter 25/50 - loss 2.27565221 - samples/sec: 81.78 - lr: 0.000244 +2021-03-26 06:41:29,649 epoch 85 - iter 30/50 - loss 2.28047132 - samples/sec: 88.82 - lr: 0.000244 +2021-03-26 06:41:31,515 epoch 85 - iter 35/50 - loss 2.26845495 - samples/sec: 85.83 - lr: 0.000244 +2021-03-26 06:41:33,374 epoch 85 - iter 40/50 - loss 2.25583084 - samples/sec: 86.16 - lr: 0.000244 +2021-03-26 06:41:35,260 epoch 85 - iter 45/50 - loss 2.22698094 - samples/sec: 84.92 - lr: 0.000244 +2021-03-26 06:41:36,857 epoch 85 - iter 50/50 - loss 2.19129711 - samples/sec: 100.31 - lr: 0.000244 +2021-03-26 06:41:36,857 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:41:36,858 EPOCH 85 done: loss 2.1913 - lr 0.0002441 +2021-03-26 06:41:37,613 DEV : loss 6.975969314575195 - score 0.9081 +2021-03-26 06:41:37,638 BAD EPOCHS (no improvement): 3 +2021-03-26 06:41:37,639 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:41:39,553 epoch 86 - iter 5/50 - loss 2.32959135 - samples/sec: 83.68 - lr: 0.000244 +2021-03-26 06:41:41,499 epoch 86 - iter 10/50 - loss 2.47228733 - samples/sec: 82.29 - lr: 0.000244 +2021-03-26 06:41:43,278 epoch 86 - iter 15/50 - loss 2.50017960 - samples/sec: 90.06 - lr: 0.000244 +2021-03-26 06:41:45,235 epoch 86 - iter 20/50 - loss 2.46814180 - samples/sec: 81.81 - lr: 0.000244 +2021-03-26 06:41:47,290 epoch 86 - iter 25/50 - loss 2.41104710 - samples/sec: 77.90 - lr: 0.000244 +2021-03-26 06:41:49,184 epoch 86 - iter 30/50 - loss 2.36386334 - samples/sec: 84.59 - lr: 0.000244 +2021-03-26 06:41:50,852 epoch 86 - iter 35/50 - loss 2.33780671 - samples/sec: 96.02 - lr: 0.000244 +2021-03-26 06:41:52,832 epoch 86 - iter 40/50 - loss 2.33240881 - samples/sec: 80.89 - lr: 0.000244 +2021-03-26 06:41:54,827 epoch 86 - iter 45/50 - loss 2.32977507 - samples/sec: 80.30 - lr: 0.000244 +2021-03-26 06:41:56,643 epoch 86 - iter 50/50 - loss 2.30426145 - samples/sec: 88.19 - lr: 0.000244 +2021-03-26 06:41:56,643 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:41:56,644 EPOCH 86 done: loss 2.3043 - lr 0.0002441 +2021-03-26 06:41:57,442 DEV : loss 6.975675106048584 - score 0.9081 +2021-03-26 06:41:57,467 BAD EPOCHS (no improvement): 4 +2021-03-26 06:41:57,467 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:41:59,463 epoch 87 - iter 5/50 - loss 2.02026136 - samples/sec: 80.25 - lr: 0.000122 +2021-03-26 06:42:01,359 epoch 87 - iter 10/50 - loss 2.17304178 - samples/sec: 84.49 - lr: 0.000122 +2021-03-26 06:42:03,282 epoch 87 - iter 15/50 - loss 2.38836434 - samples/sec: 83.31 - lr: 0.000122 +2021-03-26 06:42:05,124 epoch 87 - iter 20/50 - loss 2.35105944 - samples/sec: 87.13 - lr: 0.000122 +2021-03-26 06:42:07,194 epoch 87 - iter 25/50 - loss 2.37925279 - samples/sec: 77.36 - lr: 0.000122 +2021-03-26 06:42:09,259 epoch 87 - iter 30/50 - loss 2.36004749 - samples/sec: 77.55 - lr: 0.000122 +2021-03-26 06:42:11,066 epoch 87 - iter 35/50 - loss 2.35606966 - samples/sec: 88.64 - lr: 0.000122 +2021-03-26 06:42:13,291 epoch 87 - iter 40/50 - loss 2.37431050 - samples/sec: 71.97 - lr: 0.000122 +2021-03-26 06:42:15,124 epoch 87 - iter 45/50 - loss 2.33771084 - samples/sec: 87.39 - lr: 0.000122 +2021-03-26 06:42:16,850 epoch 87 - iter 50/50 - loss 2.25152152 - samples/sec: 92.81 - lr: 0.000122 +2021-03-26 06:42:16,851 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:42:16,851 EPOCH 87 done: loss 2.2515 - lr 0.0001221 +2021-03-26 06:42:17,616 DEV : loss 6.9754133224487305 - score 0.9081 +2021-03-26 06:42:17,641 BAD EPOCHS (no improvement): 1 +2021-03-26 06:42:17,642 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:42:19,618 epoch 88 - iter 5/50 - loss 2.34914608 - samples/sec: 81.03 - lr: 0.000122 +2021-03-26 06:42:21,605 epoch 88 - iter 10/50 - loss 2.30592394 - samples/sec: 80.62 - lr: 0.000122 +2021-03-26 06:42:23,575 epoch 88 - iter 15/50 - loss 2.33506842 - samples/sec: 81.27 - lr: 0.000122 +2021-03-26 06:42:25,397 epoch 88 - iter 20/50 - loss 2.24439114 - samples/sec: 87.89 - lr: 0.000122 +2021-03-26 06:42:27,390 epoch 88 - iter 25/50 - loss 2.22855584 - samples/sec: 80.34 - lr: 0.000122 +2021-03-26 06:42:29,345 epoch 88 - iter 30/50 - loss 2.26887201 - samples/sec: 81.92 - lr: 0.000122 +2021-03-26 06:42:31,099 epoch 88 - iter 35/50 - loss 2.28389955 - samples/sec: 91.31 - lr: 0.000122 +2021-03-26 06:42:33,054 epoch 88 - iter 40/50 - loss 2.30694303 - samples/sec: 81.91 - lr: 0.000122 +2021-03-26 06:42:34,922 epoch 88 - iter 45/50 - loss 2.35164838 - samples/sec: 85.74 - lr: 0.000122 +2021-03-26 06:42:36,791 epoch 88 - iter 50/50 - loss 2.29030984 - samples/sec: 85.72 - lr: 0.000122 +2021-03-26 06:42:36,791 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:42:36,792 EPOCH 88 done: loss 2.2903 - lr 0.0001221 +2021-03-26 06:42:37,578 DEV : loss 6.975350856781006 - score 0.9081 +2021-03-26 06:42:37,602 BAD EPOCHS (no improvement): 2 +2021-03-26 06:42:37,603 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:42:39,785 epoch 89 - iter 5/50 - loss 2.01891148 - samples/sec: 73.41 - lr: 0.000122 +2021-03-26 06:42:41,805 epoch 89 - iter 10/50 - loss 2.10284538 - samples/sec: 79.30 - lr: 0.000122 +2021-03-26 06:42:43,746 epoch 89 - iter 15/50 - loss 2.15385324 - samples/sec: 82.51 - lr: 0.000122 +2021-03-26 06:42:45,688 epoch 89 - iter 20/50 - loss 2.26479766 - samples/sec: 82.47 - lr: 0.000122 +2021-03-26 06:42:47,868 epoch 89 - iter 25/50 - loss 2.41565577 - samples/sec: 73.47 - lr: 0.000122 +2021-03-26 06:42:49,782 epoch 89 - iter 30/50 - loss 2.43289930 - samples/sec: 83.67 - lr: 0.000122 +2021-03-26 06:42:51,658 epoch 89 - iter 35/50 - loss 2.38985022 - samples/sec: 85.36 - lr: 0.000122 +2021-03-26 06:42:53,928 epoch 89 - iter 40/50 - loss 2.40961840 - samples/sec: 70.52 - lr: 0.000122 +2021-03-26 06:42:55,765 epoch 89 - iter 45/50 - loss 2.40256732 - samples/sec: 87.19 - lr: 0.000122 +2021-03-26 06:42:57,628 epoch 89 - iter 50/50 - loss 2.43918465 - samples/sec: 86.00 - lr: 0.000122 +2021-03-26 06:42:57,628 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:42:57,629 EPOCH 89 done: loss 2.4392 - lr 0.0001221 +2021-03-26 06:42:58,423 DEV : loss 6.9754157066345215 - score 0.9081 +2021-03-26 06:42:58,448 BAD EPOCHS (no improvement): 3 +2021-03-26 06:42:58,449 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:43:00,420 epoch 90 - iter 5/50 - loss 2.78787105 - samples/sec: 81.24 - lr: 0.000122 +2021-03-26 06:43:02,331 epoch 90 - iter 10/50 - loss 2.66487483 - samples/sec: 83.84 - lr: 0.000122 +2021-03-26 06:43:04,225 epoch 90 - iter 15/50 - loss 2.54435790 - samples/sec: 84.53 - lr: 0.000122 +2021-03-26 06:43:06,276 epoch 90 - iter 20/50 - loss 2.62745005 - samples/sec: 78.09 - lr: 0.000122 +2021-03-26 06:43:08,139 epoch 90 - iter 25/50 - loss 2.48686338 - samples/sec: 85.97 - lr: 0.000122 +2021-03-26 06:43:10,136 epoch 90 - iter 30/50 - loss 2.46287971 - samples/sec: 80.18 - lr: 0.000122 +2021-03-26 06:43:12,156 epoch 90 - iter 35/50 - loss 2.49387327 - samples/sec: 79.29 - lr: 0.000122 +2021-03-26 06:43:13,992 epoch 90 - iter 40/50 - loss 2.46719781 - samples/sec: 87.22 - lr: 0.000122 +2021-03-26 06:43:15,985 epoch 90 - iter 45/50 - loss 2.43868358 - samples/sec: 80.37 - lr: 0.000122 +2021-03-26 06:43:17,851 epoch 90 - iter 50/50 - loss 2.48019275 - samples/sec: 85.82 - lr: 0.000122 +2021-03-26 06:43:17,851 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:43:17,852 EPOCH 90 done: loss 2.4802 - lr 0.0001221 +2021-03-26 06:43:18,601 DEV : loss 6.975543022155762 - score 0.9081 +2021-03-26 06:43:18,627 BAD EPOCHS (no improvement): 4 +2021-03-26 06:43:18,627 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:43:18,628 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:43:18,628 learning rate too small - quitting training! +2021-03-26 06:43:18,628 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:43:27,954 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:43:27,955 Testing using best model ... +2021-03-26 06:43:27,956 loading file /home/tmp/megahedm/models/multipos/multipos_UDMADAR_4Diale-LEV_EGY_GLF_MGR__fasttext_flairbwfw__32__0.5_202103260608/best-model.pt +2021-03-26 06:43:35,839 0.9172 +2021-03-26 06:43:35,840 +Results: +- F-score (micro): 0.9134 +- F-score (macro): 0.5333 +- Accuracy (incl. no class): 0.9172 + +By class: + precision recall f1-score support + + INTJ 1.0000 1.0000 1.0000 16 + CCONJ 0.9892 1.0000 0.9946 92 + ADP 0.9640 0.9640 0.9640 111 + PRON 0.9945 0.9837 0.9891 184 + NOUN 0.9180 0.9527 0.9350 423 + PART 0.9513 0.9513 0.9513 226 + ADJ 0.8559 0.8347 0.8452 121 + ADV 0.9189 0.8793 0.8987 116 + PROPN 0.9500 0.7917 0.8636 24 + NUM 1.0000 0.8889 0.9412 27 + VERB 0.9529 0.9529 0.9529 170 + DET 0.9583 1.0000 0.9787 46 + PUNCT 1.0000 1.0000 1.0000 36 + SCONJ 0.9143 0.8889 0.9014 36 + AUX 0.9062 0.9355 0.9206 31 + PROG_PART+V 0.7812 0.9615 0.8621 26 + PROG_PART+V+PRON 0.7826 0.7500 0.7660 24 + PROG_PART+V+PREP+PRON 0.0000 0.0000 0.0000 3 + V+PRON 0.7778 0.8596 0.8167 57 + PART+PRON 1.0000 0.8333 0.9091 18 + DET+NOUN 0.9200 1.0000 0.9583 69 + CONJ+DET+NOUN 0.5556 1.0000 0.7143 5 + EOS 1.0000 1.0000 1.0000 70 + NOUN+CASE 1.0000 0.2500 0.4000 4 + ADJ+CASE 0.0000 1.0000 0.0000 0 + DET+NOUN+NSUFF 0.9310 0.9000 0.9153 30 + PREP+DET+NOUN 1.0000 1.0000 1.0000 10 + V 0.9091 0.8974 0.9032 78 + PREP 0.9853 0.9571 0.9710 70 + V+PRON+PRON 0.7500 0.6000 0.6667 5 + CONJ 0.9231 1.0000 0.9600 24 + PREP+V 1.0000 0.5000 0.6667 2 + PREP+PRON 0.8750 0.9545 0.9130 22 + PUNC 0.9933 1.0000 0.9967 149 + DET+ADJ 1.0000 0.7500 0.8571 16 + NOUN+NSUFF 0.8158 0.8857 0.8493 35 + CONJ+PART 0.7222 0.9286 0.8125 14 + NOUN+PRON 0.8571 0.8571 0.8571 56 + PREP+NOUN 0.9286 0.8667 0.8966 15 + PROG_PART 1.0000 1.0000 1.0000 1 + PREP+NOUN+PRON 1.0000 0.2500 0.4000 4 + PREP+NOUN+NSUFF+PRON 0.0000 0.0000 0.0000 1 + HASH 1.0000 1.0000 1.0000 13 + CONJ+NOUN 0.8333 0.7692 0.8000 13 + CONJ+V+PRON 0.6667 0.5000 0.5714 4 + CONJ+NOUN+PRON 0.0000 0.0000 0.0000 4 + MENTION 1.0000 1.0000 1.0000 32 + NOUN+NSUFF+PRON 0.5385 0.5833 0.5600 12 + CONJ+PRON 1.0000 1.0000 1.0000 5 + V+PREP+PRON 0.6667 0.3636 0.4706 11 + FUT_PART 1.0000 1.0000 1.0000 3 + DET+NUM 0.0000 1.0000 0.0000 0 + V+PRON+PREP+PRON 0.5000 0.3333 0.4000 3 + EMOT 1.0000 0.9545 0.9767 22 + PREP+PART+PRON 1.0000 1.0000 1.0000 3 + ADJ+NSUFF 0.8788 0.8788 0.8788 33 + PREP+DET+ADJ 1.0000 1.0000 1.0000 1 + CONJ+V 0.4286 1.0000 0.6000 3 + FUT_PART+V 0.8000 0.8000 0.8000 5 + PART+NOUN 0.7143 0.5556 0.6250 9 + PART+ADJ+PRON+PRON 0.0000 1.0000 0.0000 0 + CONJ+DET+ADJ 1.0000 0.0000 0.0000 3 + PREP+DET+NOUN+NSUFF 0.6667 1.0000 0.8000 2 + ADJ+PREP+PRON 1.0000 1.0000 1.0000 1 + FOREIGN 1.0000 1.0000 1.0000 3 + PROG_PART+V+PRON+PRON 0.0000 0.0000 0.0000 2 + URL 1.0000 1.0000 1.0000 3 + CONJ+PROG_PART+V 0.7500 1.0000 0.8571 3 + CONJ+ADJ 0.3333 0.2500 0.2857 4 + PREP+NOUN+NSUFF 1.0000 0.5000 0.6667 4 + CONJ+ADV 0.0000 1.0000 0.0000 0 + PREP+DET+PART 1.0000 0.0000 0.0000 1 + PART+NOUN+PRON 1.0000 0.3333 0.5000 3 + CONJ+PREP+DET+NOUN+NSUFF 1.0000 0.0000 0.0000 2 + CONJ+PART+V 0.0000 1.0000 0.0000 0 + DET+ADJ+NSUFF 0.6667 0.6667 0.6667 3 + PART+V+PRON 0.5000 0.5000 0.5000 2 + CONJ+ADJ+NSUFF+PREP+PRON 1.0000 0.0000 0.0000 1 + CONJ+PART+PREP 0.0000 1.0000 0.0000 0 + NOUN+PREP+PRON 1.0000 0.0000 0.0000 1 + ADJ+PRON 0.4286 0.6000 0.5000 5 + CONJ+FUT_PART+V+PREP+PRON 1.0000 0.0000 0.0000 1 + CONJ+PROG_PART+V+PRON 0.0000 1.0000 0.0000 0 + PART+PREP 1.0000 0.0000 0.0000 1 + PART+ADV 0.0000 1.0000 0.0000 0 + PROG_PART+V+PREP 0.0000 1.0000 0.0000 0 + PART+V 0.0000 0.0000 0.0000 2 + V+NOUN 1.0000 0.0000 0.0000 2 + ADJ+NSUFF+PRON 0.0000 1.0000 0.0000 0 + PRON+DET+NOUN+NSUFF 1.0000 0.0000 0.0000 1 + PRON+DET+NOUN 0.0000 1.0000 0.0000 0 + CONJ+PREP 1.0000 1.0000 1.0000 1 + FUT_PART+V+PRON+PRON 1.0000 0.0000 0.0000 1 + FUT_PART+V+PRON 0.6667 1.0000 0.8000 2 +PART+PROG_PART+V+PREP+PRON+NEG_PART 1.0000 0.0000 0.0000 1 + PART+PREP+PRON+NEG_PART 0.0000 1.0000 0.0000 0 + CONJ+PART+V+NEG_PART 1.0000 0.0000 0.0000 1 + PART+V+PREP+PRON+NEG_PART 1.0000 0.0000 0.0000 1 + PART+V+NEG_PART 0.3333 1.0000 0.5000 1 + V+NEG_PART 1.0000 0.0000 0.0000 1 + CONJ+FUT_PART+V+PRON 1.0000 0.0000 0.0000 1 + PART+NSUFF 0.0000 0.0000 0.0000 1 + PART+PREP+NEG_PART 1.0000 1.0000 1.0000 4 + PART+V+PRON+NEG_PART 0.5000 0.6667 0.5714 3 + NOUN+CASE+PRON 0.0000 1.0000 0.0000 0 + CONJ+PART+PREP+NEG_PART 1.0000 0.0000 0.0000 2 + PART+PART 1.0000 0.0000 0.0000 1 + PART+PROG_PART+V+NEG_PART 1.0000 0.3333 0.5000 3 + PROG_PART+V+PRON+PREP+PRON 1.0000 0.0000 0.0000 1 + NOUN+CASE+NSUFF+NOUN+PRON 1.0000 0.0000 0.0000 1 + CONJ+NOUN+NSUFF+PRON 0.0000 1.0000 0.0000 0 + CONJ+NOUN+NSUFF 0.6667 1.0000 0.8000 2 + NUM+NSUFF 1.0000 1.0000 1.0000 1 + ADV+NSUFF 1.0000 1.0000 1.0000 1 + PART+V+PRON+PRON+NEG_PART 1.0000 0.0000 0.0000 1 + CONJ+V+PRON+PREP+PRON 0.0000 1.0000 0.0000 0 + + micro avg 0.9134 0.9134 0.9134 2724 + macro avg 0.7269 0.6691 0.5333 2724 + weighted avg 0.9221 0.9134 0.9105 2724 + +2021-03-26 06:43:35,840 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:43:35,841 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:43:41,464 Reading data from ../../Datasets_adhoc/CSCS_corpus-GUC +2021-03-26 06:43:41,464 Train: ../../Datasets_adhoc/CSCS_corpus-GUC/all_participants.conllu +2021-03-26 06:43:41,465 Dev: None +2021-03-26 06:43:41,465 Test: None +2021-03-26 06:43:41,742 Reading data from ../../Datasets_adhoc/UD_MADAR +2021-03-26 06:43:41,743 Train: ../../Datasets_adhoc/UD_MADAR/ajp_madar-ud-test-edit.conllu +2021-03-26 06:43:41,743 Dev: None +2021-03-26 06:43:41,743 Test: None +2021-03-26 06:43:41,775 Reading data from ../../Datasets_adhoc/dialectal_arabic_resources-master +2021-03-26 06:43:41,776 Train: ../../Datasets_adhoc/dialectal_arabic_resources-master/seg_plus_pos_lev.txt +2021-03-26 06:43:41,776 Dev: None +2021-03-26 06:43:41,777 Test: None +2021-03-26 06:43:41,956 Reading data from ../../Datasets_adhoc/dialectal_arabic_resources-master +2021-03-26 06:43:41,956 Train: ../../Datasets_adhoc/dialectal_arabic_resources-master/seg_plus_pos_egy.txt +2021-03-26 06:43:41,957 Dev: None +2021-03-26 06:43:41,957 Test: None +2021-03-26 06:43:42,128 Reading data from ../../Datasets_adhoc/dialectal_arabic_resources-master +2021-03-26 06:43:42,128 Train: ../../Datasets_adhoc/dialectal_arabic_resources-master/seg_plus_pos_glf.txt +2021-03-26 06:43:42,129 Dev: None +2021-03-26 06:43:42,129 Test: None +2021-03-26 06:43:42,284 Reading data from ../../Datasets_adhoc/dialectal_arabic_resources-master +2021-03-26 06:43:42,284 Train: ../../Datasets_adhoc/dialectal_arabic_resources-master/seg_plus_pos_mgr.txt +2021-03-26 06:43:42,285 Dev: None +2021-03-26 06:43:42,285 Test: None +2021-03-26 06:43:42,444 Filtering long sentences +2021-03-26 06:43:42,480 MultiCorpus: 1575 train + 176 dev + 193 test sentences + - ColumnCorpus Corpus: 934 train + 104 dev + 115 test sentences + - ColumnCorpus Corpus: 81 train + 9 dev + 10 test sentences + - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences + - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences +2021-03-26 06:43:42,903 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:43:42,904 Model: "SequenceTagger( + (embeddings): StackedEmbeddings( + (list_embedding_0): WordEmbeddings('ar') + (list_embedding_1): FlairEmbeddings( + (lm): LanguageModel( + (drop): Dropout(p=0.1, inplace=False) + (encoder): Embedding(7125, 100) + (rnn): LSTM(100, 2048) + (decoder): Linear(in_features=2048, out_features=7125, bias=True) + ) + ) + (list_embedding_2): FlairEmbeddings( + (lm): LanguageModel( + (drop): Dropout(p=0.1, inplace=False) + (encoder): Embedding(7125, 100) + (rnn): LSTM(100, 2048) + (decoder): Linear(in_features=2048, out_features=7125, bias=True) + ) + ) + ) + (word_dropout): WordDropout(p=0.05) + (locked_dropout): LockedDropout(p=0.5) + (embedding2nn): Linear(in_features=4396, out_features=4396, bias=True) + (rnn): LSTM(4396, 256, batch_first=True, bidirectional=True) + (linear): Linear(in_features=512, out_features=206, bias=True) + (beta): 1.0 + (weights): None + (weight_tensor) None +)" +2021-03-26 06:43:42,904 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:43:42,904 Corpus: "MultiCorpus: 1575 train + 176 dev + 193 test sentences + - ColumnCorpus Corpus: 934 train + 104 dev + 115 test sentences + - ColumnCorpus Corpus: 81 train + 9 dev + 10 test sentences + - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences + - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences" +2021-03-26 06:43:42,905 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:43:42,905 Parameters: +2021-03-26 06:43:42,905 - learning_rate: "0.5" +2021-03-26 06:43:42,905 - mini_batch_size: "32" +2021-03-26 06:43:42,906 - patience: "3" +2021-03-26 06:43:42,906 - anneal_factor: "0.5" +2021-03-26 06:43:42,906 - max_epochs: "150" +2021-03-26 06:43:42,906 - shuffle: "True" +2021-03-26 06:43:42,907 - train_with_dev: "False" +2021-03-26 06:43:42,907 - batch_growth_annealing: "False" +2021-03-26 06:43:42,907 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:43:42,908 Model training base path: "/home/tmp/megahedm/models/multipos/multipos_UDMADAR_4Diale-LEV_EGY_GLF_MGR__fasttext_flairbwfw__32__0.5_202103260643" +2021-03-26 06:43:42,908 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:43:42,908 Device: cuda:0 +2021-03-26 06:43:42,908 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:43:42,909 Embeddings storage mode: cpu +2021-03-26 06:43:42,910 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:43:45,750 epoch 1 - iter 5/50 - loss 75.85967865 - samples/sec: 56.37 - lr: 0.500000 +2021-03-26 06:43:48,460 epoch 1 - iter 10/50 - loss 72.91885376 - samples/sec: 59.10 - lr: 0.500000 +2021-03-26 06:43:50,928 epoch 1 - iter 15/50 - loss 68.27410405 - samples/sec: 64.85 - lr: 0.500000 +2021-03-26 06:43:53,456 epoch 1 - iter 20/50 - loss 66.77132072 - samples/sec: 63.35 - lr: 0.500000 +2021-03-26 06:43:55,893 epoch 1 - iter 25/50 - loss 62.94671341 - samples/sec: 65.69 - lr: 0.500000 +2021-03-26 06:43:58,295 epoch 1 - iter 30/50 - loss 59.68897349 - samples/sec: 66.68 - lr: 0.500000 +2021-03-26 06:44:00,773 epoch 1 - iter 35/50 - loss 56.34382128 - samples/sec: 64.62 - lr: 0.500000 +2021-03-26 06:44:03,237 epoch 1 - iter 40/50 - loss 54.21474304 - samples/sec: 64.97 - lr: 0.500000 +2021-03-26 06:44:05,549 epoch 1 - iter 45/50 - loss 51.69305013 - samples/sec: 69.23 - lr: 0.500000 +2021-03-26 06:44:07,771 epoch 1 - iter 50/50 - loss 49.45876930 - samples/sec: 72.08 - lr: 0.500000 +2021-03-26 06:44:07,771 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:44:07,771 EPOCH 1 done: loss 49.4588 - lr 0.5000000 +2021-03-26 06:44:09,122 DEV : loss 28.84688377380371 - score 0.5287 +2021-03-26 06:44:09,151 BAD EPOCHS (no improvement): 0 +2021-03-26 06:44:18,447 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:44:20,548 epoch 2 - iter 5/50 - loss 29.69394226 - samples/sec: 76.23 - lr: 0.500000 +2021-03-26 06:44:22,528 epoch 2 - iter 10/50 - loss 29.02742157 - samples/sec: 80.86 - lr: 0.500000 +2021-03-26 06:44:24,434 epoch 2 - iter 15/50 - loss 28.63661626 - samples/sec: 84.02 - lr: 0.500000 +2021-03-26 06:44:26,392 epoch 2 - iter 20/50 - loss 28.82985716 - samples/sec: 81.81 - lr: 0.500000 +2021-03-26 06:44:28,342 epoch 2 - iter 25/50 - loss 27.57712044 - samples/sec: 82.14 - lr: 0.500000 +2021-03-26 06:44:30,190 epoch 2 - iter 30/50 - loss 26.53857161 - samples/sec: 86.65 - lr: 0.500000 +2021-03-26 06:44:32,111 epoch 2 - iter 35/50 - loss 25.69812393 - samples/sec: 83.38 - lr: 0.500000 +2021-03-26 06:44:34,036 epoch 2 - iter 40/50 - loss 25.05725961 - samples/sec: 83.17 - lr: 0.500000 +2021-03-26 06:44:35,814 epoch 2 - iter 45/50 - loss 24.72802349 - samples/sec: 90.09 - lr: 0.500000 +2021-03-26 06:44:37,773 epoch 2 - iter 50/50 - loss 24.46823097 - samples/sec: 81.77 - lr: 0.500000 +2021-03-26 06:44:37,774 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:44:37,774 EPOCH 2 done: loss 24.4682 - lr 0.5000000 +2021-03-26 06:44:38,574 DEV : loss 18.737186431884766 - score 0.6859 +2021-03-26 06:44:38,597 BAD EPOCHS (no improvement): 0 +2021-03-26 06:44:48,335 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:44:50,250 epoch 3 - iter 5/50 - loss 20.64498234 - samples/sec: 83.67 - lr: 0.500000 +2021-03-26 06:44:52,270 epoch 3 - iter 10/50 - loss 19.51538610 - samples/sec: 79.28 - lr: 0.500000 +2021-03-26 06:44:54,281 epoch 3 - iter 15/50 - loss 19.19895941 - samples/sec: 79.62 - lr: 0.500000 +2021-03-26 06:44:56,226 epoch 3 - iter 20/50 - loss 18.74911275 - samples/sec: 82.34 - lr: 0.500000 +2021-03-26 06:44:58,202 epoch 3 - iter 25/50 - loss 18.68923565 - samples/sec: 81.09 - lr: 0.500000 +2021-03-26 06:45:00,559 epoch 3 - iter 30/50 - loss 18.81346890 - samples/sec: 67.93 - lr: 0.500000 +2021-03-26 06:45:02,525 epoch 3 - iter 35/50 - loss 18.32439662 - samples/sec: 81.48 - lr: 0.500000 +2021-03-26 06:45:04,334 epoch 3 - iter 40/50 - loss 17.78430147 - samples/sec: 88.53 - lr: 0.500000 +2021-03-26 06:45:06,254 epoch 3 - iter 45/50 - loss 17.57249622 - samples/sec: 83.38 - lr: 0.500000 +2021-03-26 06:45:08,092 epoch 3 - iter 50/50 - loss 17.63071257 - samples/sec: 87.17 - lr: 0.500000 +2021-03-26 06:45:08,092 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:45:08,093 EPOCH 3 done: loss 17.6307 - lr 0.5000000 +2021-03-26 06:45:08,864 DEV : loss 12.816518783569336 - score 0.7775 +2021-03-26 06:45:08,882 BAD EPOCHS (no improvement): 0 +2021-03-26 06:45:18,098 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:45:19,934 epoch 4 - iter 5/50 - loss 13.45718422 - samples/sec: 87.25 - lr: 0.500000 +2021-03-26 06:45:21,852 epoch 4 - iter 10/50 - loss 13.95723553 - samples/sec: 83.54 - lr: 0.500000 +2021-03-26 06:45:23,589 epoch 4 - iter 15/50 - loss 14.16512273 - samples/sec: 92.21 - lr: 0.500000 +2021-03-26 06:45:25,528 epoch 4 - iter 20/50 - loss 14.24711242 - samples/sec: 82.61 - lr: 0.500000 +2021-03-26 06:45:27,528 epoch 4 - iter 25/50 - loss 14.34868843 - samples/sec: 80.08 - lr: 0.500000 +2021-03-26 06:45:29,367 epoch 4 - iter 30/50 - loss 14.10320505 - samples/sec: 87.10 - lr: 0.500000 +2021-03-26 06:45:31,207 epoch 4 - iter 35/50 - loss 14.21685265 - samples/sec: 87.03 - lr: 0.500000 +2021-03-26 06:45:33,402 epoch 4 - iter 40/50 - loss 14.32871656 - samples/sec: 72.95 - lr: 0.500000 +2021-03-26 06:45:35,550 epoch 4 - iter 45/50 - loss 14.40581788 - samples/sec: 74.58 - lr: 0.500000 +2021-03-26 06:45:37,423 epoch 4 - iter 50/50 - loss 14.52564522 - samples/sec: 85.52 - lr: 0.500000 +2021-03-26 06:45:37,424 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:45:37,425 EPOCH 4 done: loss 14.5256 - lr 0.5000000 +2021-03-26 06:45:38,232 DEV : loss 10.77255630493164 - score 0.8089 +2021-03-26 06:45:38,256 BAD EPOCHS (no improvement): 0 +2021-03-26 06:45:48,051 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:45:50,089 epoch 5 - iter 5/50 - loss 13.15967064 - samples/sec: 78.60 - lr: 0.500000 +2021-03-26 06:45:51,982 epoch 5 - iter 10/50 - loss 12.23003578 - samples/sec: 84.64 - lr: 0.500000 +2021-03-26 06:45:53,973 epoch 5 - iter 15/50 - loss 11.92497025 - samples/sec: 80.43 - lr: 0.500000 +2021-03-26 06:45:55,948 epoch 5 - iter 20/50 - loss 11.61746993 - samples/sec: 81.11 - lr: 0.500000 +2021-03-26 06:45:57,947 epoch 5 - iter 25/50 - loss 11.85876007 - samples/sec: 80.12 - lr: 0.500000 +2021-03-26 06:45:59,818 epoch 5 - iter 30/50 - loss 11.88168329 - samples/sec: 85.57 - lr: 0.500000 +2021-03-26 06:46:01,790 epoch 5 - iter 35/50 - loss 12.09282556 - samples/sec: 81.23 - lr: 0.500000 +2021-03-26 06:46:03,808 epoch 5 - iter 40/50 - loss 12.18096049 - samples/sec: 79.51 - lr: 0.500000 +2021-03-26 06:46:05,740 epoch 5 - iter 45/50 - loss 12.18161873 - samples/sec: 82.88 - lr: 0.500000 +2021-03-26 06:46:07,736 epoch 5 - iter 50/50 - loss 12.94767559 - samples/sec: 80.23 - lr: 0.500000 +2021-03-26 06:46:07,737 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:46:07,737 EPOCH 5 done: loss 12.9477 - lr 0.5000000 +2021-03-26 06:46:08,529 DEV : loss 10.317449569702148 - score 0.8217 +2021-03-26 06:46:08,554 BAD EPOCHS (no improvement): 0 +2021-03-26 06:46:18,018 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:46:19,983 epoch 6 - iter 5/50 - loss 12.50278988 - samples/sec: 81.55 - lr: 0.500000 +2021-03-26 06:46:22,231 epoch 6 - iter 10/50 - loss 11.03866415 - samples/sec: 71.24 - lr: 0.500000 +2021-03-26 06:46:24,657 epoch 6 - iter 15/50 - loss 10.50042890 - samples/sec: 66.02 - lr: 0.500000 +2021-03-26 06:46:26,706 epoch 6 - iter 20/50 - loss 10.49348035 - samples/sec: 78.15 - lr: 0.500000 +2021-03-26 06:46:28,699 epoch 6 - iter 25/50 - loss 10.91942196 - samples/sec: 80.34 - lr: 0.500000 +2021-03-26 06:46:30,795 epoch 6 - iter 30/50 - loss 11.12543367 - samples/sec: 76.40 - lr: 0.500000 +2021-03-26 06:46:32,643 epoch 6 - iter 35/50 - loss 11.14839009 - samples/sec: 86.70 - lr: 0.500000 +2021-03-26 06:46:34,602 epoch 6 - iter 40/50 - loss 11.11419473 - samples/sec: 81.76 - lr: 0.500000 +2021-03-26 06:46:36,637 epoch 6 - iter 45/50 - loss 10.98261443 - samples/sec: 78.68 - lr: 0.500000 +2021-03-26 06:46:38,625 epoch 6 - iter 50/50 - loss 11.21462681 - samples/sec: 80.55 - lr: 0.500000 +2021-03-26 06:46:38,626 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:46:38,626 EPOCH 6 done: loss 11.2146 - lr 0.5000000 +2021-03-26 06:46:39,400 DEV : loss 8.676898002624512 - score 0.8514 +2021-03-26 06:46:39,425 BAD EPOCHS (no improvement): 0 +2021-03-26 06:46:48,983 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:46:51,093 epoch 7 - iter 5/50 - loss 8.98820639 - samples/sec: 75.93 - lr: 0.500000 +2021-03-26 06:46:53,091 epoch 7 - iter 10/50 - loss 9.30682864 - samples/sec: 80.12 - lr: 0.500000 +2021-03-26 06:46:55,331 epoch 7 - iter 15/50 - loss 9.97786722 - samples/sec: 71.49 - lr: 0.500000 +2021-03-26 06:46:57,801 epoch 7 - iter 20/50 - loss 10.29002182 - samples/sec: 64.82 - lr: 0.500000 +2021-03-26 06:47:00,198 epoch 7 - iter 25/50 - loss 10.03100792 - samples/sec: 66.82 - lr: 0.500000 +2021-03-26 06:47:02,050 epoch 7 - iter 30/50 - loss 10.03738489 - samples/sec: 86.44 - lr: 0.500000 +2021-03-26 06:47:04,092 epoch 7 - iter 35/50 - loss 9.97502296 - samples/sec: 78.43 - lr: 0.500000 +2021-03-26 06:47:06,103 epoch 7 - iter 40/50 - loss 9.96401833 - samples/sec: 79.67 - lr: 0.500000 +2021-03-26 06:47:08,005 epoch 7 - iter 45/50 - loss 10.01933646 - samples/sec: 84.21 - lr: 0.500000 +2021-03-26 06:47:10,007 epoch 7 - iter 50/50 - loss 9.98463810 - samples/sec: 80.01 - lr: 0.500000 +2021-03-26 06:47:10,008 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:47:10,008 EPOCH 7 done: loss 9.9846 - lr 0.5000000 +2021-03-26 06:47:10,791 DEV : loss 8.427385330200195 - score 0.8597 +2021-03-26 06:47:10,815 BAD EPOCHS (no improvement): 0 +2021-03-26 06:47:20,332 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:47:22,127 epoch 8 - iter 5/50 - loss 8.90783539 - samples/sec: 89.29 - lr: 0.500000 +2021-03-26 06:47:23,991 epoch 8 - iter 10/50 - loss 9.33205786 - samples/sec: 85.93 - lr: 0.500000 +2021-03-26 06:47:25,895 epoch 8 - iter 15/50 - loss 9.32208843 - samples/sec: 84.11 - lr: 0.500000 +2021-03-26 06:47:27,863 epoch 8 - iter 20/50 - loss 9.05227771 - samples/sec: 81.33 - lr: 0.500000 +2021-03-26 06:47:29,891 epoch 8 - iter 25/50 - loss 9.22336864 - samples/sec: 78.98 - lr: 0.500000 +2021-03-26 06:47:31,740 epoch 8 - iter 30/50 - loss 9.01553138 - samples/sec: 86.64 - lr: 0.500000 +2021-03-26 06:47:33,679 epoch 8 - iter 35/50 - loss 9.00514094 - samples/sec: 82.61 - lr: 0.500000 +2021-03-26 06:47:35,753 epoch 8 - iter 40/50 - loss 9.07460406 - samples/sec: 77.23 - lr: 0.500000 +2021-03-26 06:47:37,789 epoch 8 - iter 45/50 - loss 9.12630397 - samples/sec: 78.63 - lr: 0.500000 +2021-03-26 06:47:39,659 epoch 8 - iter 50/50 - loss 9.14114470 - samples/sec: 85.65 - lr: 0.500000 +2021-03-26 06:47:39,660 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:47:39,660 EPOCH 8 done: loss 9.1411 - lr 0.5000000 +2021-03-26 06:47:40,404 DEV : loss 7.99708890914917 - score 0.8648 +2021-03-26 06:47:40,429 BAD EPOCHS (no improvement): 0 +2021-03-26 06:47:50,043 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:47:52,084 epoch 9 - iter 5/50 - loss 9.45290432 - samples/sec: 78.48 - lr: 0.500000 +2021-03-26 06:47:54,091 epoch 9 - iter 10/50 - loss 8.74829707 - samples/sec: 79.82 - lr: 0.500000 +2021-03-26 06:47:56,046 epoch 9 - iter 15/50 - loss 8.56488609 - samples/sec: 81.90 - lr: 0.500000 +2021-03-26 06:47:58,411 epoch 9 - iter 20/50 - loss 8.78860641 - samples/sec: 67.72 - lr: 0.500000 +2021-03-26 06:48:00,279 epoch 9 - iter 25/50 - loss 8.74139832 - samples/sec: 85.76 - lr: 0.500000 +2021-03-26 06:48:02,319 epoch 9 - iter 30/50 - loss 8.76814893 - samples/sec: 78.51 - lr: 0.500000 +2021-03-26 06:48:04,117 epoch 9 - iter 35/50 - loss 8.67930719 - samples/sec: 89.07 - lr: 0.500000 +2021-03-26 06:48:06,072 epoch 9 - iter 40/50 - loss 8.66237577 - samples/sec: 81.97 - lr: 0.500000 +2021-03-26 06:48:07,905 epoch 9 - iter 45/50 - loss 8.61329811 - samples/sec: 87.33 - lr: 0.500000 +2021-03-26 06:48:09,671 epoch 9 - iter 50/50 - loss 8.58023993 - samples/sec: 90.74 - lr: 0.500000 +2021-03-26 06:48:09,672 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:48:09,672 EPOCH 9 done: loss 8.5802 - lr 0.5000000 +2021-03-26 06:48:10,497 DEV : loss 7.6555399894714355 - score 0.8727 +2021-03-26 06:48:10,527 BAD EPOCHS (no improvement): 0 +2021-03-26 06:48:20,047 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:48:21,871 epoch 10 - iter 5/50 - loss 8.95939608 - samples/sec: 87.86 - lr: 0.500000 +2021-03-26 06:48:23,800 epoch 10 - iter 10/50 - loss 8.59934626 - samples/sec: 83.01 - lr: 0.500000 +2021-03-26 06:48:25,650 epoch 10 - iter 15/50 - loss 8.84599787 - samples/sec: 86.58 - lr: 0.500000 +2021-03-26 06:48:27,448 epoch 10 - iter 20/50 - loss 8.35713410 - samples/sec: 89.09 - lr: 0.500000 +2021-03-26 06:48:29,400 epoch 10 - iter 25/50 - loss 8.28440033 - samples/sec: 82.04 - lr: 0.500000 +2021-03-26 06:48:31,451 epoch 10 - iter 30/50 - loss 8.36959931 - samples/sec: 78.07 - lr: 0.500000 +2021-03-26 06:48:33,492 epoch 10 - iter 35/50 - loss 8.35715817 - samples/sec: 78.48 - lr: 0.500000 +2021-03-26 06:48:35,355 epoch 10 - iter 40/50 - loss 8.29464904 - samples/sec: 85.95 - lr: 0.500000 +2021-03-26 06:48:37,289 epoch 10 - iter 45/50 - loss 8.22910577 - samples/sec: 82.80 - lr: 0.500000 +2021-03-26 06:48:39,187 epoch 10 - iter 50/50 - loss 8.30617043 - samples/sec: 84.38 - lr: 0.500000 +2021-03-26 06:48:39,188 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:48:39,188 EPOCH 10 done: loss 8.3062 - lr 0.5000000 +2021-03-26 06:48:39,977 DEV : loss 7.696311950683594 - score 0.8707 +2021-03-26 06:48:40,000 BAD EPOCHS (no improvement): 1 +2021-03-26 06:48:40,001 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:48:42,095 epoch 11 - iter 5/50 - loss 7.56332026 - samples/sec: 76.46 - lr: 0.500000 +2021-03-26 06:48:44,126 epoch 11 - iter 10/50 - loss 8.00237746 - samples/sec: 78.86 - lr: 0.500000 +2021-03-26 06:48:46,114 epoch 11 - iter 15/50 - loss 7.75435925 - samples/sec: 80.55 - lr: 0.500000 +2021-03-26 06:48:48,167 epoch 11 - iter 20/50 - loss 7.90142670 - samples/sec: 78.02 - lr: 0.500000 +2021-03-26 06:48:50,158 epoch 11 - iter 25/50 - loss 7.71243017 - samples/sec: 80.45 - lr: 0.500000 +2021-03-26 06:48:52,086 epoch 11 - iter 30/50 - loss 7.78967923 - samples/sec: 83.08 - lr: 0.500000 +2021-03-26 06:48:54,002 epoch 11 - iter 35/50 - loss 7.78273422 - samples/sec: 83.57 - lr: 0.500000 +2021-03-26 06:48:55,890 epoch 11 - iter 40/50 - loss 7.72826921 - samples/sec: 84.83 - lr: 0.500000 +2021-03-26 06:48:57,931 epoch 11 - iter 45/50 - loss 7.82425939 - samples/sec: 78.46 - lr: 0.500000 +2021-03-26 06:48:59,678 epoch 11 - iter 50/50 - loss 7.68138412 - samples/sec: 91.73 - lr: 0.500000 +2021-03-26 06:48:59,679 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:48:59,679 EPOCH 11 done: loss 7.6814 - lr 0.5000000 +2021-03-26 06:49:00,450 DEV : loss 7.157756805419922 - score 0.8807 +2021-03-26 06:49:00,473 BAD EPOCHS (no improvement): 0 +2021-03-26 06:49:10,115 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:49:12,177 epoch 12 - iter 5/50 - loss 7.84549236 - samples/sec: 77.68 - lr: 0.500000 +2021-03-26 06:49:14,071 epoch 12 - iter 10/50 - loss 7.50635629 - samples/sec: 84.58 - lr: 0.500000 +2021-03-26 06:49:16,092 epoch 12 - iter 15/50 - loss 7.29152123 - samples/sec: 79.19 - lr: 0.500000 +2021-03-26 06:49:18,041 epoch 12 - iter 20/50 - loss 7.36902924 - samples/sec: 82.19 - lr: 0.500000 +2021-03-26 06:49:19,999 epoch 12 - iter 25/50 - loss 7.43501221 - samples/sec: 81.82 - lr: 0.500000 +2021-03-26 06:49:21,872 epoch 12 - iter 30/50 - loss 7.38531418 - samples/sec: 85.49 - lr: 0.500000 +2021-03-26 06:49:23,871 epoch 12 - iter 35/50 - loss 7.36810846 - samples/sec: 80.11 - lr: 0.500000 +2021-03-26 06:49:25,918 epoch 12 - iter 40/50 - loss 7.36435213 - samples/sec: 78.21 - lr: 0.500000 +2021-03-26 06:49:27,970 epoch 12 - iter 45/50 - loss 7.35478680 - samples/sec: 78.03 - lr: 0.500000 +2021-03-26 06:49:29,796 epoch 12 - iter 50/50 - loss 7.20360278 - samples/sec: 87.74 - lr: 0.500000 +2021-03-26 06:49:29,796 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:49:29,797 EPOCH 12 done: loss 7.2036 - lr 0.5000000 +2021-03-26 06:49:30,670 DEV : loss 7.709672451019287 - score 0.865 +2021-03-26 06:49:30,687 BAD EPOCHS (no improvement): 1 +2021-03-26 06:49:30,688 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:49:32,483 epoch 13 - iter 5/50 - loss 6.63320417 - samples/sec: 89.24 - lr: 0.500000 +2021-03-26 06:49:34,652 epoch 13 - iter 10/50 - loss 6.21271782 - samples/sec: 73.83 - lr: 0.500000 +2021-03-26 06:49:36,669 epoch 13 - iter 15/50 - loss 6.43153175 - samples/sec: 79.41 - lr: 0.500000 +2021-03-26 06:49:38,621 epoch 13 - iter 20/50 - loss 6.54766500 - samples/sec: 82.03 - lr: 0.500000 +2021-03-26 06:49:40,449 epoch 13 - iter 25/50 - loss 6.75074554 - samples/sec: 87.65 - lr: 0.500000 +2021-03-26 06:49:42,286 epoch 13 - iter 30/50 - loss 6.62655284 - samples/sec: 87.21 - lr: 0.500000 +2021-03-26 06:49:44,322 epoch 13 - iter 35/50 - loss 6.64460591 - samples/sec: 78.65 - lr: 0.500000 +2021-03-26 06:49:46,308 epoch 13 - iter 40/50 - loss 6.84230437 - samples/sec: 80.70 - lr: 0.500000 +2021-03-26 06:49:48,240 epoch 13 - iter 45/50 - loss 6.82575455 - samples/sec: 82.86 - lr: 0.500000 +2021-03-26 06:49:50,119 epoch 13 - iter 50/50 - loss 6.94763111 - samples/sec: 85.25 - lr: 0.500000 +2021-03-26 06:49:50,120 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:49:50,120 EPOCH 13 done: loss 6.9476 - lr 0.5000000 +2021-03-26 06:49:50,944 DEV : loss 6.670480728149414 - score 0.8864 +2021-03-26 06:49:50,968 BAD EPOCHS (no improvement): 0 +2021-03-26 06:50:00,535 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:50:02,378 epoch 14 - iter 5/50 - loss 6.48247395 - samples/sec: 86.92 - lr: 0.500000 +2021-03-26 06:50:04,288 epoch 14 - iter 10/50 - loss 6.92508960 - samples/sec: 83.88 - lr: 0.500000 +2021-03-26 06:50:06,091 epoch 14 - iter 15/50 - loss 6.34038912 - samples/sec: 88.81 - lr: 0.500000 +2021-03-26 06:50:07,901 epoch 14 - iter 20/50 - loss 6.43623948 - samples/sec: 88.48 - lr: 0.500000 +2021-03-26 06:50:09,764 epoch 14 - iter 25/50 - loss 6.37658873 - samples/sec: 85.99 - lr: 0.500000 +2021-03-26 06:50:11,878 epoch 14 - iter 30/50 - loss 6.38693593 - samples/sec: 75.71 - lr: 0.500000 +2021-03-26 06:50:13,785 epoch 14 - iter 35/50 - loss 6.41976604 - samples/sec: 83.97 - lr: 0.500000 +2021-03-26 06:50:15,822 epoch 14 - iter 40/50 - loss 6.63060038 - samples/sec: 78.63 - lr: 0.500000 +2021-03-26 06:50:17,789 epoch 14 - iter 45/50 - loss 6.71217939 - samples/sec: 81.41 - lr: 0.500000 +2021-03-26 06:50:19,565 epoch 14 - iter 50/50 - loss 6.62694312 - samples/sec: 90.22 - lr: 0.500000 +2021-03-26 06:50:19,566 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:50:19,566 EPOCH 14 done: loss 6.6269 - lr 0.5000000 +2021-03-26 06:50:20,308 DEV : loss 6.7144269943237305 - score 0.8922 +2021-03-26 06:50:20,333 BAD EPOCHS (no improvement): 0 +2021-03-26 06:50:29,818 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:50:31,598 epoch 15 - iter 5/50 - loss 6.74751863 - samples/sec: 90.04 - lr: 0.500000 +2021-03-26 06:50:33,500 epoch 15 - iter 10/50 - loss 6.18601270 - samples/sec: 84.22 - lr: 0.500000 +2021-03-26 06:50:35,390 epoch 15 - iter 15/50 - loss 6.23807548 - samples/sec: 84.73 - lr: 0.500000 +2021-03-26 06:50:37,241 epoch 15 - iter 20/50 - loss 6.25633945 - samples/sec: 86.55 - lr: 0.500000 +2021-03-26 06:50:39,233 epoch 15 - iter 25/50 - loss 6.15654793 - samples/sec: 80.42 - lr: 0.500000 +2021-03-26 06:50:41,173 epoch 15 - iter 30/50 - loss 6.16274056 - samples/sec: 82.55 - lr: 0.500000 +2021-03-26 06:50:43,216 epoch 15 - iter 35/50 - loss 6.34539483 - samples/sec: 78.39 - lr: 0.500000 +2021-03-26 06:50:45,216 epoch 15 - iter 40/50 - loss 6.36484003 - samples/sec: 80.09 - lr: 0.500000 +2021-03-26 06:50:47,143 epoch 15 - iter 45/50 - loss 6.34316796 - samples/sec: 83.10 - lr: 0.500000 +2021-03-26 06:50:49,115 epoch 15 - iter 50/50 - loss 6.36780796 - samples/sec: 81.21 - lr: 0.500000 +2021-03-26 06:50:49,116 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:50:49,116 EPOCH 15 done: loss 6.3678 - lr 0.5000000 +2021-03-26 06:50:49,900 DEV : loss 7.053887367248535 - score 0.8894 +2021-03-26 06:50:49,929 BAD EPOCHS (no improvement): 1 +2021-03-26 06:50:49,930 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:50:51,939 epoch 16 - iter 5/50 - loss 6.18410025 - samples/sec: 79.73 - lr: 0.500000 +2021-03-26 06:50:53,856 epoch 16 - iter 10/50 - loss 6.26023331 - samples/sec: 83.53 - lr: 0.500000 +2021-03-26 06:50:55,840 epoch 16 - iter 15/50 - loss 5.68440321 - samples/sec: 80.70 - lr: 0.500000 +2021-03-26 06:50:57,899 epoch 16 - iter 20/50 - loss 5.82707292 - samples/sec: 77.77 - lr: 0.500000 +2021-03-26 06:50:59,847 epoch 16 - iter 25/50 - loss 6.05982093 - samples/sec: 82.24 - lr: 0.500000 +2021-03-26 06:51:01,744 epoch 16 - iter 30/50 - loss 5.95639232 - samples/sec: 84.44 - lr: 0.500000 +2021-03-26 06:51:03,599 epoch 16 - iter 35/50 - loss 5.98666367 - samples/sec: 86.33 - lr: 0.500000 +2021-03-26 06:51:05,575 epoch 16 - iter 40/50 - loss 5.99230914 - samples/sec: 81.05 - lr: 0.500000 +2021-03-26 06:51:07,615 epoch 16 - iter 45/50 - loss 6.01807009 - samples/sec: 78.50 - lr: 0.500000 +2021-03-26 06:51:09,435 epoch 16 - iter 50/50 - loss 6.09721210 - samples/sec: 87.97 - lr: 0.500000 +2021-03-26 06:51:09,436 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:51:09,436 EPOCH 16 done: loss 6.0972 - lr 0.5000000 +2021-03-26 06:51:10,229 DEV : loss 6.481603145599365 - score 0.8997 +2021-03-26 06:51:10,255 BAD EPOCHS (no improvement): 0 +2021-03-26 06:51:19,805 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:51:21,992 epoch 17 - iter 5/50 - loss 5.81198111 - samples/sec: 73.24 - lr: 0.500000 +2021-03-26 06:51:23,985 epoch 17 - iter 10/50 - loss 5.60492916 - samples/sec: 80.39 - lr: 0.500000 +2021-03-26 06:51:25,926 epoch 17 - iter 15/50 - loss 5.65811930 - samples/sec: 82.51 - lr: 0.500000 +2021-03-26 06:51:27,982 epoch 17 - iter 20/50 - loss 5.58428044 - samples/sec: 77.92 - lr: 0.500000 +2021-03-26 06:51:30,310 epoch 17 - iter 25/50 - loss 5.61901649 - samples/sec: 68.79 - lr: 0.500000 +2021-03-26 06:51:32,473 epoch 17 - iter 30/50 - loss 5.74076101 - samples/sec: 74.03 - lr: 0.500000 +2021-03-26 06:51:34,300 epoch 17 - iter 35/50 - loss 5.79365714 - samples/sec: 87.69 - lr: 0.500000 +2021-03-26 06:51:36,214 epoch 17 - iter 40/50 - loss 5.76530095 - samples/sec: 83.67 - lr: 0.500000 +2021-03-26 06:51:38,208 epoch 17 - iter 45/50 - loss 5.71586393 - samples/sec: 80.31 - lr: 0.500000 +2021-03-26 06:51:40,087 epoch 17 - iter 50/50 - loss 5.81929769 - samples/sec: 85.23 - lr: 0.500000 +2021-03-26 06:51:40,088 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:51:40,089 EPOCH 17 done: loss 5.8193 - lr 0.5000000 +2021-03-26 06:51:40,855 DEV : loss 6.675870418548584 - score 0.8979 +2021-03-26 06:51:40,876 BAD EPOCHS (no improvement): 1 +2021-03-26 06:51:40,877 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:51:42,688 epoch 18 - iter 5/50 - loss 4.84788232 - samples/sec: 88.44 - lr: 0.500000 +2021-03-26 06:51:44,686 epoch 18 - iter 10/50 - loss 5.07229483 - samples/sec: 80.12 - lr: 0.500000 +2021-03-26 06:51:46,638 epoch 18 - iter 15/50 - loss 5.39524067 - samples/sec: 82.06 - lr: 0.500000 +2021-03-26 06:51:48,557 epoch 18 - iter 20/50 - loss 5.47071220 - samples/sec: 83.45 - lr: 0.500000 +2021-03-26 06:51:50,552 epoch 18 - iter 25/50 - loss 5.63781402 - samples/sec: 80.30 - lr: 0.500000 +2021-03-26 06:51:52,695 epoch 18 - iter 30/50 - loss 5.65345579 - samples/sec: 74.70 - lr: 0.500000 +2021-03-26 06:51:54,590 epoch 18 - iter 35/50 - loss 5.74147837 - samples/sec: 84.53 - lr: 0.500000 +2021-03-26 06:51:56,651 epoch 18 - iter 40/50 - loss 5.74021587 - samples/sec: 77.71 - lr: 0.500000 +2021-03-26 06:51:58,886 epoch 18 - iter 45/50 - loss 5.64183188 - samples/sec: 71.65 - lr: 0.500000 +2021-03-26 06:52:00,848 epoch 18 - iter 50/50 - loss 5.63788223 - samples/sec: 81.61 - lr: 0.500000 +2021-03-26 06:52:00,849 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:52:00,849 EPOCH 18 done: loss 5.6379 - lr 0.5000000 +2021-03-26 06:52:01,630 DEV : loss 6.928553581237793 - score 0.8888 +2021-03-26 06:52:01,655 BAD EPOCHS (no improvement): 2 +2021-03-26 06:52:01,656 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:52:03,675 epoch 19 - iter 5/50 - loss 6.10535774 - samples/sec: 79.36 - lr: 0.500000 +2021-03-26 06:52:05,646 epoch 19 - iter 10/50 - loss 5.71634462 - samples/sec: 81.23 - lr: 0.500000 +2021-03-26 06:52:07,554 epoch 19 - iter 15/50 - loss 5.74245432 - samples/sec: 83.90 - lr: 0.500000 +2021-03-26 06:52:09,566 epoch 19 - iter 20/50 - loss 5.63408655 - samples/sec: 79.61 - lr: 0.500000 +2021-03-26 06:52:11,580 epoch 19 - iter 25/50 - loss 5.59964517 - samples/sec: 79.52 - lr: 0.500000 +2021-03-26 06:52:13,424 epoch 19 - iter 30/50 - loss 5.48802720 - samples/sec: 86.89 - lr: 0.500000 +2021-03-26 06:52:15,428 epoch 19 - iter 35/50 - loss 5.43323846 - samples/sec: 79.91 - lr: 0.500000 +2021-03-26 06:52:17,222 epoch 19 - iter 40/50 - loss 5.37883350 - samples/sec: 89.29 - lr: 0.500000 +2021-03-26 06:52:19,185 epoch 19 - iter 45/50 - loss 5.49508470 - samples/sec: 81.58 - lr: 0.500000 +2021-03-26 06:52:21,043 epoch 19 - iter 50/50 - loss 5.50686664 - samples/sec: 86.21 - lr: 0.500000 +2021-03-26 06:52:21,044 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:52:21,045 EPOCH 19 done: loss 5.5069 - lr 0.5000000 +2021-03-26 06:52:21,840 DEV : loss 6.811120510101318 - score 0.8956 +2021-03-26 06:52:21,864 BAD EPOCHS (no improvement): 3 +2021-03-26 06:52:21,865 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:52:23,719 epoch 20 - iter 5/50 - loss 5.16742334 - samples/sec: 86.43 - lr: 0.500000 +2021-03-26 06:52:25,647 epoch 20 - iter 10/50 - loss 5.16280818 - samples/sec: 83.07 - lr: 0.500000 +2021-03-26 06:52:27,586 epoch 20 - iter 15/50 - loss 5.03337404 - samples/sec: 82.58 - lr: 0.500000 +2021-03-26 06:52:29,544 epoch 20 - iter 20/50 - loss 4.95772830 - samples/sec: 81.79 - lr: 0.500000 +2021-03-26 06:52:31,663 epoch 20 - iter 25/50 - loss 5.16223987 - samples/sec: 75.59 - lr: 0.500000 +2021-03-26 06:52:33,671 epoch 20 - iter 30/50 - loss 5.12701021 - samples/sec: 79.72 - lr: 0.500000 +2021-03-26 06:52:35,726 epoch 20 - iter 35/50 - loss 5.08359601 - samples/sec: 77.94 - lr: 0.500000 +2021-03-26 06:52:37,891 epoch 20 - iter 40/50 - loss 5.22774757 - samples/sec: 73.97 - lr: 0.500000 +2021-03-26 06:52:39,850 epoch 20 - iter 45/50 - loss 5.34838710 - samples/sec: 81.75 - lr: 0.500000 +2021-03-26 06:52:41,877 epoch 20 - iter 50/50 - loss 5.35895552 - samples/sec: 78.97 - lr: 0.500000 +2021-03-26 06:52:41,878 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:52:41,878 EPOCH 20 done: loss 5.3590 - lr 0.5000000 +2021-03-26 06:52:42,646 DEV : loss 6.63265323638916 - score 0.8984 +2021-03-26 06:52:42,670 BAD EPOCHS (no improvement): 4 +2021-03-26 06:52:42,671 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:52:44,695 epoch 21 - iter 5/50 - loss 4.11205826 - samples/sec: 79.12 - lr: 0.250000 +2021-03-26 06:52:46,598 epoch 21 - iter 10/50 - loss 4.39735847 - samples/sec: 84.14 - lr: 0.250000 +2021-03-26 06:52:48,690 epoch 21 - iter 15/50 - loss 4.48490636 - samples/sec: 76.57 - lr: 0.250000 +2021-03-26 06:52:50,870 epoch 21 - iter 20/50 - loss 4.59468763 - samples/sec: 73.44 - lr: 0.250000 +2021-03-26 06:52:52,921 epoch 21 - iter 25/50 - loss 4.42129483 - samples/sec: 78.08 - lr: 0.250000 +2021-03-26 06:52:55,056 epoch 21 - iter 30/50 - loss 4.40010382 - samples/sec: 75.02 - lr: 0.250000 +2021-03-26 06:52:57,027 epoch 21 - iter 35/50 - loss 4.41746838 - samples/sec: 81.27 - lr: 0.250000 +2021-03-26 06:52:59,240 epoch 21 - iter 40/50 - loss 4.45754075 - samples/sec: 72.35 - lr: 0.250000 +2021-03-26 06:53:01,148 epoch 21 - iter 45/50 - loss 4.47268793 - samples/sec: 83.94 - lr: 0.250000 +2021-03-26 06:53:03,087 epoch 21 - iter 50/50 - loss 4.48687317 - samples/sec: 82.61 - lr: 0.250000 +2021-03-26 06:53:03,088 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:53:03,088 EPOCH 21 done: loss 4.4869 - lr 0.2500000 +2021-03-26 06:53:03,846 DEV : loss 6.153275966644287 - score 0.9111 +2021-03-26 06:53:03,870 BAD EPOCHS (no improvement): 0 +2021-03-26 06:53:13,292 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:53:15,249 epoch 22 - iter 5/50 - loss 4.61283131 - samples/sec: 81.87 - lr: 0.250000 +2021-03-26 06:53:17,388 epoch 22 - iter 10/50 - loss 4.34403934 - samples/sec: 74.88 - lr: 0.250000 +2021-03-26 06:53:19,318 epoch 22 - iter 15/50 - loss 4.35565983 - samples/sec: 82.97 - lr: 0.250000 +2021-03-26 06:53:21,313 epoch 22 - iter 20/50 - loss 4.36567014 - samples/sec: 80.28 - lr: 0.250000 +2021-03-26 06:53:23,246 epoch 22 - iter 25/50 - loss 4.31828299 - samples/sec: 82.83 - lr: 0.250000 +2021-03-26 06:53:25,194 epoch 22 - iter 30/50 - loss 4.21510568 - samples/sec: 82.22 - lr: 0.250000 +2021-03-26 06:53:27,255 epoch 22 - iter 35/50 - loss 4.25309601 - samples/sec: 77.73 - lr: 0.250000 +2021-03-26 06:53:29,444 epoch 22 - iter 40/50 - loss 4.32016158 - samples/sec: 73.13 - lr: 0.250000 +2021-03-26 06:53:31,403 epoch 22 - iter 45/50 - loss 4.35405540 - samples/sec: 81.75 - lr: 0.250000 +2021-03-26 06:53:33,192 epoch 22 - iter 50/50 - loss 4.37461680 - samples/sec: 89.57 - lr: 0.250000 +2021-03-26 06:53:33,192 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:53:33,193 EPOCH 22 done: loss 4.3746 - lr 0.2500000 +2021-03-26 06:53:34,000 DEV : loss 6.201013565063477 - score 0.9111 +2021-03-26 06:53:34,026 BAD EPOCHS (no improvement): 1 +2021-03-26 06:53:34,026 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:53:35,966 epoch 23 - iter 5/50 - loss 3.95178127 - samples/sec: 82.75 - lr: 0.250000 +2021-03-26 06:53:37,791 epoch 23 - iter 10/50 - loss 3.83079326 - samples/sec: 87.78 - lr: 0.250000 +2021-03-26 06:53:39,907 epoch 23 - iter 15/50 - loss 3.91210388 - samples/sec: 75.69 - lr: 0.250000 +2021-03-26 06:53:41,936 epoch 23 - iter 20/50 - loss 3.97783825 - samples/sec: 78.94 - lr: 0.250000 +2021-03-26 06:53:43,957 epoch 23 - iter 25/50 - loss 4.08968901 - samples/sec: 79.27 - lr: 0.250000 +2021-03-26 06:53:46,037 epoch 23 - iter 30/50 - loss 4.01160220 - samples/sec: 77.01 - lr: 0.250000 +2021-03-26 06:53:47,974 epoch 23 - iter 35/50 - loss 3.99960697 - samples/sec: 82.66 - lr: 0.250000 +2021-03-26 06:53:50,038 epoch 23 - iter 40/50 - loss 4.04271683 - samples/sec: 77.59 - lr: 0.250000 +2021-03-26 06:53:52,097 epoch 23 - iter 45/50 - loss 4.03870177 - samples/sec: 77.78 - lr: 0.250000 +2021-03-26 06:53:53,915 epoch 23 - iter 50/50 - loss 4.17525750 - samples/sec: 88.07 - lr: 0.250000 +2021-03-26 06:53:53,916 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:53:53,917 EPOCH 23 done: loss 4.1753 - lr 0.2500000 +2021-03-26 06:53:54,707 DEV : loss 6.3719329833984375 - score 0.9071 +2021-03-26 06:53:54,731 BAD EPOCHS (no improvement): 2 +2021-03-26 06:53:54,732 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:53:56,617 epoch 24 - iter 5/50 - loss 3.20824995 - samples/sec: 84.99 - lr: 0.250000 +2021-03-26 06:53:58,559 epoch 24 - iter 10/50 - loss 3.63779755 - samples/sec: 82.49 - lr: 0.250000 +2021-03-26 06:54:00,565 epoch 24 - iter 15/50 - loss 3.66257393 - samples/sec: 79.83 - lr: 0.250000 +2021-03-26 06:54:02,510 epoch 24 - iter 20/50 - loss 3.57795765 - samples/sec: 82.35 - lr: 0.250000 +2021-03-26 06:54:04,570 epoch 24 - iter 25/50 - loss 3.67160597 - samples/sec: 77.74 - lr: 0.250000 +2021-03-26 06:54:06,748 epoch 24 - iter 30/50 - loss 3.83814465 - samples/sec: 73.53 - lr: 0.250000 +2021-03-26 06:54:08,685 epoch 24 - iter 35/50 - loss 3.87231894 - samples/sec: 82.70 - lr: 0.250000 +2021-03-26 06:54:10,597 epoch 24 - iter 40/50 - loss 3.93506041 - samples/sec: 83.75 - lr: 0.250000 +2021-03-26 06:54:12,489 epoch 24 - iter 45/50 - loss 3.92431094 - samples/sec: 84.62 - lr: 0.250000 +2021-03-26 06:54:14,299 epoch 24 - iter 50/50 - loss 3.97243820 - samples/sec: 88.53 - lr: 0.250000 +2021-03-26 06:54:14,300 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:54:14,300 EPOCH 24 done: loss 3.9724 - lr 0.2500000 +2021-03-26 06:54:15,067 DEV : loss 6.205985069274902 - score 0.9077 +2021-03-26 06:54:15,084 BAD EPOCHS (no improvement): 3 +2021-03-26 06:54:15,085 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:54:16,911 epoch 25 - iter 5/50 - loss 3.35670197 - samples/sec: 87.75 - lr: 0.250000 +2021-03-26 06:54:18,825 epoch 25 - iter 10/50 - loss 3.88174726 - samples/sec: 83.65 - lr: 0.250000 +2021-03-26 06:54:20,851 epoch 25 - iter 15/50 - loss 3.94504937 - samples/sec: 79.06 - lr: 0.250000 +2021-03-26 06:54:22,718 epoch 25 - iter 20/50 - loss 3.86929995 - samples/sec: 85.81 - lr: 0.250000 +2021-03-26 06:54:24,663 epoch 25 - iter 25/50 - loss 3.85905517 - samples/sec: 82.31 - lr: 0.250000 +2021-03-26 06:54:26,772 epoch 25 - iter 30/50 - loss 3.91150273 - samples/sec: 75.92 - lr: 0.250000 +2021-03-26 06:54:28,774 epoch 25 - iter 35/50 - loss 3.95813450 - samples/sec: 79.99 - lr: 0.250000 +2021-03-26 06:54:30,700 epoch 25 - iter 40/50 - loss 3.95463861 - samples/sec: 83.15 - lr: 0.250000 +2021-03-26 06:54:32,766 epoch 25 - iter 45/50 - loss 3.94753084 - samples/sec: 77.53 - lr: 0.250000 +2021-03-26 06:54:34,540 epoch 25 - iter 50/50 - loss 3.90389370 - samples/sec: 90.32 - lr: 0.250000 +2021-03-26 06:54:34,541 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:54:34,541 EPOCH 25 done: loss 3.9039 - lr 0.2500000 +2021-03-26 06:54:35,316 DEV : loss 6.084229946136475 - score 0.9119 +2021-03-26 06:54:35,342 BAD EPOCHS (no improvement): 0 +2021-03-26 06:54:44,799 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:54:46,667 epoch 26 - iter 5/50 - loss 3.07638960 - samples/sec: 85.82 - lr: 0.250000 +2021-03-26 06:54:48,604 epoch 26 - iter 10/50 - loss 3.45474722 - samples/sec: 82.70 - lr: 0.250000 +2021-03-26 06:54:50,592 epoch 26 - iter 15/50 - loss 3.80810453 - samples/sec: 80.55 - lr: 0.250000 +2021-03-26 06:54:52,617 epoch 26 - iter 20/50 - loss 3.68002245 - samples/sec: 79.07 - lr: 0.250000 +2021-03-26 06:54:54,533 epoch 26 - iter 25/50 - loss 3.78536674 - samples/sec: 83.60 - lr: 0.250000 +2021-03-26 06:54:56,643 epoch 26 - iter 30/50 - loss 3.72170803 - samples/sec: 75.88 - lr: 0.250000 +2021-03-26 06:54:59,727 epoch 26 - iter 35/50 - loss 3.70023379 - samples/sec: 51.92 - lr: 0.250000 +2021-03-26 06:55:01,693 epoch 26 - iter 40/50 - loss 3.70014390 - samples/sec: 81.46 - lr: 0.250000 +2021-03-26 06:55:03,743 epoch 26 - iter 45/50 - loss 3.71391151 - samples/sec: 78.10 - lr: 0.250000 +2021-03-26 06:55:05,534 epoch 26 - iter 50/50 - loss 3.66467248 - samples/sec: 89.39 - lr: 0.250000 +2021-03-26 06:55:05,535 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:55:05,536 EPOCH 26 done: loss 3.6647 - lr 0.2500000 +2021-03-26 06:55:06,325 DEV : loss 6.3462018966674805 - score 0.9103 +2021-03-26 06:55:06,349 BAD EPOCHS (no improvement): 1 +2021-03-26 06:55:06,350 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:55:08,437 epoch 27 - iter 5/50 - loss 4.35266724 - samples/sec: 76.75 - lr: 0.250000 +2021-03-26 06:55:10,217 epoch 27 - iter 10/50 - loss 3.97600597 - samples/sec: 89.99 - lr: 0.250000 +2021-03-26 06:55:12,205 epoch 27 - iter 15/50 - loss 3.96660554 - samples/sec: 80.57 - lr: 0.250000 +2021-03-26 06:55:14,359 epoch 27 - iter 20/50 - loss 3.75834839 - samples/sec: 74.32 - lr: 0.250000 +2021-03-26 06:55:16,446 epoch 27 - iter 25/50 - loss 3.81192220 - samples/sec: 76.75 - lr: 0.250000 +2021-03-26 06:55:18,343 epoch 27 - iter 30/50 - loss 3.70215019 - samples/sec: 84.40 - lr: 0.250000 +2021-03-26 06:55:20,359 epoch 27 - iter 35/50 - loss 3.66418975 - samples/sec: 79.47 - lr: 0.250000 +2021-03-26 06:55:22,303 epoch 27 - iter 40/50 - loss 3.71305492 - samples/sec: 82.37 - lr: 0.250000 +2021-03-26 06:55:24,188 epoch 27 - iter 45/50 - loss 3.75636695 - samples/sec: 84.99 - lr: 0.250000 +2021-03-26 06:55:26,120 epoch 27 - iter 50/50 - loss 3.79689440 - samples/sec: 82.91 - lr: 0.250000 +2021-03-26 06:55:26,120 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:55:26,121 EPOCH 27 done: loss 3.7969 - lr 0.2500000 +2021-03-26 06:55:26,899 DEV : loss 6.2635931968688965 - score 0.9075 +2021-03-26 06:55:26,924 BAD EPOCHS (no improvement): 2 +2021-03-26 06:55:26,925 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:55:28,807 epoch 28 - iter 5/50 - loss 3.31051455 - samples/sec: 85.09 - lr: 0.250000 +2021-03-26 06:55:30,804 epoch 28 - iter 10/50 - loss 3.48266208 - samples/sec: 80.23 - lr: 0.250000 +2021-03-26 06:55:32,674 epoch 28 - iter 15/50 - loss 3.33151848 - samples/sec: 85.69 - lr: 0.250000 +2021-03-26 06:55:34,657 epoch 28 - iter 20/50 - loss 3.58922390 - samples/sec: 80.72 - lr: 0.250000 +2021-03-26 06:55:36,628 epoch 28 - iter 25/50 - loss 3.52472383 - samples/sec: 81.27 - lr: 0.250000 +2021-03-26 06:55:38,604 epoch 28 - iter 30/50 - loss 3.60845091 - samples/sec: 81.07 - lr: 0.250000 +2021-03-26 06:55:40,671 epoch 28 - iter 35/50 - loss 3.65303677 - samples/sec: 77.51 - lr: 0.250000 +2021-03-26 06:55:42,851 epoch 28 - iter 40/50 - loss 3.65428785 - samples/sec: 73.44 - lr: 0.250000 +2021-03-26 06:55:44,715 epoch 28 - iter 45/50 - loss 3.69842546 - samples/sec: 85.97 - lr: 0.250000 +2021-03-26 06:55:46,756 epoch 28 - iter 50/50 - loss 3.68927022 - samples/sec: 78.46 - lr: 0.250000 +2021-03-26 06:55:46,757 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:55:46,757 EPOCH 28 done: loss 3.6893 - lr 0.2500000 +2021-03-26 06:55:47,558 DEV : loss 6.469142913818359 - score 0.9119 +2021-03-26 06:55:47,578 BAD EPOCHS (no improvement): 3 +2021-03-26 06:55:47,578 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:55:49,695 epoch 29 - iter 5/50 - loss 3.68538837 - samples/sec: 75.63 - lr: 0.250000 +2021-03-26 06:55:51,593 epoch 29 - iter 10/50 - loss 3.49097276 - samples/sec: 84.41 - lr: 0.250000 +2021-03-26 06:55:53,500 epoch 29 - iter 15/50 - loss 3.51349881 - samples/sec: 83.97 - lr: 0.250000 +2021-03-26 06:55:55,462 epoch 29 - iter 20/50 - loss 3.47587315 - samples/sec: 81.63 - lr: 0.250000 +2021-03-26 06:55:57,701 epoch 29 - iter 25/50 - loss 3.52188293 - samples/sec: 71.52 - lr: 0.250000 +2021-03-26 06:55:59,674 epoch 29 - iter 30/50 - loss 3.52639737 - samples/sec: 81.19 - lr: 0.250000 +2021-03-26 06:56:01,531 epoch 29 - iter 35/50 - loss 3.54751299 - samples/sec: 86.20 - lr: 0.250000 +2021-03-26 06:56:03,585 epoch 29 - iter 40/50 - loss 3.59073097 - samples/sec: 77.98 - lr: 0.250000 +2021-03-26 06:56:05,983 epoch 29 - iter 45/50 - loss 3.64443576 - samples/sec: 66.76 - lr: 0.250000 +2021-03-26 06:56:07,729 epoch 29 - iter 50/50 - loss 3.62032253 - samples/sec: 91.72 - lr: 0.250000 +2021-03-26 06:56:07,730 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:56:07,730 EPOCH 29 done: loss 3.6203 - lr 0.2500000 +2021-03-26 06:56:08,499 DEV : loss 6.374938488006592 - score 0.9095 +2021-03-26 06:56:08,524 BAD EPOCHS (no improvement): 4 +2021-03-26 06:56:08,525 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:56:10,462 epoch 30 - iter 5/50 - loss 3.23985057 - samples/sec: 82.68 - lr: 0.125000 +2021-03-26 06:56:12,372 epoch 30 - iter 10/50 - loss 3.15554574 - samples/sec: 83.90 - lr: 0.125000 +2021-03-26 06:56:14,327 epoch 30 - iter 15/50 - loss 3.30265152 - samples/sec: 81.89 - lr: 0.125000 +2021-03-26 06:56:16,296 epoch 30 - iter 20/50 - loss 3.36633397 - samples/sec: 81.35 - lr: 0.125000 +2021-03-26 06:56:18,228 epoch 30 - iter 25/50 - loss 3.36648575 - samples/sec: 82.89 - lr: 0.125000 +2021-03-26 06:56:20,251 epoch 30 - iter 30/50 - loss 3.46024215 - samples/sec: 79.19 - lr: 0.125000 +2021-03-26 06:56:22,427 epoch 30 - iter 35/50 - loss 3.40659271 - samples/sec: 73.62 - lr: 0.125000 +2021-03-26 06:56:24,481 epoch 30 - iter 40/50 - loss 3.42356200 - samples/sec: 77.99 - lr: 0.125000 +2021-03-26 06:56:26,488 epoch 30 - iter 45/50 - loss 3.38766970 - samples/sec: 79.78 - lr: 0.125000 +2021-03-26 06:56:28,404 epoch 30 - iter 50/50 - loss 3.30463873 - samples/sec: 83.57 - lr: 0.125000 +2021-03-26 06:56:28,405 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:56:28,406 EPOCH 30 done: loss 3.3046 - lr 0.1250000 +2021-03-26 06:56:29,163 DEV : loss 6.366065979003906 - score 0.9118 +2021-03-26 06:56:29,180 BAD EPOCHS (no improvement): 1 +2021-03-26 06:56:29,181 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:56:31,012 epoch 31 - iter 5/50 - loss 3.19004385 - samples/sec: 87.48 - lr: 0.125000 +2021-03-26 06:56:32,754 epoch 31 - iter 10/50 - loss 2.96236542 - samples/sec: 91.96 - lr: 0.125000 +2021-03-26 06:56:34,698 epoch 31 - iter 15/50 - loss 3.00070031 - samples/sec: 82.38 - lr: 0.125000 +2021-03-26 06:56:36,535 epoch 31 - iter 20/50 - loss 3.12568991 - samples/sec: 87.15 - lr: 0.125000 +2021-03-26 06:56:38,533 epoch 31 - iter 25/50 - loss 3.18731164 - samples/sec: 80.20 - lr: 0.125000 +2021-03-26 06:56:40,448 epoch 31 - iter 30/50 - loss 3.14024767 - samples/sec: 83.60 - lr: 0.125000 +2021-03-26 06:56:42,443 epoch 31 - iter 35/50 - loss 3.16921918 - samples/sec: 80.28 - lr: 0.125000 +2021-03-26 06:56:44,223 epoch 31 - iter 40/50 - loss 3.11370867 - samples/sec: 90.01 - lr: 0.125000 +2021-03-26 06:56:46,127 epoch 31 - iter 45/50 - loss 3.06799367 - samples/sec: 84.11 - lr: 0.125000 +2021-03-26 06:56:47,960 epoch 31 - iter 50/50 - loss 3.13836832 - samples/sec: 87.41 - lr: 0.125000 +2021-03-26 06:56:47,961 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:56:47,961 EPOCH 31 done: loss 3.1384 - lr 0.1250000 +2021-03-26 06:56:48,724 DEV : loss 6.220050811767578 - score 0.916 +2021-03-26 06:56:48,749 BAD EPOCHS (no improvement): 0 +2021-03-26 06:56:58,306 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:57:00,061 epoch 32 - iter 5/50 - loss 2.85820818 - samples/sec: 91.32 - lr: 0.125000 +2021-03-26 06:57:02,183 epoch 32 - iter 10/50 - loss 3.07046874 - samples/sec: 75.47 - lr: 0.125000 +2021-03-26 06:57:04,132 epoch 32 - iter 15/50 - loss 2.93962463 - samples/sec: 82.19 - lr: 0.125000 +2021-03-26 06:57:05,988 epoch 32 - iter 20/50 - loss 3.03024350 - samples/sec: 86.31 - lr: 0.125000 +2021-03-26 06:57:07,979 epoch 32 - iter 25/50 - loss 3.07685027 - samples/sec: 80.44 - lr: 0.125000 +2021-03-26 06:57:09,770 epoch 32 - iter 30/50 - loss 3.12704571 - samples/sec: 89.43 - lr: 0.125000 +2021-03-26 06:57:11,865 epoch 32 - iter 35/50 - loss 3.12945336 - samples/sec: 76.44 - lr: 0.125000 +2021-03-26 06:57:13,974 epoch 32 - iter 40/50 - loss 3.21813447 - samples/sec: 75.92 - lr: 0.125000 +2021-03-26 06:57:16,025 epoch 32 - iter 45/50 - loss 3.22640948 - samples/sec: 78.08 - lr: 0.125000 +2021-03-26 06:57:17,793 epoch 32 - iter 50/50 - loss 3.19685128 - samples/sec: 90.64 - lr: 0.125000 +2021-03-26 06:57:17,794 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:57:17,794 EPOCH 32 done: loss 3.1969 - lr 0.1250000 +2021-03-26 06:57:18,620 DEV : loss 6.164405822753906 - score 0.918 +2021-03-26 06:57:18,654 BAD EPOCHS (no improvement): 0 +2021-03-26 06:57:28,508 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:57:30,475 epoch 33 - iter 5/50 - loss 2.54982162 - samples/sec: 81.44 - lr: 0.125000 +2021-03-26 06:57:32,367 epoch 33 - iter 10/50 - loss 2.87348752 - samples/sec: 84.61 - lr: 0.125000 +2021-03-26 06:57:34,396 epoch 33 - iter 15/50 - loss 2.85248617 - samples/sec: 78.95 - lr: 0.125000 +2021-03-26 06:57:36,316 epoch 33 - iter 20/50 - loss 2.77764983 - samples/sec: 83.42 - lr: 0.125000 +2021-03-26 06:57:38,474 epoch 33 - iter 25/50 - loss 2.80861367 - samples/sec: 74.17 - lr: 0.125000 +2021-03-26 06:57:40,469 epoch 33 - iter 30/50 - loss 2.89748362 - samples/sec: 80.29 - lr: 0.125000 +2021-03-26 06:57:42,439 epoch 33 - iter 35/50 - loss 2.89348922 - samples/sec: 81.29 - lr: 0.125000 +2021-03-26 06:57:44,478 epoch 33 - iter 40/50 - loss 2.92309944 - samples/sec: 78.55 - lr: 0.125000 +2021-03-26 06:57:46,640 epoch 33 - iter 45/50 - loss 2.87363632 - samples/sec: 74.07 - lr: 0.125000 +2021-03-26 06:57:48,648 epoch 33 - iter 50/50 - loss 2.93893275 - samples/sec: 79.75 - lr: 0.125000 +2021-03-26 06:57:48,649 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:57:48,650 EPOCH 33 done: loss 2.9389 - lr 0.1250000 +2021-03-26 06:57:49,453 DEV : loss 6.186984062194824 - score 0.917 +2021-03-26 06:57:49,475 BAD EPOCHS (no improvement): 1 +2021-03-26 06:57:49,476 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:57:51,745 epoch 34 - iter 5/50 - loss 3.01934357 - samples/sec: 70.54 - lr: 0.125000 +2021-03-26 06:57:53,900 epoch 34 - iter 10/50 - loss 2.74939032 - samples/sec: 74.32 - lr: 0.125000 +2021-03-26 06:57:55,908 epoch 34 - iter 15/50 - loss 2.77931755 - samples/sec: 79.76 - lr: 0.125000 +2021-03-26 06:57:57,846 epoch 34 - iter 20/50 - loss 2.89119451 - samples/sec: 82.65 - lr: 0.125000 +2021-03-26 06:57:59,694 epoch 34 - iter 25/50 - loss 2.82531148 - samples/sec: 86.70 - lr: 0.125000 +2021-03-26 06:58:01,704 epoch 34 - iter 30/50 - loss 2.80726229 - samples/sec: 79.68 - lr: 0.125000 +2021-03-26 06:58:03,539 epoch 34 - iter 35/50 - loss 2.83326924 - samples/sec: 87.27 - lr: 0.125000 +2021-03-26 06:58:05,540 epoch 34 - iter 40/50 - loss 2.87008840 - samples/sec: 80.07 - lr: 0.125000 +2021-03-26 06:58:07,448 epoch 34 - iter 45/50 - loss 2.94134395 - samples/sec: 83.94 - lr: 0.125000 +2021-03-26 06:58:09,333 epoch 34 - iter 50/50 - loss 3.07199399 - samples/sec: 84.96 - lr: 0.125000 +2021-03-26 06:58:09,334 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:58:09,334 EPOCH 34 done: loss 3.0720 - lr 0.1250000 +2021-03-26 06:58:10,124 DEV : loss 6.177824974060059 - score 0.9192 +2021-03-26 06:58:10,148 BAD EPOCHS (no improvement): 0 +2021-03-26 06:58:19,752 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:58:21,539 epoch 35 - iter 5/50 - loss 3.17444773 - samples/sec: 89.67 - lr: 0.125000 +2021-03-26 06:58:23,452 epoch 35 - iter 10/50 - loss 3.35550301 - samples/sec: 83.71 - lr: 0.125000 +2021-03-26 06:58:25,379 epoch 35 - iter 15/50 - loss 3.17661854 - samples/sec: 83.08 - lr: 0.125000 +2021-03-26 06:58:27,562 epoch 35 - iter 20/50 - loss 3.04253340 - samples/sec: 73.37 - lr: 0.125000 +2021-03-26 06:58:29,476 epoch 35 - iter 25/50 - loss 3.03141157 - samples/sec: 83.68 - lr: 0.125000 +2021-03-26 06:58:31,266 epoch 35 - iter 30/50 - loss 2.99109411 - samples/sec: 89.43 - lr: 0.125000 +2021-03-26 06:58:33,409 epoch 35 - iter 35/50 - loss 3.02970564 - samples/sec: 74.75 - lr: 0.125000 +2021-03-26 06:58:35,343 epoch 35 - iter 40/50 - loss 2.96278723 - samples/sec: 82.79 - lr: 0.125000 +2021-03-26 06:58:37,293 epoch 35 - iter 45/50 - loss 2.96387931 - samples/sec: 82.14 - lr: 0.125000 +2021-03-26 06:58:39,079 epoch 35 - iter 50/50 - loss 2.94057366 - samples/sec: 89.68 - lr: 0.125000 +2021-03-26 06:58:39,080 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:58:39,080 EPOCH 35 done: loss 2.9406 - lr 0.1250000 +2021-03-26 06:58:39,869 DEV : loss 6.2086615562438965 - score 0.9176 +2021-03-26 06:58:39,887 BAD EPOCHS (no improvement): 1 +2021-03-26 06:58:39,888 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:58:41,841 epoch 36 - iter 5/50 - loss 2.91071506 - samples/sec: 82.00 - lr: 0.125000 +2021-03-26 06:58:43,859 epoch 36 - iter 10/50 - loss 2.79274061 - samples/sec: 79.36 - lr: 0.125000 +2021-03-26 06:58:45,663 epoch 36 - iter 15/50 - loss 2.83839884 - samples/sec: 88.79 - lr: 0.125000 +2021-03-26 06:58:47,575 epoch 36 - iter 20/50 - loss 2.74050820 - samples/sec: 83.77 - lr: 0.125000 +2021-03-26 06:58:49,517 epoch 36 - iter 25/50 - loss 2.80581013 - samples/sec: 82.47 - lr: 0.125000 +2021-03-26 06:58:51,403 epoch 36 - iter 30/50 - loss 2.79912020 - samples/sec: 84.91 - lr: 0.125000 +2021-03-26 06:58:53,307 epoch 36 - iter 35/50 - loss 2.82536783 - samples/sec: 84.10 - lr: 0.125000 +2021-03-26 06:58:55,291 epoch 36 - iter 40/50 - loss 2.78613535 - samples/sec: 80.73 - lr: 0.125000 +2021-03-26 06:58:57,144 epoch 36 - iter 45/50 - loss 2.75108689 - samples/sec: 86.65 - lr: 0.125000 +2021-03-26 06:58:59,082 epoch 36 - iter 50/50 - loss 2.76818178 - samples/sec: 82.61 - lr: 0.125000 +2021-03-26 06:58:59,083 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:58:59,083 EPOCH 36 done: loss 2.7682 - lr 0.1250000 +2021-03-26 06:58:59,869 DEV : loss 6.127795696258545 - score 0.9174 +2021-03-26 06:58:59,886 BAD EPOCHS (no improvement): 2 +2021-03-26 06:58:59,886 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:59:01,782 epoch 37 - iter 5/50 - loss 2.71080043 - samples/sec: 84.48 - lr: 0.125000 +2021-03-26 06:59:03,754 epoch 37 - iter 10/50 - loss 2.78810500 - samples/sec: 81.22 - lr: 0.125000 +2021-03-26 06:59:05,672 epoch 37 - iter 15/50 - loss 2.70681345 - samples/sec: 83.50 - lr: 0.125000 +2021-03-26 06:59:07,555 epoch 37 - iter 20/50 - loss 2.85075440 - samples/sec: 85.05 - lr: 0.125000 +2021-03-26 06:59:09,645 epoch 37 - iter 25/50 - loss 2.78323064 - samples/sec: 76.63 - lr: 0.125000 +2021-03-26 06:59:11,596 epoch 37 - iter 30/50 - loss 2.73008518 - samples/sec: 82.09 - lr: 0.125000 +2021-03-26 06:59:13,941 epoch 37 - iter 35/50 - loss 2.70971954 - samples/sec: 68.27 - lr: 0.125000 +2021-03-26 06:59:15,808 epoch 37 - iter 40/50 - loss 2.74770900 - samples/sec: 85.79 - lr: 0.125000 +2021-03-26 06:59:17,907 epoch 37 - iter 45/50 - loss 2.71176132 - samples/sec: 76.28 - lr: 0.125000 +2021-03-26 06:59:19,929 epoch 37 - iter 50/50 - loss 2.73245367 - samples/sec: 79.21 - lr: 0.125000 +2021-03-26 06:59:19,930 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:59:19,930 EPOCH 37 done: loss 2.7325 - lr 0.1250000 +2021-03-26 06:59:20,707 DEV : loss 6.2698516845703125 - score 0.9136 +2021-03-26 06:59:20,732 BAD EPOCHS (no improvement): 3 +2021-03-26 06:59:20,733 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:59:22,756 epoch 38 - iter 5/50 - loss 2.90655355 - samples/sec: 79.18 - lr: 0.125000 +2021-03-26 06:59:24,734 epoch 38 - iter 10/50 - loss 2.93488872 - samples/sec: 81.01 - lr: 0.125000 +2021-03-26 06:59:26,744 epoch 38 - iter 15/50 - loss 2.86677569 - samples/sec: 79.68 - lr: 0.125000 +2021-03-26 06:59:28,976 epoch 38 - iter 20/50 - loss 2.91202615 - samples/sec: 71.74 - lr: 0.125000 +2021-03-26 06:59:30,891 epoch 38 - iter 25/50 - loss 2.86307866 - samples/sec: 83.65 - lr: 0.125000 +2021-03-26 06:59:33,072 epoch 38 - iter 30/50 - loss 2.95075897 - samples/sec: 73.44 - lr: 0.125000 +2021-03-26 06:59:34,952 epoch 38 - iter 35/50 - loss 2.90607083 - samples/sec: 85.16 - lr: 0.125000 +2021-03-26 06:59:36,765 epoch 38 - iter 40/50 - loss 2.84376124 - samples/sec: 88.34 - lr: 0.125000 +2021-03-26 06:59:38,899 epoch 38 - iter 45/50 - loss 2.79523445 - samples/sec: 75.02 - lr: 0.125000 +2021-03-26 06:59:40,948 epoch 38 - iter 50/50 - loss 2.86676959 - samples/sec: 78.18 - lr: 0.125000 +2021-03-26 06:59:40,949 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:59:40,949 EPOCH 38 done: loss 2.8668 - lr 0.1250000 +2021-03-26 06:59:41,715 DEV : loss 6.26828145980835 - score 0.9162 +2021-03-26 06:59:41,740 BAD EPOCHS (no improvement): 4 +2021-03-26 06:59:41,741 ---------------------------------------------------------------------------------------------------- +2021-03-26 06:59:43,849 epoch 39 - iter 5/50 - loss 2.46924899 - samples/sec: 75.96 - lr: 0.062500 +2021-03-26 06:59:46,083 epoch 39 - iter 10/50 - loss 2.86362134 - samples/sec: 71.68 - lr: 0.062500 +2021-03-26 06:59:48,163 epoch 39 - iter 15/50 - loss 2.78012599 - samples/sec: 77.00 - lr: 0.062500 +2021-03-26 06:59:49,991 epoch 39 - iter 20/50 - loss 2.71098604 - samples/sec: 87.64 - lr: 0.062500 +2021-03-26 06:59:52,191 epoch 39 - iter 25/50 - loss 2.65226630 - samples/sec: 72.76 - lr: 0.062500 +2021-03-26 06:59:54,168 epoch 39 - iter 30/50 - loss 2.74109042 - samples/sec: 81.03 - lr: 0.062500 +2021-03-26 06:59:56,261 epoch 39 - iter 35/50 - loss 2.78488238 - samples/sec: 76.50 - lr: 0.062500 +2021-03-26 06:59:58,198 epoch 39 - iter 40/50 - loss 2.74619337 - samples/sec: 82.68 - lr: 0.062500 +2021-03-26 06:59:59,987 epoch 39 - iter 45/50 - loss 2.72437237 - samples/sec: 89.50 - lr: 0.062500 +2021-03-26 07:00:01,796 epoch 39 - iter 50/50 - loss 2.68863890 - samples/sec: 88.52 - lr: 0.062500 +2021-03-26 07:00:01,797 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:00:01,797 EPOCH 39 done: loss 2.6886 - lr 0.0625000 +2021-03-26 07:00:02,589 DEV : loss 6.284448146820068 - score 0.917 +2021-03-26 07:00:02,614 BAD EPOCHS (no improvement): 1 +2021-03-26 07:00:02,614 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:00:04,580 epoch 40 - iter 5/50 - loss 2.98166246 - samples/sec: 81.48 - lr: 0.062500 +2021-03-26 07:00:06,572 epoch 40 - iter 10/50 - loss 2.72216311 - samples/sec: 80.40 - lr: 0.062500 +2021-03-26 07:00:08,496 epoch 40 - iter 15/50 - loss 2.75526333 - samples/sec: 83.23 - lr: 0.062500 +2021-03-26 07:00:10,561 epoch 40 - iter 20/50 - loss 2.78459678 - samples/sec: 77.58 - lr: 0.062500 +2021-03-26 07:00:13,042 epoch 40 - iter 25/50 - loss 2.79560489 - samples/sec: 64.55 - lr: 0.062500 +2021-03-26 07:00:14,886 epoch 40 - iter 30/50 - loss 2.71052564 - samples/sec: 86.84 - lr: 0.062500 +2021-03-26 07:00:16,780 epoch 40 - iter 35/50 - loss 2.66883559 - samples/sec: 84.57 - lr: 0.062500 +2021-03-26 07:00:18,875 epoch 40 - iter 40/50 - loss 2.69405829 - samples/sec: 76.44 - lr: 0.062500 +2021-03-26 07:00:20,935 epoch 40 - iter 45/50 - loss 2.69044249 - samples/sec: 77.72 - lr: 0.062500 +2021-03-26 07:00:22,728 epoch 40 - iter 50/50 - loss 2.72203983 - samples/sec: 89.33 - lr: 0.062500 +2021-03-26 07:00:22,729 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:00:22,730 EPOCH 40 done: loss 2.7220 - lr 0.0625000 +2021-03-26 07:00:23,521 DEV : loss 6.248233795166016 - score 0.9182 +2021-03-26 07:00:23,546 BAD EPOCHS (no improvement): 2 +2021-03-26 07:00:23,547 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:00:25,680 epoch 41 - iter 5/50 - loss 2.46897027 - samples/sec: 75.08 - lr: 0.062500 +2021-03-26 07:00:27,579 epoch 41 - iter 10/50 - loss 2.66373817 - samples/sec: 84.32 - lr: 0.062500 +2021-03-26 07:00:29,420 epoch 41 - iter 15/50 - loss 2.72111547 - samples/sec: 87.02 - lr: 0.062500 +2021-03-26 07:00:31,279 epoch 41 - iter 20/50 - loss 2.65721455 - samples/sec: 86.12 - lr: 0.062500 +2021-03-26 07:00:33,235 epoch 41 - iter 25/50 - loss 2.60619400 - samples/sec: 81.86 - lr: 0.062500 +2021-03-26 07:00:35,120 epoch 41 - iter 30/50 - loss 2.56057245 - samples/sec: 84.99 - lr: 0.062500 +2021-03-26 07:00:37,214 epoch 41 - iter 35/50 - loss 2.58174192 - samples/sec: 76.45 - lr: 0.062500 +2021-03-26 07:00:39,073 epoch 41 - iter 40/50 - loss 2.62893503 - samples/sec: 86.18 - lr: 0.062500 +2021-03-26 07:00:41,160 epoch 41 - iter 45/50 - loss 2.67516147 - samples/sec: 76.72 - lr: 0.062500 +2021-03-26 07:00:42,937 epoch 41 - iter 50/50 - loss 2.65079946 - samples/sec: 90.12 - lr: 0.062500 +2021-03-26 07:00:42,937 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:00:42,938 EPOCH 41 done: loss 2.6508 - lr 0.0625000 +2021-03-26 07:00:43,724 DEV : loss 6.362326622009277 - score 0.9166 +2021-03-26 07:00:43,748 BAD EPOCHS (no improvement): 3 +2021-03-26 07:00:43,749 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:00:45,638 epoch 42 - iter 5/50 - loss 2.71794162 - samples/sec: 84.83 - lr: 0.062500 +2021-03-26 07:00:47,540 epoch 42 - iter 10/50 - loss 2.52051284 - samples/sec: 84.19 - lr: 0.062500 +2021-03-26 07:00:49,791 epoch 42 - iter 15/50 - loss 2.58148971 - samples/sec: 71.13 - lr: 0.062500 +2021-03-26 07:00:51,879 epoch 42 - iter 20/50 - loss 2.66884552 - samples/sec: 76.72 - lr: 0.062500 +2021-03-26 07:00:53,852 epoch 42 - iter 25/50 - loss 2.65866809 - samples/sec: 81.15 - lr: 0.062500 +2021-03-26 07:00:55,757 epoch 42 - iter 30/50 - loss 2.65247952 - samples/sec: 84.11 - lr: 0.062500 +2021-03-26 07:00:57,712 epoch 42 - iter 35/50 - loss 2.66691832 - samples/sec: 81.89 - lr: 0.062500 +2021-03-26 07:00:59,609 epoch 42 - iter 40/50 - loss 2.69011677 - samples/sec: 84.47 - lr: 0.062500 +2021-03-26 07:01:01,585 epoch 42 - iter 45/50 - loss 2.70725158 - samples/sec: 81.06 - lr: 0.062500 +2021-03-26 07:01:03,305 epoch 42 - iter 50/50 - loss 2.74755251 - samples/sec: 93.14 - lr: 0.062500 +2021-03-26 07:01:03,306 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:01:03,306 EPOCH 42 done: loss 2.7476 - lr 0.0625000 +2021-03-26 07:01:04,111 DEV : loss 6.264382839202881 - score 0.917 +2021-03-26 07:01:04,135 BAD EPOCHS (no improvement): 4 +2021-03-26 07:01:04,136 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:01:06,096 epoch 43 - iter 5/50 - loss 2.62440243 - samples/sec: 81.72 - lr: 0.031250 +2021-03-26 07:01:08,236 epoch 43 - iter 10/50 - loss 2.86344099 - samples/sec: 74.87 - lr: 0.031250 +2021-03-26 07:01:10,153 epoch 43 - iter 15/50 - loss 2.81902952 - samples/sec: 83.53 - lr: 0.031250 +2021-03-26 07:01:12,006 epoch 43 - iter 20/50 - loss 2.73979611 - samples/sec: 86.45 - lr: 0.031250 +2021-03-26 07:01:13,986 epoch 43 - iter 25/50 - loss 2.61896025 - samples/sec: 80.89 - lr: 0.031250 +2021-03-26 07:01:15,972 epoch 43 - iter 30/50 - loss 2.68594955 - samples/sec: 80.61 - lr: 0.031250 +2021-03-26 07:01:18,047 epoch 43 - iter 35/50 - loss 2.70499782 - samples/sec: 77.19 - lr: 0.031250 +2021-03-26 07:01:20,037 epoch 43 - iter 40/50 - loss 2.66383172 - samples/sec: 80.45 - lr: 0.031250 +2021-03-26 07:01:21,974 epoch 43 - iter 45/50 - loss 2.66746508 - samples/sec: 82.67 - lr: 0.031250 +2021-03-26 07:01:23,966 epoch 43 - iter 50/50 - loss 2.67940281 - samples/sec: 80.39 - lr: 0.031250 +2021-03-26 07:01:23,967 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:01:23,968 EPOCH 43 done: loss 2.6794 - lr 0.0312500 +2021-03-26 07:01:24,820 DEV : loss 6.249972820281982 - score 0.9186 +2021-03-26 07:01:24,855 BAD EPOCHS (no improvement): 1 +2021-03-26 07:01:24,855 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:01:27,155 epoch 44 - iter 5/50 - loss 2.88372693 - samples/sec: 69.63 - lr: 0.031250 +2021-03-26 07:01:29,087 epoch 44 - iter 10/50 - loss 2.74055066 - samples/sec: 82.92 - lr: 0.031250 +2021-03-26 07:01:31,005 epoch 44 - iter 15/50 - loss 2.69428252 - samples/sec: 83.48 - lr: 0.031250 +2021-03-26 07:01:33,184 epoch 44 - iter 20/50 - loss 2.59563210 - samples/sec: 73.47 - lr: 0.031250 +2021-03-26 07:01:35,087 epoch 44 - iter 25/50 - loss 2.51464007 - samples/sec: 84.19 - lr: 0.031250 +2021-03-26 07:01:36,900 epoch 44 - iter 30/50 - loss 2.48493527 - samples/sec: 88.38 - lr: 0.031250 +2021-03-26 07:01:38,948 epoch 44 - iter 35/50 - loss 2.50545301 - samples/sec: 78.18 - lr: 0.031250 +2021-03-26 07:01:41,152 epoch 44 - iter 40/50 - loss 2.50784069 - samples/sec: 72.66 - lr: 0.031250 +2021-03-26 07:01:43,004 epoch 44 - iter 45/50 - loss 2.53496263 - samples/sec: 86.51 - lr: 0.031250 +2021-03-26 07:01:44,704 epoch 44 - iter 50/50 - loss 2.51157553 - samples/sec: 94.19 - lr: 0.031250 +2021-03-26 07:01:44,705 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:01:44,706 EPOCH 44 done: loss 2.5116 - lr 0.0312500 +2021-03-26 07:01:45,546 DEV : loss 6.207180023193359 - score 0.9186 +2021-03-26 07:01:45,578 BAD EPOCHS (no improvement): 2 +2021-03-26 07:01:45,578 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:01:47,291 epoch 45 - iter 5/50 - loss 2.37124238 - samples/sec: 93.49 - lr: 0.031250 +2021-03-26 07:01:49,119 epoch 45 - iter 10/50 - loss 2.76336849 - samples/sec: 87.65 - lr: 0.031250 +2021-03-26 07:01:51,108 epoch 45 - iter 15/50 - loss 2.62719330 - samples/sec: 80.52 - lr: 0.031250 +2021-03-26 07:01:53,185 epoch 45 - iter 20/50 - loss 2.60141521 - samples/sec: 77.08 - lr: 0.031250 +2021-03-26 07:01:55,152 epoch 45 - iter 25/50 - loss 2.53654681 - samples/sec: 81.41 - lr: 0.031250 +2021-03-26 07:01:57,178 epoch 45 - iter 30/50 - loss 2.52064539 - samples/sec: 79.05 - lr: 0.031250 +2021-03-26 07:01:59,147 epoch 45 - iter 35/50 - loss 2.49631712 - samples/sec: 81.32 - lr: 0.031250 +2021-03-26 07:02:01,260 epoch 45 - iter 40/50 - loss 2.51912695 - samples/sec: 75.83 - lr: 0.031250 +2021-03-26 07:02:03,368 epoch 45 - iter 45/50 - loss 2.55996369 - samples/sec: 75.93 - lr: 0.031250 +2021-03-26 07:02:05,176 epoch 45 - iter 50/50 - loss 2.54872012 - samples/sec: 88.59 - lr: 0.031250 +2021-03-26 07:02:05,176 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:02:05,177 EPOCH 45 done: loss 2.5487 - lr 0.0312500 +2021-03-26 07:02:05,990 DEV : loss 6.239845275878906 - score 0.9178 +2021-03-26 07:02:06,018 BAD EPOCHS (no improvement): 3 +2021-03-26 07:02:06,019 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:02:07,934 epoch 46 - iter 5/50 - loss 2.08215249 - samples/sec: 83.63 - lr: 0.031250 +2021-03-26 07:02:09,916 epoch 46 - iter 10/50 - loss 2.39230992 - samples/sec: 80.81 - lr: 0.031250 +2021-03-26 07:02:11,888 epoch 46 - iter 15/50 - loss 2.51131487 - samples/sec: 81.21 - lr: 0.031250 +2021-03-26 07:02:13,878 epoch 46 - iter 20/50 - loss 2.56332803 - samples/sec: 80.47 - lr: 0.031250 +2021-03-26 07:02:15,898 epoch 46 - iter 25/50 - loss 2.58750481 - samples/sec: 79.24 - lr: 0.031250 +2021-03-26 07:02:18,080 epoch 46 - iter 30/50 - loss 2.56808805 - samples/sec: 73.39 - lr: 0.031250 +2021-03-26 07:02:20,000 epoch 46 - iter 35/50 - loss 2.59616448 - samples/sec: 83.43 - lr: 0.031250 +2021-03-26 07:02:21,798 epoch 46 - iter 40/50 - loss 2.54724448 - samples/sec: 89.11 - lr: 0.031250 +2021-03-26 07:02:23,840 epoch 46 - iter 45/50 - loss 2.60226175 - samples/sec: 78.42 - lr: 0.031250 +2021-03-26 07:02:25,524 epoch 46 - iter 50/50 - loss 2.62334806 - samples/sec: 95.10 - lr: 0.031250 +2021-03-26 07:02:25,525 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:02:25,526 EPOCH 46 done: loss 2.6233 - lr 0.0312500 +2021-03-26 07:02:26,305 DEV : loss 6.2371320724487305 - score 0.9186 +2021-03-26 07:02:26,330 BAD EPOCHS (no improvement): 4 +2021-03-26 07:02:26,331 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:02:28,374 epoch 47 - iter 5/50 - loss 2.41000776 - samples/sec: 78.39 - lr: 0.015625 +2021-03-26 07:02:30,413 epoch 47 - iter 10/50 - loss 2.40392948 - samples/sec: 78.56 - lr: 0.015625 +2021-03-26 07:02:32,296 epoch 47 - iter 15/50 - loss 2.49342973 - samples/sec: 85.04 - lr: 0.015625 +2021-03-26 07:02:34,113 epoch 47 - iter 20/50 - loss 2.41740447 - samples/sec: 88.16 - lr: 0.015625 +2021-03-26 07:02:35,975 epoch 47 - iter 25/50 - loss 2.43529265 - samples/sec: 86.01 - lr: 0.015625 +2021-03-26 07:02:38,034 epoch 47 - iter 30/50 - loss 2.50073948 - samples/sec: 77.74 - lr: 0.015625 +2021-03-26 07:02:40,083 epoch 47 - iter 35/50 - loss 2.48987853 - samples/sec: 78.17 - lr: 0.015625 +2021-03-26 07:02:41,987 epoch 47 - iter 40/50 - loss 2.52052880 - samples/sec: 84.12 - lr: 0.015625 +2021-03-26 07:02:43,994 epoch 47 - iter 45/50 - loss 2.52663625 - samples/sec: 79.79 - lr: 0.015625 +2021-03-26 07:02:45,779 epoch 47 - iter 50/50 - loss 2.56214527 - samples/sec: 89.70 - lr: 0.015625 +2021-03-26 07:02:45,780 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:02:45,780 EPOCH 47 done: loss 2.5621 - lr 0.0156250 +2021-03-26 07:02:46,552 DEV : loss 6.216527462005615 - score 0.9182 +2021-03-26 07:02:46,577 BAD EPOCHS (no improvement): 1 +2021-03-26 07:02:46,577 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:02:48,736 epoch 48 - iter 5/50 - loss 2.62342062 - samples/sec: 74.16 - lr: 0.015625 +2021-03-26 07:02:50,661 epoch 48 - iter 10/50 - loss 2.50259314 - samples/sec: 83.22 - lr: 0.015625 +2021-03-26 07:02:52,640 epoch 48 - iter 15/50 - loss 2.41186643 - samples/sec: 80.92 - lr: 0.015625 +2021-03-26 07:02:54,835 epoch 48 - iter 20/50 - loss 2.50129139 - samples/sec: 72.96 - lr: 0.015625 +2021-03-26 07:02:56,807 epoch 48 - iter 25/50 - loss 2.53100896 - samples/sec: 81.22 - lr: 0.015625 +2021-03-26 07:02:58,817 epoch 48 - iter 30/50 - loss 2.56003957 - samples/sec: 79.65 - lr: 0.015625 +2021-03-26 07:03:00,548 epoch 48 - iter 35/50 - loss 2.57986932 - samples/sec: 92.56 - lr: 0.015625 +2021-03-26 07:03:02,387 epoch 48 - iter 40/50 - loss 2.54486949 - samples/sec: 87.06 - lr: 0.015625 +2021-03-26 07:03:04,379 epoch 48 - iter 45/50 - loss 2.50048207 - samples/sec: 80.40 - lr: 0.015625 +2021-03-26 07:03:06,137 epoch 48 - iter 50/50 - loss 2.47580104 - samples/sec: 91.09 - lr: 0.015625 +2021-03-26 07:03:06,138 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:03:06,138 EPOCH 48 done: loss 2.4758 - lr 0.0156250 +2021-03-26 07:03:06,958 DEV : loss 6.240680694580078 - score 0.9178 +2021-03-26 07:03:06,980 BAD EPOCHS (no improvement): 2 +2021-03-26 07:03:06,980 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:03:09,284 epoch 49 - iter 5/50 - loss 2.28204064 - samples/sec: 69.50 - lr: 0.015625 +2021-03-26 07:03:11,353 epoch 49 - iter 10/50 - loss 2.35861111 - samples/sec: 77.45 - lr: 0.015625 +2021-03-26 07:03:13,375 epoch 49 - iter 15/50 - loss 2.40004193 - samples/sec: 79.18 - lr: 0.015625 +2021-03-26 07:03:15,352 epoch 49 - iter 20/50 - loss 2.59549707 - samples/sec: 81.01 - lr: 0.015625 +2021-03-26 07:03:17,290 epoch 49 - iter 25/50 - loss 2.60552569 - samples/sec: 82.64 - lr: 0.015625 +2021-03-26 07:03:19,361 epoch 49 - iter 30/50 - loss 2.58463446 - samples/sec: 77.34 - lr: 0.015625 +2021-03-26 07:03:21,146 epoch 49 - iter 35/50 - loss 2.56579349 - samples/sec: 89.72 - lr: 0.015625 +2021-03-26 07:03:23,055 epoch 49 - iter 40/50 - loss 2.57229652 - samples/sec: 83.90 - lr: 0.015625 +2021-03-26 07:03:25,090 epoch 49 - iter 45/50 - loss 2.51541369 - samples/sec: 78.69 - lr: 0.015625 +2021-03-26 07:03:26,932 epoch 49 - iter 50/50 - loss 2.54103005 - samples/sec: 86.93 - lr: 0.015625 +2021-03-26 07:03:26,933 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:03:26,933 EPOCH 49 done: loss 2.5410 - lr 0.0156250 +2021-03-26 07:03:27,705 DEV : loss 6.239748954772949 - score 0.917 +2021-03-26 07:03:27,730 BAD EPOCHS (no improvement): 3 +2021-03-26 07:03:27,731 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:03:29,696 epoch 50 - iter 5/50 - loss 2.28539340 - samples/sec: 81.51 - lr: 0.015625 +2021-03-26 07:03:31,503 epoch 50 - iter 10/50 - loss 2.48737829 - samples/sec: 88.64 - lr: 0.015625 +2021-03-26 07:03:33,520 epoch 50 - iter 15/50 - loss 2.59285909 - samples/sec: 79.41 - lr: 0.015625 +2021-03-26 07:03:35,439 epoch 50 - iter 20/50 - loss 2.56907600 - samples/sec: 83.43 - lr: 0.015625 +2021-03-26 07:03:37,519 epoch 50 - iter 25/50 - loss 2.49343590 - samples/sec: 77.02 - lr: 0.015625 +2021-03-26 07:03:39,594 epoch 50 - iter 30/50 - loss 2.54235819 - samples/sec: 77.17 - lr: 0.015625 +2021-03-26 07:03:41,490 epoch 50 - iter 35/50 - loss 2.49685275 - samples/sec: 84.47 - lr: 0.015625 +2021-03-26 07:03:43,392 epoch 50 - iter 40/50 - loss 2.48172221 - samples/sec: 84.19 - lr: 0.015625 +2021-03-26 07:03:45,403 epoch 50 - iter 45/50 - loss 2.52431675 - samples/sec: 79.63 - lr: 0.015625 +2021-03-26 07:03:47,179 epoch 50 - iter 50/50 - loss 2.48997502 - samples/sec: 90.19 - lr: 0.015625 +2021-03-26 07:03:47,179 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:03:47,179 EPOCH 50 done: loss 2.4900 - lr 0.0156250 +2021-03-26 07:03:47,947 DEV : loss 6.231075286865234 - score 0.917 +2021-03-26 07:03:47,971 BAD EPOCHS (no improvement): 4 +2021-03-26 07:03:47,972 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:03:49,943 epoch 51 - iter 5/50 - loss 2.58739772 - samples/sec: 81.26 - lr: 0.007812 +2021-03-26 07:03:51,900 epoch 51 - iter 10/50 - loss 2.75475857 - samples/sec: 81.82 - lr: 0.007812 +2021-03-26 07:03:53,847 epoch 51 - iter 15/50 - loss 2.64987794 - samples/sec: 82.25 - lr: 0.007812 +2021-03-26 07:03:55,996 epoch 51 - iter 20/50 - loss 2.58137577 - samples/sec: 74.53 - lr: 0.007812 +2021-03-26 07:03:58,032 epoch 51 - iter 25/50 - loss 2.63140382 - samples/sec: 78.68 - lr: 0.007812 +2021-03-26 07:03:59,964 epoch 51 - iter 30/50 - loss 2.57393547 - samples/sec: 82.87 - lr: 0.007812 +2021-03-26 07:04:01,820 epoch 51 - iter 35/50 - loss 2.55677542 - samples/sec: 86.33 - lr: 0.007812 +2021-03-26 07:04:03,802 epoch 51 - iter 40/50 - loss 2.55924032 - samples/sec: 80.81 - lr: 0.007812 +2021-03-26 07:04:05,813 epoch 51 - iter 45/50 - loss 2.55842340 - samples/sec: 79.66 - lr: 0.007812 +2021-03-26 07:04:07,493 epoch 51 - iter 50/50 - loss 2.51336263 - samples/sec: 95.37 - lr: 0.007812 +2021-03-26 07:04:07,494 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:04:07,494 EPOCH 51 done: loss 2.5134 - lr 0.0078125 +2021-03-26 07:04:08,281 DEV : loss 6.22976016998291 - score 0.9174 +2021-03-26 07:04:08,306 BAD EPOCHS (no improvement): 1 +2021-03-26 07:04:08,307 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:04:10,185 epoch 52 - iter 5/50 - loss 2.12111568 - samples/sec: 85.30 - lr: 0.007812 +2021-03-26 07:04:12,259 epoch 52 - iter 10/50 - loss 2.28783898 - samples/sec: 77.20 - lr: 0.007812 +2021-03-26 07:04:14,173 epoch 52 - iter 15/50 - loss 2.27034943 - samples/sec: 83.65 - lr: 0.007812 +2021-03-26 07:04:15,945 epoch 52 - iter 20/50 - loss 2.35835313 - samples/sec: 90.39 - lr: 0.007812 +2021-03-26 07:04:17,875 epoch 52 - iter 25/50 - loss 2.32353603 - samples/sec: 82.97 - lr: 0.007812 +2021-03-26 07:04:19,849 epoch 52 - iter 30/50 - loss 2.36913104 - samples/sec: 81.15 - lr: 0.007812 +2021-03-26 07:04:21,828 epoch 52 - iter 35/50 - loss 2.35468242 - samples/sec: 80.91 - lr: 0.007812 +2021-03-26 07:04:23,653 epoch 52 - iter 40/50 - loss 2.32059786 - samples/sec: 87.77 - lr: 0.007812 +2021-03-26 07:04:25,634 epoch 52 - iter 45/50 - loss 2.38650590 - samples/sec: 80.83 - lr: 0.007812 +2021-03-26 07:04:27,572 epoch 52 - iter 50/50 - loss 2.41335446 - samples/sec: 82.62 - lr: 0.007812 +2021-03-26 07:04:27,573 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:04:27,573 EPOCH 52 done: loss 2.4134 - lr 0.0078125 +2021-03-26 07:04:28,309 DEV : loss 6.234460353851318 - score 0.9168 +2021-03-26 07:04:28,333 BAD EPOCHS (no improvement): 2 +2021-03-26 07:04:28,334 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:04:30,365 epoch 53 - iter 5/50 - loss 2.29304843 - samples/sec: 78.86 - lr: 0.007812 +2021-03-26 07:04:32,302 epoch 53 - iter 10/50 - loss 2.36527128 - samples/sec: 82.68 - lr: 0.007812 +2021-03-26 07:04:34,216 epoch 53 - iter 15/50 - loss 2.42961562 - samples/sec: 84.15 - lr: 0.007812 +2021-03-26 07:04:36,211 epoch 53 - iter 20/50 - loss 2.49262825 - samples/sec: 82.64 - lr: 0.007812 +2021-03-26 07:04:38,041 epoch 53 - iter 25/50 - loss 2.42180495 - samples/sec: 87.49 - lr: 0.007812 +2021-03-26 07:04:39,948 epoch 53 - iter 30/50 - loss 2.48365995 - samples/sec: 83.98 - lr: 0.007812 +2021-03-26 07:04:41,756 epoch 53 - iter 35/50 - loss 2.49418128 - samples/sec: 88.59 - lr: 0.007812 +2021-03-26 07:04:43,746 epoch 53 - iter 40/50 - loss 2.52012618 - samples/sec: 80.48 - lr: 0.007812 +2021-03-26 07:04:45,773 epoch 53 - iter 45/50 - loss 2.51719693 - samples/sec: 79.01 - lr: 0.007812 +2021-03-26 07:04:47,665 epoch 53 - iter 50/50 - loss 2.44318517 - samples/sec: 84.65 - lr: 0.007812 +2021-03-26 07:04:47,666 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:04:47,667 EPOCH 53 done: loss 2.4432 - lr 0.0078125 +2021-03-26 07:04:48,427 DEV : loss 6.224849700927734 - score 0.9174 +2021-03-26 07:04:48,452 BAD EPOCHS (no improvement): 3 +2021-03-26 07:04:48,453 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:04:50,426 epoch 54 - iter 5/50 - loss 2.27110686 - samples/sec: 81.16 - lr: 0.007812 +2021-03-26 07:04:52,346 epoch 54 - iter 10/50 - loss 2.36324615 - samples/sec: 83.42 - lr: 0.007812 +2021-03-26 07:04:54,189 epoch 54 - iter 15/50 - loss 2.39531080 - samples/sec: 86.89 - lr: 0.007812 +2021-03-26 07:04:56,261 epoch 54 - iter 20/50 - loss 2.34100553 - samples/sec: 77.30 - lr: 0.007812 +2021-03-26 07:04:58,123 epoch 54 - iter 25/50 - loss 2.32279901 - samples/sec: 86.03 - lr: 0.007812 +2021-03-26 07:04:59,964 epoch 54 - iter 30/50 - loss 2.30034845 - samples/sec: 86.99 - lr: 0.007812 +2021-03-26 07:05:01,816 epoch 54 - iter 35/50 - loss 2.32203776 - samples/sec: 86.51 - lr: 0.007812 +2021-03-26 07:05:03,781 epoch 54 - iter 40/50 - loss 2.35916250 - samples/sec: 81.48 - lr: 0.007812 +2021-03-26 07:05:05,783 epoch 54 - iter 45/50 - loss 2.31716502 - samples/sec: 80.00 - lr: 0.007812 +2021-03-26 07:05:07,576 epoch 54 - iter 50/50 - loss 2.39397667 - samples/sec: 89.33 - lr: 0.007812 +2021-03-26 07:05:07,576 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:05:07,577 EPOCH 54 done: loss 2.3940 - lr 0.0078125 +2021-03-26 07:05:08,342 DEV : loss 6.229462623596191 - score 0.9174 +2021-03-26 07:05:08,368 BAD EPOCHS (no improvement): 4 +2021-03-26 07:05:08,368 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:05:10,228 epoch 55 - iter 5/50 - loss 2.43591893 - samples/sec: 86.12 - lr: 0.003906 +2021-03-26 07:05:12,297 epoch 55 - iter 10/50 - loss 2.29127170 - samples/sec: 77.40 - lr: 0.003906 +2021-03-26 07:05:14,303 epoch 55 - iter 15/50 - loss 2.29653165 - samples/sec: 79.82 - lr: 0.003906 +2021-03-26 07:05:16,052 epoch 55 - iter 20/50 - loss 2.30765195 - samples/sec: 91.57 - lr: 0.003906 +2021-03-26 07:05:17,788 epoch 55 - iter 25/50 - loss 2.35338634 - samples/sec: 92.31 - lr: 0.003906 +2021-03-26 07:05:19,830 epoch 55 - iter 30/50 - loss 2.39339325 - samples/sec: 78.42 - lr: 0.003906 +2021-03-26 07:05:21,748 epoch 55 - iter 35/50 - loss 2.35602188 - samples/sec: 83.50 - lr: 0.003906 +2021-03-26 07:05:23,679 epoch 55 - iter 40/50 - loss 2.36002087 - samples/sec: 82.94 - lr: 0.003906 +2021-03-26 07:05:25,625 epoch 55 - iter 45/50 - loss 2.38230093 - samples/sec: 82.29 - lr: 0.003906 +2021-03-26 07:05:27,669 epoch 55 - iter 50/50 - loss 2.42242070 - samples/sec: 78.35 - lr: 0.003906 +2021-03-26 07:05:27,670 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:05:27,670 EPOCH 55 done: loss 2.4224 - lr 0.0039062 +2021-03-26 07:05:28,484 DEV : loss 6.228795528411865 - score 0.9174 +2021-03-26 07:05:28,508 BAD EPOCHS (no improvement): 1 +2021-03-26 07:05:28,509 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:05:30,536 epoch 56 - iter 5/50 - loss 2.19912152 - samples/sec: 79.06 - lr: 0.003906 +2021-03-26 07:05:32,433 epoch 56 - iter 10/50 - loss 2.17522593 - samples/sec: 84.42 - lr: 0.003906 +2021-03-26 07:05:34,525 epoch 56 - iter 15/50 - loss 2.18695093 - samples/sec: 76.55 - lr: 0.003906 +2021-03-26 07:05:36,694 epoch 56 - iter 20/50 - loss 2.24990177 - samples/sec: 73.82 - lr: 0.003906 +2021-03-26 07:05:38,712 epoch 56 - iter 25/50 - loss 2.33317802 - samples/sec: 79.36 - lr: 0.003906 +2021-03-26 07:05:40,630 epoch 56 - iter 30/50 - loss 2.36618576 - samples/sec: 83.54 - lr: 0.003906 +2021-03-26 07:05:42,494 epoch 56 - iter 35/50 - loss 2.34973713 - samples/sec: 85.89 - lr: 0.003906 +2021-03-26 07:05:44,488 epoch 56 - iter 40/50 - loss 2.37181876 - samples/sec: 80.34 - lr: 0.003906 +2021-03-26 07:05:46,501 epoch 56 - iter 45/50 - loss 2.38742908 - samples/sec: 79.55 - lr: 0.003906 +2021-03-26 07:05:48,183 epoch 56 - iter 50/50 - loss 2.36818949 - samples/sec: 95.22 - lr: 0.003906 +2021-03-26 07:05:48,184 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:05:48,185 EPOCH 56 done: loss 2.3682 - lr 0.0039062 +2021-03-26 07:05:49,075 DEV : loss 6.230513572692871 - score 0.9174 +2021-03-26 07:05:49,102 BAD EPOCHS (no improvement): 2 +2021-03-26 07:05:49,103 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:05:51,070 epoch 57 - iter 5/50 - loss 2.74660211 - samples/sec: 81.45 - lr: 0.003906 +2021-03-26 07:05:53,166 epoch 57 - iter 10/50 - loss 2.75735655 - samples/sec: 76.42 - lr: 0.003906 +2021-03-26 07:05:55,067 epoch 57 - iter 15/50 - loss 2.63150340 - samples/sec: 84.25 - lr: 0.003906 +2021-03-26 07:05:56,888 epoch 57 - iter 20/50 - loss 2.65938060 - samples/sec: 87.96 - lr: 0.003906 +2021-03-26 07:05:58,773 epoch 57 - iter 25/50 - loss 2.65044247 - samples/sec: 84.96 - lr: 0.003906 +2021-03-26 07:06:00,679 epoch 57 - iter 30/50 - loss 2.63459703 - samples/sec: 84.03 - lr: 0.003906 +2021-03-26 07:06:02,698 epoch 57 - iter 35/50 - loss 2.60287085 - samples/sec: 79.28 - lr: 0.003906 +2021-03-26 07:06:04,687 epoch 57 - iter 40/50 - loss 2.59769873 - samples/sec: 80.54 - lr: 0.003906 +2021-03-26 07:06:06,605 epoch 57 - iter 45/50 - loss 2.61791177 - samples/sec: 83.51 - lr: 0.003906 +2021-03-26 07:06:08,472 epoch 57 - iter 50/50 - loss 2.57899422 - samples/sec: 85.76 - lr: 0.003906 +2021-03-26 07:06:08,473 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:06:08,473 EPOCH 57 done: loss 2.5790 - lr 0.0039062 +2021-03-26 07:06:09,227 DEV : loss 6.234955787658691 - score 0.9174 +2021-03-26 07:06:09,252 BAD EPOCHS (no improvement): 3 +2021-03-26 07:06:09,252 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:06:11,192 epoch 58 - iter 5/50 - loss 2.11452603 - samples/sec: 82.58 - lr: 0.003906 +2021-03-26 07:06:13,175 epoch 58 - iter 10/50 - loss 2.48117228 - samples/sec: 80.77 - lr: 0.003906 +2021-03-26 07:06:14,938 epoch 58 - iter 15/50 - loss 2.51002020 - samples/sec: 90.85 - lr: 0.003906 +2021-03-26 07:06:16,856 epoch 58 - iter 20/50 - loss 2.51059042 - samples/sec: 83.48 - lr: 0.003906 +2021-03-26 07:06:18,958 epoch 58 - iter 25/50 - loss 2.51444347 - samples/sec: 76.18 - lr: 0.003906 +2021-03-26 07:06:20,793 epoch 58 - iter 30/50 - loss 2.46463509 - samples/sec: 87.30 - lr: 0.003906 +2021-03-26 07:06:22,700 epoch 58 - iter 35/50 - loss 2.54111838 - samples/sec: 84.01 - lr: 0.003906 +2021-03-26 07:06:24,620 epoch 58 - iter 40/50 - loss 2.48917502 - samples/sec: 83.44 - lr: 0.003906 +2021-03-26 07:06:26,621 epoch 58 - iter 45/50 - loss 2.50563401 - samples/sec: 80.04 - lr: 0.003906 +2021-03-26 07:06:28,525 epoch 58 - iter 50/50 - loss 2.51603883 - samples/sec: 84.14 - lr: 0.003906 +2021-03-26 07:06:28,525 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:06:28,526 EPOCH 58 done: loss 2.5160 - lr 0.0039062 +2021-03-26 07:06:29,281 DEV : loss 6.235045909881592 - score 0.9174 +2021-03-26 07:06:29,304 BAD EPOCHS (no improvement): 4 +2021-03-26 07:06:29,305 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:06:31,310 epoch 59 - iter 5/50 - loss 2.58779755 - samples/sec: 79.89 - lr: 0.001953 +2021-03-26 07:06:33,249 epoch 59 - iter 10/50 - loss 2.31887306 - samples/sec: 82.58 - lr: 0.001953 +2021-03-26 07:06:35,243 epoch 59 - iter 15/50 - loss 2.39448693 - samples/sec: 80.34 - lr: 0.001953 +2021-03-26 07:06:37,895 epoch 59 - iter 20/50 - loss 2.40186156 - samples/sec: 60.37 - lr: 0.001953 +2021-03-26 07:06:40,190 epoch 59 - iter 25/50 - loss 2.38810690 - samples/sec: 69.76 - lr: 0.001953 +2021-03-26 07:06:41,923 epoch 59 - iter 30/50 - loss 2.42196012 - samples/sec: 92.44 - lr: 0.001953 +2021-03-26 07:06:43,954 epoch 59 - iter 35/50 - loss 2.43881528 - samples/sec: 78.85 - lr: 0.001953 +2021-03-26 07:06:46,001 epoch 59 - iter 40/50 - loss 2.42435353 - samples/sec: 78.23 - lr: 0.001953 +2021-03-26 07:06:48,009 epoch 59 - iter 45/50 - loss 2.41908374 - samples/sec: 79.76 - lr: 0.001953 +2021-03-26 07:06:49,660 epoch 59 - iter 50/50 - loss 2.39769382 - samples/sec: 96.98 - lr: 0.001953 +2021-03-26 07:06:49,661 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:06:49,661 EPOCH 59 done: loss 2.3977 - lr 0.0019531 +2021-03-26 07:06:50,422 DEV : loss 6.2370758056640625 - score 0.917 +2021-03-26 07:06:50,446 BAD EPOCHS (no improvement): 1 +2021-03-26 07:06:50,447 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:06:52,362 epoch 60 - iter 5/50 - loss 2.21496704 - samples/sec: 83.65 - lr: 0.001953 +2021-03-26 07:06:54,187 epoch 60 - iter 10/50 - loss 2.42700636 - samples/sec: 87.78 - lr: 0.001953 +2021-03-26 07:06:56,086 epoch 60 - iter 15/50 - loss 2.53753303 - samples/sec: 84.35 - lr: 0.001953 +2021-03-26 07:06:58,182 epoch 60 - iter 20/50 - loss 2.60033162 - samples/sec: 76.40 - lr: 0.001953 +2021-03-26 07:07:00,224 epoch 60 - iter 25/50 - loss 2.51529428 - samples/sec: 78.42 - lr: 0.001953 +2021-03-26 07:07:02,296 epoch 60 - iter 30/50 - loss 2.49559573 - samples/sec: 77.28 - lr: 0.001953 +2021-03-26 07:07:04,332 epoch 60 - iter 35/50 - loss 2.51090207 - samples/sec: 78.66 - lr: 0.001953 +2021-03-26 07:07:06,404 epoch 60 - iter 40/50 - loss 2.51477104 - samples/sec: 77.30 - lr: 0.001953 +2021-03-26 07:07:08,233 epoch 60 - iter 45/50 - loss 2.50222202 - samples/sec: 87.58 - lr: 0.001953 +2021-03-26 07:07:09,957 epoch 60 - iter 50/50 - loss 2.44377190 - samples/sec: 92.90 - lr: 0.001953 +2021-03-26 07:07:09,957 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:07:09,958 EPOCH 60 done: loss 2.4438 - lr 0.0019531 +2021-03-26 07:07:10,699 DEV : loss 6.238881587982178 - score 0.9172 +2021-03-26 07:07:10,723 BAD EPOCHS (no improvement): 2 +2021-03-26 07:07:10,724 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:07:12,525 epoch 61 - iter 5/50 - loss 2.12126102 - samples/sec: 88.95 - lr: 0.001953 +2021-03-26 07:07:14,428 epoch 61 - iter 10/50 - loss 2.54890535 - samples/sec: 84.16 - lr: 0.001953 +2021-03-26 07:07:16,408 epoch 61 - iter 15/50 - loss 2.51401180 - samples/sec: 80.91 - lr: 0.001953 +2021-03-26 07:07:18,503 epoch 61 - iter 20/50 - loss 2.49502721 - samples/sec: 76.41 - lr: 0.001953 +2021-03-26 07:07:20,446 epoch 61 - iter 25/50 - loss 2.57242790 - samples/sec: 82.49 - lr: 0.001953 +2021-03-26 07:07:22,349 epoch 61 - iter 30/50 - loss 2.60423574 - samples/sec: 84.13 - lr: 0.001953 +2021-03-26 07:07:24,326 epoch 61 - iter 35/50 - loss 2.54501983 - samples/sec: 81.05 - lr: 0.001953 +2021-03-26 07:07:26,422 epoch 61 - iter 40/50 - loss 2.52295736 - samples/sec: 76.37 - lr: 0.001953 +2021-03-26 07:07:28,231 epoch 61 - iter 45/50 - loss 2.53937517 - samples/sec: 88.56 - lr: 0.001953 +2021-03-26 07:07:30,169 epoch 61 - iter 50/50 - loss 2.51024843 - samples/sec: 82.65 - lr: 0.001953 +2021-03-26 07:07:30,170 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:07:30,170 EPOCH 61 done: loss 2.5102 - lr 0.0019531 +2021-03-26 07:07:30,921 DEV : loss 6.2310614585876465 - score 0.917 +2021-03-26 07:07:30,945 BAD EPOCHS (no improvement): 3 +2021-03-26 07:07:30,946 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:07:33,212 epoch 62 - iter 5/50 - loss 2.69974794 - samples/sec: 70.66 - lr: 0.001953 +2021-03-26 07:07:35,175 epoch 62 - iter 10/50 - loss 2.58760433 - samples/sec: 81.58 - lr: 0.001953 +2021-03-26 07:07:37,300 epoch 62 - iter 15/50 - loss 2.45208565 - samples/sec: 75.38 - lr: 0.001953 +2021-03-26 07:07:39,445 epoch 62 - iter 20/50 - loss 2.47846313 - samples/sec: 74.67 - lr: 0.001953 +2021-03-26 07:07:41,524 epoch 62 - iter 25/50 - loss 2.44997159 - samples/sec: 76.99 - lr: 0.001953 +2021-03-26 07:07:43,774 epoch 62 - iter 30/50 - loss 2.38889544 - samples/sec: 71.20 - lr: 0.001953 +2021-03-26 07:07:45,679 epoch 62 - iter 35/50 - loss 2.36844786 - samples/sec: 84.05 - lr: 0.001953 +2021-03-26 07:07:47,775 epoch 62 - iter 40/50 - loss 2.38948717 - samples/sec: 76.41 - lr: 0.001953 +2021-03-26 07:07:49,904 epoch 62 - iter 45/50 - loss 2.37542345 - samples/sec: 75.19 - lr: 0.001953 +2021-03-26 07:07:51,797 epoch 62 - iter 50/50 - loss 2.37077458 - samples/sec: 84.61 - lr: 0.001953 +2021-03-26 07:07:51,798 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:07:51,799 EPOCH 62 done: loss 2.3708 - lr 0.0019531 +2021-03-26 07:07:52,608 DEV : loss 6.230165004730225 - score 0.9182 +2021-03-26 07:07:52,634 BAD EPOCHS (no improvement): 4 +2021-03-26 07:07:52,635 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:07:54,592 epoch 63 - iter 5/50 - loss 2.13501656 - samples/sec: 81.85 - lr: 0.000977 +2021-03-26 07:07:56,438 epoch 63 - iter 10/50 - loss 2.24644828 - samples/sec: 86.76 - lr: 0.000977 +2021-03-26 07:07:58,325 epoch 63 - iter 15/50 - loss 2.22856548 - samples/sec: 84.85 - lr: 0.000977 +2021-03-26 07:08:00,141 epoch 63 - iter 20/50 - loss 2.17708767 - samples/sec: 88.23 - lr: 0.000977 +2021-03-26 07:08:02,279 epoch 63 - iter 25/50 - loss 2.24079601 - samples/sec: 74.87 - lr: 0.000977 +2021-03-26 07:08:04,266 epoch 63 - iter 30/50 - loss 2.28632747 - samples/sec: 80.59 - lr: 0.000977 +2021-03-26 07:08:06,002 epoch 63 - iter 35/50 - loss 2.30275180 - samples/sec: 92.26 - lr: 0.000977 +2021-03-26 07:08:07,813 epoch 63 - iter 40/50 - loss 2.37092484 - samples/sec: 88.44 - lr: 0.000977 +2021-03-26 07:08:09,693 epoch 63 - iter 45/50 - loss 2.41254362 - samples/sec: 85.23 - lr: 0.000977 +2021-03-26 07:08:11,485 epoch 63 - iter 50/50 - loss 2.44104299 - samples/sec: 89.36 - lr: 0.000977 +2021-03-26 07:08:11,485 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:08:11,486 EPOCH 63 done: loss 2.4410 - lr 0.0009766 +2021-03-26 07:08:12,242 DEV : loss 6.228281021118164 - score 0.9186 +2021-03-26 07:08:12,267 BAD EPOCHS (no improvement): 1 +2021-03-26 07:08:12,268 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:08:14,289 epoch 64 - iter 5/50 - loss 2.34046373 - samples/sec: 79.26 - lr: 0.000977 +2021-03-26 07:08:16,307 epoch 64 - iter 10/50 - loss 2.34239235 - samples/sec: 79.38 - lr: 0.000977 +2021-03-26 07:08:18,078 epoch 64 - iter 15/50 - loss 2.39339416 - samples/sec: 90.42 - lr: 0.000977 +2021-03-26 07:08:20,052 epoch 64 - iter 20/50 - loss 2.39595767 - samples/sec: 81.10 - lr: 0.000977 +2021-03-26 07:08:22,033 epoch 64 - iter 25/50 - loss 2.36385040 - samples/sec: 80.87 - lr: 0.000977 +2021-03-26 07:08:23,979 epoch 64 - iter 30/50 - loss 2.38750821 - samples/sec: 82.31 - lr: 0.000977 +2021-03-26 07:08:25,919 epoch 64 - iter 35/50 - loss 2.35466352 - samples/sec: 82.54 - lr: 0.000977 +2021-03-26 07:08:27,796 epoch 64 - iter 40/50 - loss 2.36880653 - samples/sec: 85.35 - lr: 0.000977 +2021-03-26 07:08:29,596 epoch 64 - iter 45/50 - loss 2.35453144 - samples/sec: 88.94 - lr: 0.000977 +2021-03-26 07:08:31,425 epoch 64 - iter 50/50 - loss 2.41410687 - samples/sec: 87.55 - lr: 0.000977 +2021-03-26 07:08:31,426 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:08:31,427 EPOCH 64 done: loss 2.4141 - lr 0.0009766 +2021-03-26 07:08:32,197 DEV : loss 6.226396560668945 - score 0.9186 +2021-03-26 07:08:32,221 BAD EPOCHS (no improvement): 2 +2021-03-26 07:08:32,222 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:08:34,197 epoch 65 - iter 5/50 - loss 2.37803488 - samples/sec: 81.13 - lr: 0.000977 +2021-03-26 07:08:36,191 epoch 65 - iter 10/50 - loss 2.32778922 - samples/sec: 80.32 - lr: 0.000977 +2021-03-26 07:08:38,083 epoch 65 - iter 15/50 - loss 2.34324029 - samples/sec: 84.66 - lr: 0.000977 +2021-03-26 07:08:40,155 epoch 65 - iter 20/50 - loss 2.35554613 - samples/sec: 77.26 - lr: 0.000977 +2021-03-26 07:08:42,065 epoch 65 - iter 25/50 - loss 2.43224439 - samples/sec: 83.88 - lr: 0.000977 +2021-03-26 07:08:43,994 epoch 65 - iter 30/50 - loss 2.49575868 - samples/sec: 83.04 - lr: 0.000977 +2021-03-26 07:08:45,917 epoch 65 - iter 35/50 - loss 2.46905589 - samples/sec: 83.30 - lr: 0.000977 +2021-03-26 07:08:48,010 epoch 65 - iter 40/50 - loss 2.46037631 - samples/sec: 76.53 - lr: 0.000977 +2021-03-26 07:08:49,954 epoch 65 - iter 45/50 - loss 2.51430530 - samples/sec: 82.34 - lr: 0.000977 +2021-03-26 07:08:51,587 epoch 65 - iter 50/50 - loss 2.51490304 - samples/sec: 98.08 - lr: 0.000977 +2021-03-26 07:08:51,588 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:08:51,589 EPOCH 65 done: loss 2.5149 - lr 0.0009766 +2021-03-26 07:08:52,356 DEV : loss 6.223830223083496 - score 0.9182 +2021-03-26 07:08:52,377 BAD EPOCHS (no improvement): 3 +2021-03-26 07:08:52,378 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:08:54,353 epoch 66 - iter 5/50 - loss 2.32253141 - samples/sec: 81.07 - lr: 0.000977 +2021-03-26 07:08:56,536 epoch 66 - iter 10/50 - loss 2.18922327 - samples/sec: 73.34 - lr: 0.000977 +2021-03-26 07:08:58,433 epoch 66 - iter 15/50 - loss 2.25993318 - samples/sec: 84.45 - lr: 0.000977 +2021-03-26 07:09:00,259 epoch 66 - iter 20/50 - loss 2.23364434 - samples/sec: 87.70 - lr: 0.000977 +2021-03-26 07:09:02,355 epoch 66 - iter 25/50 - loss 2.26889848 - samples/sec: 76.41 - lr: 0.000977 +2021-03-26 07:09:04,329 epoch 66 - iter 30/50 - loss 2.27924853 - samples/sec: 81.15 - lr: 0.000977 +2021-03-26 07:09:06,233 epoch 66 - iter 35/50 - loss 2.26974100 - samples/sec: 84.09 - lr: 0.000977 +2021-03-26 07:09:08,058 epoch 66 - iter 40/50 - loss 2.27447847 - samples/sec: 87.76 - lr: 0.000977 +2021-03-26 07:09:09,847 epoch 66 - iter 45/50 - loss 2.25675712 - samples/sec: 89.55 - lr: 0.000977 +2021-03-26 07:09:11,649 epoch 66 - iter 50/50 - loss 2.29599657 - samples/sec: 88.85 - lr: 0.000977 +2021-03-26 07:09:11,650 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:09:11,650 EPOCH 66 done: loss 2.2960 - lr 0.0009766 +2021-03-26 07:09:12,413 DEV : loss 6.223401069641113 - score 0.9182 +2021-03-26 07:09:12,438 BAD EPOCHS (no improvement): 4 +2021-03-26 07:09:12,439 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:09:14,400 epoch 67 - iter 5/50 - loss 2.42672076 - samples/sec: 81.68 - lr: 0.000488 +2021-03-26 07:09:16,353 epoch 67 - iter 10/50 - loss 2.32563019 - samples/sec: 82.02 - lr: 0.000488 +2021-03-26 07:09:18,545 epoch 67 - iter 15/50 - loss 2.35495213 - samples/sec: 73.06 - lr: 0.000488 +2021-03-26 07:09:20,412 epoch 67 - iter 20/50 - loss 2.39149110 - samples/sec: 85.75 - lr: 0.000488 +2021-03-26 07:09:22,504 epoch 67 - iter 25/50 - loss 2.39185100 - samples/sec: 76.56 - lr: 0.000488 +2021-03-26 07:09:24,438 epoch 67 - iter 30/50 - loss 2.41217314 - samples/sec: 82.80 - lr: 0.000488 +2021-03-26 07:09:26,420 epoch 67 - iter 35/50 - loss 2.40400888 - samples/sec: 80.81 - lr: 0.000488 +2021-03-26 07:09:28,313 epoch 67 - iter 40/50 - loss 2.38190263 - samples/sec: 84.60 - lr: 0.000488 +2021-03-26 07:09:30,201 epoch 67 - iter 45/50 - loss 2.38458229 - samples/sec: 84.82 - lr: 0.000488 +2021-03-26 07:09:32,145 epoch 67 - iter 50/50 - loss 2.40125116 - samples/sec: 82.40 - lr: 0.000488 +2021-03-26 07:09:32,145 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:09:32,146 EPOCH 67 done: loss 2.4013 - lr 0.0004883 +2021-03-26 07:09:32,925 DEV : loss 6.223155498504639 - score 0.9182 +2021-03-26 07:09:32,951 BAD EPOCHS (no improvement): 1 +2021-03-26 07:09:32,951 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:09:35,054 epoch 68 - iter 5/50 - loss 2.63773403 - samples/sec: 76.19 - lr: 0.000488 +2021-03-26 07:09:36,951 epoch 68 - iter 10/50 - loss 2.35623841 - samples/sec: 84.40 - lr: 0.000488 +2021-03-26 07:09:38,783 epoch 68 - iter 15/50 - loss 2.23468730 - samples/sec: 87.43 - lr: 0.000488 +2021-03-26 07:09:40,653 epoch 68 - iter 20/50 - loss 2.27483031 - samples/sec: 85.62 - lr: 0.000488 +2021-03-26 07:09:42,595 epoch 68 - iter 25/50 - loss 2.26047467 - samples/sec: 82.49 - lr: 0.000488 +2021-03-26 07:09:44,477 epoch 68 - iter 30/50 - loss 2.24738997 - samples/sec: 85.11 - lr: 0.000488 +2021-03-26 07:09:46,274 epoch 68 - iter 35/50 - loss 2.27638966 - samples/sec: 89.09 - lr: 0.000488 +2021-03-26 07:09:48,185 epoch 68 - iter 40/50 - loss 2.37632107 - samples/sec: 83.80 - lr: 0.000488 +2021-03-26 07:09:50,100 epoch 68 - iter 45/50 - loss 2.42216489 - samples/sec: 83.62 - lr: 0.000488 +2021-03-26 07:09:51,873 epoch 68 - iter 50/50 - loss 2.41903520 - samples/sec: 90.36 - lr: 0.000488 +2021-03-26 07:09:51,873 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:09:51,873 EPOCH 68 done: loss 2.4190 - lr 0.0004883 +2021-03-26 07:09:52,634 DEV : loss 6.222766876220703 - score 0.9182 +2021-03-26 07:09:52,659 BAD EPOCHS (no improvement): 2 +2021-03-26 07:09:52,659 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:09:54,578 epoch 69 - iter 5/50 - loss 2.19501238 - samples/sec: 83.46 - lr: 0.000488 +2021-03-26 07:09:56,609 epoch 69 - iter 10/50 - loss 2.28023877 - samples/sec: 78.86 - lr: 0.000488 +2021-03-26 07:09:58,571 epoch 69 - iter 15/50 - loss 2.18687286 - samples/sec: 81.64 - lr: 0.000488 +2021-03-26 07:10:00,551 epoch 69 - iter 20/50 - loss 2.21458565 - samples/sec: 80.88 - lr: 0.000488 +2021-03-26 07:10:02,397 epoch 69 - iter 25/50 - loss 2.32697690 - samples/sec: 86.77 - lr: 0.000488 +2021-03-26 07:10:04,341 epoch 69 - iter 30/50 - loss 2.25094236 - samples/sec: 82.37 - lr: 0.000488 +2021-03-26 07:10:06,141 epoch 69 - iter 35/50 - loss 2.25566019 - samples/sec: 88.97 - lr: 0.000488 +2021-03-26 07:10:07,971 epoch 69 - iter 40/50 - loss 2.31639599 - samples/sec: 87.50 - lr: 0.000488 +2021-03-26 07:10:09,862 epoch 69 - iter 45/50 - loss 2.33205634 - samples/sec: 84.72 - lr: 0.000488 +2021-03-26 07:10:11,657 epoch 69 - iter 50/50 - loss 2.35981682 - samples/sec: 89.25 - lr: 0.000488 +2021-03-26 07:10:11,657 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:10:11,658 EPOCH 69 done: loss 2.3598 - lr 0.0004883 +2021-03-26 07:10:12,476 DEV : loss 6.222655296325684 - score 0.9178 +2021-03-26 07:10:12,507 BAD EPOCHS (no improvement): 3 +2021-03-26 07:10:12,508 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:10:14,565 epoch 70 - iter 5/50 - loss 2.74699225 - samples/sec: 77.88 - lr: 0.000488 +2021-03-26 07:10:16,709 epoch 70 - iter 10/50 - loss 2.38880478 - samples/sec: 74.68 - lr: 0.000488 +2021-03-26 07:10:18,691 epoch 70 - iter 15/50 - loss 2.40105462 - samples/sec: 80.81 - lr: 0.000488 +2021-03-26 07:10:20,770 epoch 70 - iter 20/50 - loss 2.48934642 - samples/sec: 77.01 - lr: 0.000488 +2021-03-26 07:10:22,632 epoch 70 - iter 25/50 - loss 2.50072574 - samples/sec: 86.01 - lr: 0.000488 +2021-03-26 07:10:24,441 epoch 70 - iter 30/50 - loss 2.47702555 - samples/sec: 88.52 - lr: 0.000488 +2021-03-26 07:10:26,167 epoch 70 - iter 35/50 - loss 2.47407045 - samples/sec: 92.78 - lr: 0.000488 +2021-03-26 07:10:28,018 epoch 70 - iter 40/50 - loss 2.45811075 - samples/sec: 86.51 - lr: 0.000488 +2021-03-26 07:10:29,873 epoch 70 - iter 45/50 - loss 2.47587674 - samples/sec: 86.34 - lr: 0.000488 +2021-03-26 07:10:31,727 epoch 70 - iter 50/50 - loss 2.49858140 - samples/sec: 86.41 - lr: 0.000488 +2021-03-26 07:10:31,728 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:10:31,728 EPOCH 70 done: loss 2.4986 - lr 0.0004883 +2021-03-26 07:10:32,504 DEV : loss 6.222859859466553 - score 0.9178 +2021-03-26 07:10:32,528 BAD EPOCHS (no improvement): 4 +2021-03-26 07:10:32,529 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:10:34,406 epoch 71 - iter 5/50 - loss 2.86681695 - samples/sec: 85.36 - lr: 0.000244 +2021-03-26 07:10:36,252 epoch 71 - iter 10/50 - loss 2.64170929 - samples/sec: 86.77 - lr: 0.000244 +2021-03-26 07:10:38,417 epoch 71 - iter 15/50 - loss 2.50964533 - samples/sec: 73.98 - lr: 0.000244 +2021-03-26 07:10:40,297 epoch 71 - iter 20/50 - loss 2.50780855 - samples/sec: 85.19 - lr: 0.000244 +2021-03-26 07:10:42,640 epoch 71 - iter 25/50 - loss 2.55021320 - samples/sec: 68.33 - lr: 0.000244 +2021-03-26 07:10:44,470 epoch 71 - iter 30/50 - loss 2.58371531 - samples/sec: 87.56 - lr: 0.000244 +2021-03-26 07:10:46,490 epoch 71 - iter 35/50 - loss 2.58910989 - samples/sec: 79.27 - lr: 0.000244 +2021-03-26 07:10:48,399 epoch 71 - iter 40/50 - loss 2.56937031 - samples/sec: 84.06 - lr: 0.000244 +2021-03-26 07:10:50,358 epoch 71 - iter 45/50 - loss 2.55463270 - samples/sec: 81.73 - lr: 0.000244 +2021-03-26 07:10:52,191 epoch 71 - iter 50/50 - loss 2.50627966 - samples/sec: 87.38 - lr: 0.000244 +2021-03-26 07:10:52,192 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:10:52,193 EPOCH 71 done: loss 2.5063 - lr 0.0002441 +2021-03-26 07:10:52,969 DEV : loss 6.223049163818359 - score 0.9178 +2021-03-26 07:10:52,986 BAD EPOCHS (no improvement): 1 +2021-03-26 07:10:52,986 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:10:54,835 epoch 72 - iter 5/50 - loss 2.39624953 - samples/sec: 86.62 - lr: 0.000244 +2021-03-26 07:10:56,682 epoch 72 - iter 10/50 - loss 2.36824905 - samples/sec: 86.74 - lr: 0.000244 +2021-03-26 07:10:58,365 epoch 72 - iter 15/50 - loss 2.43008339 - samples/sec: 95.10 - lr: 0.000244 +2021-03-26 07:11:00,466 epoch 72 - iter 20/50 - loss 2.38753402 - samples/sec: 76.22 - lr: 0.000244 +2021-03-26 07:11:02,453 epoch 72 - iter 25/50 - loss 2.36955923 - samples/sec: 80.62 - lr: 0.000244 +2021-03-26 07:11:04,570 epoch 72 - iter 30/50 - loss 2.41096191 - samples/sec: 75.66 - lr: 0.000244 +2021-03-26 07:11:06,661 epoch 72 - iter 35/50 - loss 2.52839747 - samples/sec: 76.70 - lr: 0.000244 +2021-03-26 07:11:08,590 epoch 72 - iter 40/50 - loss 2.53110349 - samples/sec: 83.06 - lr: 0.000244 +2021-03-26 07:11:10,380 epoch 72 - iter 45/50 - loss 2.48378272 - samples/sec: 89.48 - lr: 0.000244 +2021-03-26 07:11:12,240 epoch 72 - iter 50/50 - loss 2.47316787 - samples/sec: 86.09 - lr: 0.000244 +2021-03-26 07:11:12,241 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:11:12,241 EPOCH 72 done: loss 2.4732 - lr 0.0002441 +2021-03-26 07:11:13,002 DEV : loss 6.223275661468506 - score 0.9178 +2021-03-26 07:11:13,027 BAD EPOCHS (no improvement): 2 +2021-03-26 07:11:13,028 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:11:14,920 epoch 73 - iter 5/50 - loss 2.70434468 - samples/sec: 84.67 - lr: 0.000244 +2021-03-26 07:11:16,812 epoch 73 - iter 10/50 - loss 2.45531214 - samples/sec: 84.65 - lr: 0.000244 +2021-03-26 07:11:18,591 epoch 73 - iter 15/50 - loss 2.30294273 - samples/sec: 90.03 - lr: 0.000244 +2021-03-26 07:11:20,686 epoch 73 - iter 20/50 - loss 2.35064186 - samples/sec: 76.42 - lr: 0.000244 +2021-03-26 07:11:22,626 epoch 73 - iter 25/50 - loss 2.28706394 - samples/sec: 82.57 - lr: 0.000244 +2021-03-26 07:11:24,619 epoch 73 - iter 30/50 - loss 2.27880787 - samples/sec: 80.36 - lr: 0.000244 +2021-03-26 07:11:26,533 epoch 73 - iter 35/50 - loss 2.33573643 - samples/sec: 83.68 - lr: 0.000244 +2021-03-26 07:11:28,426 epoch 73 - iter 40/50 - loss 2.39343770 - samples/sec: 84.60 - lr: 0.000244 +2021-03-26 07:11:30,385 epoch 73 - iter 45/50 - loss 2.37945244 - samples/sec: 81.77 - lr: 0.000244 +2021-03-26 07:11:32,151 epoch 73 - iter 50/50 - loss 2.40987703 - samples/sec: 90.69 - lr: 0.000244 +2021-03-26 07:11:32,152 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:11:32,152 EPOCH 73 done: loss 2.4099 - lr 0.0002441 +2021-03-26 07:11:32,982 DEV : loss 6.222825527191162 - score 0.9178 +2021-03-26 07:11:33,014 BAD EPOCHS (no improvement): 3 +2021-03-26 07:11:33,015 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:11:35,001 epoch 74 - iter 5/50 - loss 2.38879180 - samples/sec: 80.67 - lr: 0.000244 +2021-03-26 07:11:37,166 epoch 74 - iter 10/50 - loss 2.39860876 - samples/sec: 73.95 - lr: 0.000244 +2021-03-26 07:11:39,146 epoch 74 - iter 15/50 - loss 2.40750088 - samples/sec: 80.90 - lr: 0.000244 +2021-03-26 07:11:41,415 epoch 74 - iter 20/50 - loss 2.48535473 - samples/sec: 70.56 - lr: 0.000244 +2021-03-26 07:11:43,546 epoch 74 - iter 25/50 - loss 2.52484987 - samples/sec: 75.14 - lr: 0.000244 +2021-03-26 07:11:45,422 epoch 74 - iter 30/50 - loss 2.46785964 - samples/sec: 85.38 - lr: 0.000244 +2021-03-26 07:11:47,381 epoch 74 - iter 35/50 - loss 2.49764286 - samples/sec: 81.75 - lr: 0.000244 +2021-03-26 07:11:49,942 epoch 74 - iter 40/50 - loss 2.50870748 - samples/sec: 62.51 - lr: 0.000244 +2021-03-26 07:11:52,122 epoch 74 - iter 45/50 - loss 2.45298528 - samples/sec: 73.46 - lr: 0.000244 +2021-03-26 07:11:54,078 epoch 74 - iter 50/50 - loss 2.48597799 - samples/sec: 81.88 - lr: 0.000244 +2021-03-26 07:11:54,079 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:11:54,079 EPOCH 74 done: loss 2.4860 - lr 0.0002441 +2021-03-26 07:11:54,879 DEV : loss 6.222982406616211 - score 0.9174 +2021-03-26 07:11:54,899 BAD EPOCHS (no improvement): 4 +2021-03-26 07:11:54,900 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:11:56,828 epoch 75 - iter 5/50 - loss 2.88134527 - samples/sec: 83.06 - lr: 0.000122 +2021-03-26 07:11:58,775 epoch 75 - iter 10/50 - loss 2.43894054 - samples/sec: 82.27 - lr: 0.000122 +2021-03-26 07:12:00,731 epoch 75 - iter 15/50 - loss 2.42260317 - samples/sec: 81.87 - lr: 0.000122 +2021-03-26 07:12:02,762 epoch 75 - iter 20/50 - loss 2.35718715 - samples/sec: 78.84 - lr: 0.000122 +2021-03-26 07:12:04,664 epoch 75 - iter 25/50 - loss 2.36030079 - samples/sec: 84.24 - lr: 0.000122 +2021-03-26 07:12:06,571 epoch 75 - iter 30/50 - loss 2.34997718 - samples/sec: 83.97 - lr: 0.000122 +2021-03-26 07:12:08,758 epoch 75 - iter 35/50 - loss 2.34546312 - samples/sec: 73.21 - lr: 0.000122 +2021-03-26 07:12:10,806 epoch 75 - iter 40/50 - loss 2.33144889 - samples/sec: 78.21 - lr: 0.000122 +2021-03-26 07:12:12,805 epoch 75 - iter 45/50 - loss 2.33342309 - samples/sec: 80.11 - lr: 0.000122 +2021-03-26 07:12:15,445 epoch 75 - iter 50/50 - loss 2.34604343 - samples/sec: 60.65 - lr: 0.000122 +2021-03-26 07:12:15,446 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:12:15,446 EPOCH 75 done: loss 2.3460 - lr 0.0001221 +2021-03-26 07:12:16,193 DEV : loss 6.222970962524414 - score 0.9174 +2021-03-26 07:12:16,215 BAD EPOCHS (no improvement): 1 +2021-03-26 07:12:16,216 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:12:17,898 epoch 76 - iter 5/50 - loss 2.58834620 - samples/sec: 95.27 - lr: 0.000122 +2021-03-26 07:12:19,610 epoch 76 - iter 10/50 - loss 2.46520443 - samples/sec: 93.58 - lr: 0.000122 +2021-03-26 07:12:21,781 epoch 76 - iter 15/50 - loss 2.60440030 - samples/sec: 73.76 - lr: 0.000122 +2021-03-26 07:12:23,690 epoch 76 - iter 20/50 - loss 2.48082805 - samples/sec: 83.89 - lr: 0.000122 +2021-03-26 07:12:25,820 epoch 76 - iter 25/50 - loss 2.49374959 - samples/sec: 75.17 - lr: 0.000122 +2021-03-26 07:12:28,936 epoch 76 - iter 30/50 - loss 2.49200382 - samples/sec: 51.38 - lr: 0.000122 +2021-03-26 07:12:31,047 epoch 76 - iter 35/50 - loss 2.49267069 - samples/sec: 75.84 - lr: 0.000122 +2021-03-26 07:12:32,928 epoch 76 - iter 40/50 - loss 2.51199569 - samples/sec: 85.16 - lr: 0.000122 +2021-03-26 07:12:34,799 epoch 76 - iter 45/50 - loss 2.55731874 - samples/sec: 85.61 - lr: 0.000122 +2021-03-26 07:12:36,678 epoch 76 - iter 50/50 - loss 2.53093982 - samples/sec: 85.21 - lr: 0.000122 +2021-03-26 07:12:36,679 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:12:36,679 EPOCH 76 done: loss 2.5309 - lr 0.0001221 +2021-03-26 07:12:37,482 DEV : loss 6.222939968109131 - score 0.9178 +2021-03-26 07:12:37,503 BAD EPOCHS (no improvement): 2 +2021-03-26 07:12:37,504 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:12:39,530 epoch 77 - iter 5/50 - loss 3.01930332 - samples/sec: 79.08 - lr: 0.000122 +2021-03-26 07:12:41,490 epoch 77 - iter 10/50 - loss 2.82609091 - samples/sec: 81.71 - lr: 0.000122 +2021-03-26 07:12:43,424 epoch 77 - iter 15/50 - loss 2.54770860 - samples/sec: 82.78 - lr: 0.000122 +2021-03-26 07:12:45,203 epoch 77 - iter 20/50 - loss 2.54026788 - samples/sec: 90.03 - lr: 0.000122 +2021-03-26 07:12:47,071 epoch 77 - iter 25/50 - loss 2.49590103 - samples/sec: 85.77 - lr: 0.000122 +2021-03-26 07:12:49,215 epoch 77 - iter 30/50 - loss 2.51122611 - samples/sec: 74.68 - lr: 0.000122 +2021-03-26 07:12:51,031 epoch 77 - iter 35/50 - loss 2.51107687 - samples/sec: 88.21 - lr: 0.000122 +2021-03-26 07:12:52,890 epoch 77 - iter 40/50 - loss 2.45614245 - samples/sec: 86.11 - lr: 0.000122 +2021-03-26 07:12:54,806 epoch 77 - iter 45/50 - loss 2.47441014 - samples/sec: 83.59 - lr: 0.000122 +2021-03-26 07:12:56,636 epoch 77 - iter 50/50 - loss 2.48978061 - samples/sec: 87.54 - lr: 0.000122 +2021-03-26 07:12:56,637 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:12:56,637 EPOCH 77 done: loss 2.4898 - lr 0.0001221 +2021-03-26 07:12:57,396 DEV : loss 6.222973823547363 - score 0.9178 +2021-03-26 07:12:57,419 BAD EPOCHS (no improvement): 3 +2021-03-26 07:12:57,420 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:12:59,638 epoch 78 - iter 5/50 - loss 2.42860546 - samples/sec: 72.22 - lr: 0.000122 +2021-03-26 07:13:01,613 epoch 78 - iter 10/50 - loss 2.37586279 - samples/sec: 81.09 - lr: 0.000122 +2021-03-26 07:13:03,592 epoch 78 - iter 15/50 - loss 2.51110738 - samples/sec: 80.97 - lr: 0.000122 +2021-03-26 07:13:05,713 epoch 78 - iter 20/50 - loss 2.53387340 - samples/sec: 75.52 - lr: 0.000122 +2021-03-26 07:13:07,510 epoch 78 - iter 25/50 - loss 2.56651618 - samples/sec: 89.11 - lr: 0.000122 +2021-03-26 07:13:09,289 epoch 78 - iter 30/50 - loss 2.50188101 - samples/sec: 90.08 - lr: 0.000122 +2021-03-26 07:13:11,360 epoch 78 - iter 35/50 - loss 2.51097650 - samples/sec: 77.30 - lr: 0.000122 +2021-03-26 07:13:13,312 epoch 78 - iter 40/50 - loss 2.52860793 - samples/sec: 82.06 - lr: 0.000122 +2021-03-26 07:13:15,210 epoch 78 - iter 45/50 - loss 2.50384797 - samples/sec: 84.44 - lr: 0.000122 +2021-03-26 07:13:16,980 epoch 78 - iter 50/50 - loss 2.56408679 - samples/sec: 90.44 - lr: 0.000122 +2021-03-26 07:13:16,981 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:13:16,981 EPOCH 78 done: loss 2.5641 - lr 0.0001221 +2021-03-26 07:13:17,758 DEV : loss 6.22302770614624 - score 0.9178 +2021-03-26 07:13:17,779 BAD EPOCHS (no improvement): 4 +2021-03-26 07:13:17,779 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:13:17,780 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:13:17,780 learning rate too small - quitting training! +2021-03-26 07:13:17,780 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:13:26,869 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:13:26,870 Testing using best model ... +2021-03-26 07:13:26,870 loading file /home/tmp/megahedm/models/multipos/multipos_UDMADAR_4Diale-LEV_EGY_GLF_MGR__fasttext_flairbwfw__32__0.5_202103260643/best-model.pt +2021-03-26 07:13:34,253 0.9029 +2021-03-26 07:13:34,254 +Results: +- F-score (micro): 0.8996 +- F-score (macro): 0.576 +- Accuracy (incl. no class): 0.9029 + +By class: + precision recall f1-score support + + INTJ 0.9375 0.9375 0.9375 16 + NOUN 0.9203 0.9245 0.9224 437 + NUM 0.8000 0.8000 0.8000 15 + ADJ 0.6983 0.8265 0.7570 98 + PRON 0.9751 0.9800 0.9776 200 + VERB 0.9500 0.9301 0.9399 143 + AUX 0.9535 0.9535 0.9535 43 + PROPN 0.8400 0.8750 0.8571 24 + ADV 0.9083 0.8390 0.8722 118 + DET 0.9623 0.9444 0.9533 54 + ADP 0.9909 0.9478 0.9689 115 + SCONJ 0.8387 0.9630 0.8966 27 + PART 0.9333 0.9655 0.9492 174 + CCONJ 0.9545 0.9545 0.9545 88 + PUNCT 1.0000 1.0000 1.0000 30 + V 0.9294 0.8229 0.8729 96 + PREP+PRON 0.8500 0.8947 0.8718 19 + V+PRON+PRON 0.3636 0.3333 0.3478 12 + NOUN+PRON 0.8644 0.8226 0.8430 62 + NOUN+NSUFF 0.9259 0.8621 0.8929 58 + PREP+NOUN+PRON 0.6667 1.0000 0.8000 2 + PUNC 0.9805 1.0000 0.9902 151 + V+PRON 0.7963 0.8113 0.8037 53 + V+PRON+PREP+PRON 1.0000 0.0000 0.0000 2 + V+PREP+PRON 0.6000 0.6000 0.6000 5 + DET+NOUN+NSUFF 0.7619 0.8889 0.8205 18 + NOUN+NSUFF+PRON 0.6957 0.8421 0.7619 19 + CONJ 0.9714 0.9714 0.9714 35 + FOREIGN 1.0000 0.8333 0.9091 6 + MENTION 0.8800 1.0000 0.9362 22 + EOS 1.0000 1.0000 1.0000 70 + PREP 1.0000 0.9714 0.9855 70 + DET+NOUN 0.9315 0.9577 0.9444 71 + PROG_PART+V 0.8529 0.7632 0.8056 38 + CONJ+NOUN+PRON 0.5000 0.3333 0.4000 3 + CONJ+NOUN 0.7500 0.9000 0.8182 10 + ADJ+NSUFF 0.7879 0.8387 0.8125 31 + PART+PRON 1.0000 1.0000 1.0000 15 + PREP+DET+NOUN 0.9412 1.0000 0.9697 16 + CONJ+V+PRON 0.7500 0.8571 0.8000 7 + PROG_PART+V+PRON 0.6471 0.9167 0.7586 12 + CONJ+V 0.7500 0.6667 0.7059 9 + ADJ+PRON 0.2500 0.2500 0.2500 4 + FUT_PART 1.0000 1.0000 1.0000 5 + PRON+DET+NOUN+NSUFF 1.0000 1.0000 1.0000 1 + DET+ADJ+NSUFF 0.6667 0.4000 0.5000 5 + PREP+DET+NUM+NSUFF 1.0000 0.0000 0.0000 1 + PREP+NOUN+NSUFF 0.7500 1.0000 0.8571 3 + NOUN+CASE 1.0000 1.0000 1.0000 3 + PREP+PART 1.0000 1.0000 1.0000 1 + FUT_PART+V 0.8571 0.5455 0.6667 11 + PREP+NOUN 0.2727 0.5000 0.3529 6 + CONJ+DET+NOUN 1.0000 1.0000 1.0000 5 + PREP+PRON+DET 1.0000 0.0000 0.0000 1 + HASH 0.9375 0.8333 0.8824 18 + PART+V+PRON+PRON 0.0000 1.0000 0.0000 0 + PREP+DET+NOUN+NSUFF 1.0000 0.5000 0.6667 4 + EMOT 1.0000 0.9062 0.9508 32 + CONJ+FUT_PART+V 1.0000 0.2500 0.4000 4 + CONJ+PROG_PART+V 0.1667 1.0000 0.2857 1 + CONJ+NOUN+NSUFF 1.0000 0.3333 0.5000 3 + CONJ+ADJ 0.0000 1.0000 0.0000 0 + DET+ADJ 0.5000 0.6000 0.5455 5 + CONJ+PRON 1.0000 1.0000 1.0000 6 + PREP+V 1.0000 0.5000 0.6667 2 + CONJ+PREP+V 1.0000 0.0000 0.0000 1 + PREP+V+PRON 1.0000 0.0000 0.0000 1 + URL 1.0000 1.0000 1.0000 3 + PRON+DET+NOUN 1.0000 1.0000 1.0000 1 + CONJ+PREP 1.0000 0.6667 0.8000 3 + CONJ+PART 0.7500 0.8571 0.8000 7 + CONJ+PREP+NOUN 1.0000 0.0000 0.0000 1 + PREP+NOUN+NSUFF+PRON 1.0000 1.0000 1.0000 3 + FUT_PART+V+PREP+PRON 0.0000 1.0000 0.0000 0 + V+NEG_PART 1.0000 0.0000 0.0000 1 + PROG_PART+V+NEG_PART 1.0000 1.0000 1.0000 1 + ADJ+CASE 0.3333 1.0000 0.5000 1 + PREP+PART+PRON 1.0000 1.0000 1.0000 2 + ADJ+PREP+PRON 1.0000 0.0000 0.0000 1 + PART+NOUN+PRON 0.0000 1.0000 0.0000 0 + PART+V 1.0000 0.0000 0.0000 1 + PART+ADJ+PRON+PRON 1.0000 0.0000 0.0000 1 + PART+NOUN 0.8333 0.8333 0.8333 6 + CONJ+ADV 1.0000 0.5000 0.6667 2 + CONJ+ADJ+NSUFF 0.0000 1.0000 0.0000 0 + PREP+PROG_PART+V+PRON 1.0000 0.0000 0.0000 1 + PART+PROG_PART+V 1.0000 0.0000 0.0000 1 + PART+V+NEG_PART 0.0000 0.0000 0.0000 6 + PART+V+PRON+NEG_PART 0.2000 1.0000 0.3333 2 + PROG_PART+V+PREP+PRON 1.0000 0.0000 0.0000 1 + PART+NOUN+PRON+NEG_PART 1.0000 0.0000 0.0000 1 + NUM+NSUFF 1.0000 0.0000 0.0000 2 + PART+PREP+NEG_PART 0.7500 1.0000 0.8571 3 + FUT_PART+V+PRON 0.5000 1.0000 0.6667 1 + PART+PROG_PART+V+NEG_PART 0.3333 0.3333 0.3333 3 + DET+NUM 1.0000 0.0000 0.0000 2 + PART+NOUN+NEG_PART 1.0000 0.0000 0.0000 1 + ADJ+NSUFF+PRON 0.0000 1.0000 0.0000 0 + PART+V+PRON+PRON+NEG_PART 1.0000 0.0000 0.0000 1 + PART+PROG_PART+V+PRON+NEG_PART 0.0000 1.0000 0.0000 0 +CONJ+PART+PROG_PART+V+PREP+PRON+NEG_PART 1.0000 0.0000 0.0000 1 + CONJ+PART+V+PRON+NEG_PART 0.0000 1.0000 0.0000 0 + ADV+NSUFF 1.0000 1.0000 1.0000 1 + PART+V+PRON 1.0000 0.0000 0.0000 1 + PREP+PRON+DET+NOUN 0.0000 1.0000 0.0000 0 + + micro avg 0.8996 0.8996 0.8996 2679 + macro avg 0.7806 0.6813 0.5760 2679 + weighted avg 0.9102 0.8996 0.8981 2679 + +2021-03-26 07:13:34,254 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:13:34,254 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:13:38,055 Reading data from ../../Datasets_adhoc/CSCS_corpus-GUC +2021-03-26 07:13:38,055 Train: ../../Datasets_adhoc/CSCS_corpus-GUC/all_participants.conllu +2021-03-26 07:13:38,056 Dev: None +2021-03-26 07:13:38,056 Test: None +2021-03-26 07:13:38,351 Reading data from ../../Datasets_adhoc/UD_MADAR +2021-03-26 07:13:38,351 Train: ../../Datasets_adhoc/UD_MADAR/ajp_madar-ud-test-edit.conllu +2021-03-26 07:13:38,351 Dev: None +2021-03-26 07:13:38,352 Test: None +2021-03-26 07:13:38,384 Reading data from ../../Datasets_adhoc/dialectal_arabic_resources-master +2021-03-26 07:13:38,384 Train: ../../Datasets_adhoc/dialectal_arabic_resources-master/seg_plus_pos_lev.txt +2021-03-26 07:13:38,384 Dev: None +2021-03-26 07:13:38,385 Test: None +2021-03-26 07:13:38,564 Reading data from ../../Datasets_adhoc/dialectal_arabic_resources-master +2021-03-26 07:13:38,565 Train: ../../Datasets_adhoc/dialectal_arabic_resources-master/seg_plus_pos_egy.txt +2021-03-26 07:13:38,565 Dev: None +2021-03-26 07:13:38,566 Test: None +2021-03-26 07:13:40,620 Reading data from ../../Datasets_adhoc/dialectal_arabic_resources-master +2021-03-26 07:13:40,621 Train: ../../Datasets_adhoc/dialectal_arabic_resources-master/seg_plus_pos_glf.txt +2021-03-26 07:13:40,621 Dev: None +2021-03-26 07:13:40,621 Test: None +2021-03-26 07:13:40,776 Reading data from ../../Datasets_adhoc/dialectal_arabic_resources-master +2021-03-26 07:13:40,776 Train: ../../Datasets_adhoc/dialectal_arabic_resources-master/seg_plus_pos_mgr.txt +2021-03-26 07:13:40,777 Dev: None +2021-03-26 07:13:40,777 Test: None +2021-03-26 07:13:40,916 Filtering long sentences +2021-03-26 07:13:40,958 MultiCorpus: 1574 train + 177 dev + 193 test sentences + - ColumnCorpus Corpus: 934 train + 104 dev + 115 test sentences + - ColumnCorpus Corpus: 81 train + 9 dev + 10 test sentences + - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences + - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences +2021-03-26 07:13:41,340 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:13:41,340 Model: "SequenceTagger( + (embeddings): StackedEmbeddings( + (list_embedding_0): WordEmbeddings('ar') + (list_embedding_1): FlairEmbeddings( + (lm): LanguageModel( + (drop): Dropout(p=0.1, inplace=False) + (encoder): Embedding(7125, 100) + (rnn): LSTM(100, 2048) + (decoder): Linear(in_features=2048, out_features=7125, bias=True) + ) + ) + (list_embedding_2): FlairEmbeddings( + (lm): LanguageModel( + (drop): Dropout(p=0.1, inplace=False) + (encoder): Embedding(7125, 100) + (rnn): LSTM(100, 2048) + (decoder): Linear(in_features=2048, out_features=7125, bias=True) + ) + ) + ) + (word_dropout): WordDropout(p=0.05) + (locked_dropout): LockedDropout(p=0.5) + (embedding2nn): Linear(in_features=4396, out_features=4396, bias=True) + (rnn): LSTM(4396, 256, batch_first=True, bidirectional=True) + (linear): Linear(in_features=512, out_features=206, bias=True) + (beta): 1.0 + (weights): None + (weight_tensor) None +)" +2021-03-26 07:13:41,341 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:13:41,341 Corpus: "MultiCorpus: 1574 train + 177 dev + 193 test sentences + - ColumnCorpus Corpus: 934 train + 104 dev + 115 test sentences + - ColumnCorpus Corpus: 81 train + 9 dev + 10 test sentences + - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences + - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences" +2021-03-26 07:13:41,341 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:13:41,342 Parameters: +2021-03-26 07:13:41,342 - learning_rate: "0.5" +2021-03-26 07:13:41,342 - mini_batch_size: "32" +2021-03-26 07:13:41,343 - patience: "3" +2021-03-26 07:13:41,343 - anneal_factor: "0.5" +2021-03-26 07:13:41,343 - max_epochs: "150" +2021-03-26 07:13:41,344 - shuffle: "True" +2021-03-26 07:13:41,344 - train_with_dev: "False" +2021-03-26 07:13:41,344 - batch_growth_annealing: "False" +2021-03-26 07:13:41,344 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:13:41,345 Model training base path: "/home/tmp/megahedm/models/multipos/multipos_UDMADAR_4Diale-LEV_EGY_GLF_MGR__fasttext_flairbwfw__32__0.5_202103260713" +2021-03-26 07:13:41,345 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:13:41,345 Device: cuda:0 +2021-03-26 07:13:41,346 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:13:41,346 Embeddings storage mode: cpu +2021-03-26 07:13:41,347 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:13:44,081 epoch 1 - iter 5/50 - loss 85.61149902 - samples/sec: 58.57 - lr: 0.500000 +2021-03-26 07:13:46,470 epoch 1 - iter 10/50 - loss 77.91009293 - samples/sec: 67.01 - lr: 0.500000 +2021-03-26 07:13:48,885 epoch 1 - iter 15/50 - loss 71.52193502 - samples/sec: 66.30 - lr: 0.500000 +2021-03-26 07:13:51,430 epoch 1 - iter 20/50 - loss 67.45240936 - samples/sec: 62.91 - lr: 0.500000 +2021-03-26 07:13:53,859 epoch 1 - iter 25/50 - loss 63.35490433 - samples/sec: 65.91 - lr: 0.500000 +2021-03-26 07:13:56,148 epoch 1 - iter 30/50 - loss 59.82105052 - samples/sec: 69.96 - lr: 0.500000 +2021-03-26 07:13:58,638 epoch 1 - iter 35/50 - loss 57.17276971 - samples/sec: 64.30 - lr: 0.500000 +2021-03-26 07:14:01,151 epoch 1 - iter 40/50 - loss 54.94282775 - samples/sec: 63.73 - lr: 0.500000 +2021-03-26 07:14:03,378 epoch 1 - iter 45/50 - loss 52.87651193 - samples/sec: 71.90 - lr: 0.500000 +2021-03-26 07:14:05,763 epoch 1 - iter 50/50 - loss 50.76319237 - samples/sec: 67.11 - lr: 0.500000 +2021-03-26 07:14:05,764 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:14:05,765 EPOCH 1 done: loss 50.7632 - lr 0.5000000 +2021-03-26 07:14:07,067 DEV : loss 29.096771240234375 - score 0.5047 +2021-03-26 07:14:07,088 BAD EPOCHS (no improvement): 0 +2021-03-26 07:14:16,389 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:14:18,245 epoch 2 - iter 5/50 - loss 28.80629158 - samples/sec: 86.31 - lr: 0.500000 +2021-03-26 07:14:20,218 epoch 2 - iter 10/50 - loss 28.44908276 - samples/sec: 81.18 - lr: 0.500000 +2021-03-26 07:14:22,159 epoch 2 - iter 15/50 - loss 28.31970825 - samples/sec: 82.53 - lr: 0.500000 +2021-03-26 07:14:24,044 epoch 2 - iter 20/50 - loss 27.87325191 - samples/sec: 84.93 - lr: 0.500000 +2021-03-26 07:14:26,137 epoch 2 - iter 25/50 - loss 27.86790703 - samples/sec: 76.55 - lr: 0.500000 +2021-03-26 07:14:27,876 epoch 2 - iter 30/50 - loss 27.33799852 - samples/sec: 92.06 - lr: 0.500000 +2021-03-26 07:14:29,563 epoch 2 - iter 35/50 - loss 26.45283541 - samples/sec: 94.96 - lr: 0.500000 +2021-03-26 07:14:31,480 epoch 2 - iter 40/50 - loss 26.06411510 - samples/sec: 83.53 - lr: 0.500000 +2021-03-26 07:14:33,605 epoch 2 - iter 45/50 - loss 25.72350714 - samples/sec: 75.35 - lr: 0.500000 +2021-03-26 07:14:35,452 epoch 2 - iter 50/50 - loss 25.18292456 - samples/sec: 86.72 - lr: 0.500000 +2021-03-26 07:14:35,453 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:14:35,453 EPOCH 2 done: loss 25.1829 - lr 0.5000000 +2021-03-26 07:14:36,255 DEV : loss 17.214523315429688 - score 0.7089 +2021-03-26 07:14:36,278 BAD EPOCHS (no improvement): 0 +2021-03-26 07:14:45,899 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:14:47,996 epoch 3 - iter 5/50 - loss 21.90074081 - samples/sec: 76.41 - lr: 0.500000 +2021-03-26 07:14:49,788 epoch 3 - iter 10/50 - loss 20.25820999 - samples/sec: 89.43 - lr: 0.500000 +2021-03-26 07:14:51,799 epoch 3 - iter 15/50 - loss 20.09863516 - samples/sec: 79.61 - lr: 0.500000 +2021-03-26 07:14:53,730 epoch 3 - iter 20/50 - loss 19.78721623 - samples/sec: 82.94 - lr: 0.500000 +2021-03-26 07:14:55,675 epoch 3 - iter 25/50 - loss 19.39301216 - samples/sec: 82.37 - lr: 0.500000 +2021-03-26 07:14:57,547 epoch 3 - iter 30/50 - loss 18.84088933 - samples/sec: 85.52 - lr: 0.500000 +2021-03-26 07:14:59,587 epoch 3 - iter 35/50 - loss 18.47238004 - samples/sec: 78.51 - lr: 0.500000 +2021-03-26 07:15:01,545 epoch 3 - iter 40/50 - loss 18.23260474 - samples/sec: 81.80 - lr: 0.500000 +2021-03-26 07:15:03,478 epoch 3 - iter 45/50 - loss 17.99070036 - samples/sec: 82.85 - lr: 0.500000 +2021-03-26 07:15:05,314 epoch 3 - iter 50/50 - loss 17.81347260 - samples/sec: 87.38 - lr: 0.500000 +2021-03-26 07:15:05,315 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:15:05,315 EPOCH 3 done: loss 17.8135 - lr 0.5000000 +2021-03-26 07:15:06,085 DEV : loss 15.27471923828125 - score 0.7544 +2021-03-26 07:15:06,110 BAD EPOCHS (no improvement): 0 +2021-03-26 07:15:15,597 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:15:17,570 epoch 4 - iter 5/50 - loss 14.33682823 - samples/sec: 81.25 - lr: 0.500000 +2021-03-26 07:15:19,459 epoch 4 - iter 10/50 - loss 14.34253550 - samples/sec: 84.75 - lr: 0.500000 +2021-03-26 07:15:21,335 epoch 4 - iter 15/50 - loss 15.17600791 - samples/sec: 85.40 - lr: 0.500000 +2021-03-26 07:15:23,263 epoch 4 - iter 20/50 - loss 14.81236367 - samples/sec: 83.07 - lr: 0.500000 +2021-03-26 07:15:25,123 epoch 4 - iter 25/50 - loss 14.28828472 - samples/sec: 86.14 - lr: 0.500000 +2021-03-26 07:15:27,060 epoch 4 - iter 30/50 - loss 14.58110215 - samples/sec: 82.66 - lr: 0.500000 +2021-03-26 07:15:29,032 epoch 4 - iter 35/50 - loss 14.54754723 - samples/sec: 81.20 - lr: 0.500000 +2021-03-26 07:15:30,904 epoch 4 - iter 40/50 - loss 14.51193528 - samples/sec: 85.57 - lr: 0.500000 +2021-03-26 07:15:32,835 epoch 4 - iter 45/50 - loss 14.46083404 - samples/sec: 82.91 - lr: 0.500000 +2021-03-26 07:15:34,540 epoch 4 - iter 50/50 - loss 14.43629816 - samples/sec: 94.01 - lr: 0.500000 +2021-03-26 07:15:34,541 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:15:34,542 EPOCH 4 done: loss 14.4363 - lr 0.5000000 +2021-03-26 07:15:35,319 DEV : loss 11.61365795135498 - score 0.7946 +2021-03-26 07:15:35,343 BAD EPOCHS (no improvement): 0 +2021-03-26 07:15:44,962 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:15:46,876 epoch 5 - iter 5/50 - loss 11.90889206 - samples/sec: 83.74 - lr: 0.500000 +2021-03-26 07:15:48,752 epoch 5 - iter 10/50 - loss 11.91375709 - samples/sec: 85.33 - lr: 0.500000 +2021-03-26 07:15:50,891 epoch 5 - iter 15/50 - loss 12.07819735 - samples/sec: 74.86 - lr: 0.500000 +2021-03-26 07:15:52,813 epoch 5 - iter 20/50 - loss 12.28275967 - samples/sec: 83.32 - lr: 0.500000 +2021-03-26 07:15:54,664 epoch 5 - iter 25/50 - loss 12.37690090 - samples/sec: 86.50 - lr: 0.500000 +2021-03-26 07:15:56,634 epoch 5 - iter 30/50 - loss 12.64451520 - samples/sec: 81.32 - lr: 0.500000 +2021-03-26 07:15:58,420 epoch 5 - iter 35/50 - loss 12.54120459 - samples/sec: 89.70 - lr: 0.500000 +2021-03-26 07:16:00,227 epoch 5 - iter 40/50 - loss 12.50868628 - samples/sec: 88.64 - lr: 0.500000 +2021-03-26 07:16:02,407 epoch 5 - iter 45/50 - loss 12.45480209 - samples/sec: 73.47 - lr: 0.500000 +2021-03-26 07:16:04,282 epoch 5 - iter 50/50 - loss 12.34371593 - samples/sec: 85.40 - lr: 0.500000 +2021-03-26 07:16:04,283 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:16:04,284 EPOCH 5 done: loss 12.3437 - lr 0.5000000 +2021-03-26 07:16:05,070 DEV : loss 10.551742553710938 - score 0.8147 +2021-03-26 07:16:05,089 BAD EPOCHS (no improvement): 0 +2021-03-26 07:16:14,569 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:16:16,407 epoch 6 - iter 5/50 - loss 10.71872311 - samples/sec: 87.23 - lr: 0.500000 +2021-03-26 07:16:18,174 epoch 6 - iter 10/50 - loss 10.88694544 - samples/sec: 90.60 - lr: 0.500000 +2021-03-26 07:16:20,041 epoch 6 - iter 15/50 - loss 10.37633387 - samples/sec: 85.80 - lr: 0.500000 +2021-03-26 07:16:22,004 epoch 6 - iter 20/50 - loss 10.93236151 - samples/sec: 81.59 - lr: 0.500000 +2021-03-26 07:16:24,159 epoch 6 - iter 25/50 - loss 11.00346493 - samples/sec: 74.31 - lr: 0.500000 +2021-03-26 07:16:26,102 epoch 6 - iter 30/50 - loss 11.11314979 - samples/sec: 82.41 - lr: 0.500000 +2021-03-26 07:16:27,821 epoch 6 - iter 35/50 - loss 11.19503303 - samples/sec: 93.18 - lr: 0.500000 +2021-03-26 07:16:29,628 epoch 6 - iter 40/50 - loss 11.21847228 - samples/sec: 88.64 - lr: 0.500000 +2021-03-26 07:16:31,576 epoch 6 - iter 45/50 - loss 11.18473713 - samples/sec: 82.23 - lr: 0.500000 +2021-03-26 07:16:33,331 epoch 6 - iter 50/50 - loss 11.14374526 - samples/sec: 91.25 - lr: 0.500000 +2021-03-26 07:16:33,332 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:16:33,332 EPOCH 6 done: loss 11.1437 - lr 0.5000000 +2021-03-26 07:16:34,111 DEV : loss 9.204329490661621 - score 0.8331 +2021-03-26 07:16:34,135 BAD EPOCHS (no improvement): 0 +2021-03-26 07:16:43,722 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:16:45,606 epoch 7 - iter 5/50 - loss 9.51087551 - samples/sec: 85.07 - lr: 0.500000 +2021-03-26 07:16:47,488 epoch 7 - iter 10/50 - loss 9.76788254 - samples/sec: 85.10 - lr: 0.500000 +2021-03-26 07:16:49,407 epoch 7 - iter 15/50 - loss 10.30387751 - samples/sec: 83.47 - lr: 0.500000 +2021-03-26 07:16:51,248 epoch 7 - iter 20/50 - loss 10.07285285 - samples/sec: 86.98 - lr: 0.500000 +2021-03-26 07:16:53,197 epoch 7 - iter 25/50 - loss 9.87292212 - samples/sec: 82.17 - lr: 0.500000 +2021-03-26 07:16:55,852 epoch 7 - iter 30/50 - loss 9.99077923 - samples/sec: 60.30 - lr: 0.500000 +2021-03-26 07:16:58,247 epoch 7 - iter 35/50 - loss 10.18544400 - samples/sec: 66.88 - lr: 0.500000 +2021-03-26 07:17:00,211 epoch 7 - iter 40/50 - loss 10.22825085 - samples/sec: 81.54 - lr: 0.500000 +2021-03-26 07:17:02,108 epoch 7 - iter 45/50 - loss 10.17230478 - samples/sec: 84.45 - lr: 0.500000 +2021-03-26 07:17:03,681 epoch 7 - iter 50/50 - loss 9.96814963 - samples/sec: 101.79 - lr: 0.500000 +2021-03-26 07:17:03,682 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:17:03,682 EPOCH 7 done: loss 9.9681 - lr 0.5000000 +2021-03-26 07:17:04,465 DEV : loss 8.45942211151123 - score 0.8574 +2021-03-26 07:17:04,483 BAD EPOCHS (no improvement): 0 +2021-03-26 07:17:13,790 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:17:15,695 epoch 8 - iter 5/50 - loss 9.15951490 - samples/sec: 84.10 - lr: 0.500000 +2021-03-26 07:17:17,390 epoch 8 - iter 10/50 - loss 8.63786511 - samples/sec: 94.54 - lr: 0.500000 +2021-03-26 07:17:19,199 epoch 8 - iter 15/50 - loss 8.81899722 - samples/sec: 88.52 - lr: 0.500000 +2021-03-26 07:17:20,948 epoch 8 - iter 20/50 - loss 8.66737792 - samples/sec: 91.61 - lr: 0.500000 +2021-03-26 07:17:22,752 epoch 8 - iter 25/50 - loss 8.90464464 - samples/sec: 88.75 - lr: 0.500000 +2021-03-26 07:17:24,692 epoch 8 - iter 30/50 - loss 8.98467898 - samples/sec: 82.60 - lr: 0.500000 +2021-03-26 07:17:26,695 epoch 8 - iter 35/50 - loss 9.11239944 - samples/sec: 79.95 - lr: 0.500000 +2021-03-26 07:17:28,507 epoch 8 - iter 40/50 - loss 9.09702935 - samples/sec: 88.40 - lr: 0.500000 +2021-03-26 07:17:30,483 epoch 8 - iter 45/50 - loss 9.25233818 - samples/sec: 81.03 - lr: 0.500000 +2021-03-26 07:17:32,190 epoch 8 - iter 50/50 - loss 9.19621006 - samples/sec: 93.82 - lr: 0.500000 +2021-03-26 07:17:32,191 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:17:32,192 EPOCH 8 done: loss 9.1962 - lr 0.5000000 +2021-03-26 07:17:33,000 DEV : loss 8.076202392578125 - score 0.8576 +2021-03-26 07:17:33,025 BAD EPOCHS (no improvement): 0 +2021-03-26 07:17:42,590 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:17:44,748 epoch 9 - iter 5/50 - loss 9.24322968 - samples/sec: 74.24 - lr: 0.500000 +2021-03-26 07:17:46,621 epoch 9 - iter 10/50 - loss 9.26248417 - samples/sec: 85.55 - lr: 0.500000 +2021-03-26 07:17:48,630 epoch 9 - iter 15/50 - loss 8.89867210 - samples/sec: 79.69 - lr: 0.500000 +2021-03-26 07:17:50,482 epoch 9 - iter 20/50 - loss 8.52454841 - samples/sec: 86.50 - lr: 0.500000 +2021-03-26 07:17:52,639 epoch 9 - iter 25/50 - loss 8.52478699 - samples/sec: 74.23 - lr: 0.500000 +2021-03-26 07:17:54,490 epoch 9 - iter 30/50 - loss 8.67076422 - samples/sec: 86.55 - lr: 0.500000 +2021-03-26 07:17:56,353 epoch 9 - iter 35/50 - loss 8.56702987 - samples/sec: 85.98 - lr: 0.500000 +2021-03-26 07:17:58,147 epoch 9 - iter 40/50 - loss 8.77756805 - samples/sec: 89.26 - lr: 0.500000 +2021-03-26 07:18:00,078 epoch 9 - iter 45/50 - loss 8.78899221 - samples/sec: 82.94 - lr: 0.500000 +2021-03-26 07:18:01,952 epoch 9 - iter 50/50 - loss 8.89824164 - samples/sec: 85.49 - lr: 0.500000 +2021-03-26 07:18:01,953 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:18:01,953 EPOCH 9 done: loss 8.8982 - lr 0.5000000 +2021-03-26 07:18:02,706 DEV : loss 8.47732162475586 - score 0.8572 +2021-03-26 07:18:02,729 BAD EPOCHS (no improvement): 1 +2021-03-26 07:18:02,730 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:18:04,654 epoch 10 - iter 5/50 - loss 7.55866394 - samples/sec: 83.23 - lr: 0.500000 +2021-03-26 07:18:06,436 epoch 10 - iter 10/50 - loss 7.83560810 - samples/sec: 89.94 - lr: 0.500000 +2021-03-26 07:18:08,507 epoch 10 - iter 15/50 - loss 8.03101371 - samples/sec: 77.30 - lr: 0.500000 +2021-03-26 07:18:10,516 epoch 10 - iter 20/50 - loss 8.23381851 - samples/sec: 79.72 - lr: 0.500000 +2021-03-26 07:18:12,494 epoch 10 - iter 25/50 - loss 8.40272551 - samples/sec: 80.95 - lr: 0.500000 +2021-03-26 07:18:14,446 epoch 10 - iter 30/50 - loss 8.41102055 - samples/sec: 82.06 - lr: 0.500000 +2021-03-26 07:18:16,363 epoch 10 - iter 35/50 - loss 8.35400145 - samples/sec: 83.54 - lr: 0.500000 +2021-03-26 07:18:18,414 epoch 10 - iter 40/50 - loss 8.27732370 - samples/sec: 78.06 - lr: 0.500000 +2021-03-26 07:18:20,306 epoch 10 - iter 45/50 - loss 8.28684916 - samples/sec: 84.63 - lr: 0.500000 +2021-03-26 07:18:22,175 epoch 10 - iter 50/50 - loss 8.46044763 - samples/sec: 85.70 - lr: 0.500000 +2021-03-26 07:18:22,175 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:18:22,176 EPOCH 10 done: loss 8.4604 - lr 0.5000000 +2021-03-26 07:18:22,990 DEV : loss 7.925993919372559 - score 0.8629 +2021-03-26 07:18:23,023 BAD EPOCHS (no improvement): 0 +2021-03-26 07:18:32,404 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:18:34,243 epoch 11 - iter 5/50 - loss 6.82128010 - samples/sec: 87.14 - lr: 0.500000 +2021-03-26 07:18:36,094 epoch 11 - iter 10/50 - loss 6.75115950 - samples/sec: 86.53 - lr: 0.500000 +2021-03-26 07:18:37,971 epoch 11 - iter 15/50 - loss 7.84271013 - samples/sec: 85.33 - lr: 0.500000 +2021-03-26 07:18:39,877 epoch 11 - iter 20/50 - loss 8.39340039 - samples/sec: 84.03 - lr: 0.500000 +2021-03-26 07:18:41,753 epoch 11 - iter 25/50 - loss 8.26747842 - samples/sec: 85.35 - lr: 0.500000 +2021-03-26 07:18:43,611 epoch 11 - iter 30/50 - loss 8.06985706 - samples/sec: 86.23 - lr: 0.500000 +2021-03-26 07:18:45,418 epoch 11 - iter 35/50 - loss 8.02684945 - samples/sec: 88.63 - lr: 0.500000 +2021-03-26 07:18:47,267 epoch 11 - iter 40/50 - loss 7.87398350 - samples/sec: 86.60 - lr: 0.500000 +2021-03-26 07:18:49,091 epoch 11 - iter 45/50 - loss 7.81394832 - samples/sec: 87.83 - lr: 0.500000 +2021-03-26 07:18:50,969 epoch 11 - iter 50/50 - loss 7.85806360 - samples/sec: 85.24 - lr: 0.500000 +2021-03-26 07:18:50,970 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:18:50,970 EPOCH 11 done: loss 7.8581 - lr 0.5000000 +2021-03-26 07:18:51,702 DEV : loss 8.213061332702637 - score 0.8627 +2021-03-26 07:18:51,725 BAD EPOCHS (no improvement): 1 +2021-03-26 07:18:51,726 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:18:53,648 epoch 12 - iter 5/50 - loss 8.13135443 - samples/sec: 83.32 - lr: 0.500000 +2021-03-26 07:18:55,463 epoch 12 - iter 10/50 - loss 7.90176258 - samples/sec: 88.28 - lr: 0.500000 +2021-03-26 07:18:57,262 epoch 12 - iter 15/50 - loss 7.70501353 - samples/sec: 89.01 - lr: 0.500000 +2021-03-26 07:18:59,057 epoch 12 - iter 20/50 - loss 7.43266449 - samples/sec: 89.23 - lr: 0.500000 +2021-03-26 07:19:00,976 epoch 12 - iter 25/50 - loss 7.29033449 - samples/sec: 83.45 - lr: 0.500000 +2021-03-26 07:19:02,949 epoch 12 - iter 30/50 - loss 7.39221203 - samples/sec: 81.14 - lr: 0.500000 +2021-03-26 07:19:04,752 epoch 12 - iter 35/50 - loss 7.36656352 - samples/sec: 88.83 - lr: 0.500000 +2021-03-26 07:19:06,694 epoch 12 - iter 40/50 - loss 7.25957453 - samples/sec: 82.48 - lr: 0.500000 +2021-03-26 07:19:08,666 epoch 12 - iter 45/50 - loss 7.20568540 - samples/sec: 81.22 - lr: 0.500000 +2021-03-26 07:19:10,519 epoch 12 - iter 50/50 - loss 7.38290733 - samples/sec: 86.40 - lr: 0.500000 +2021-03-26 07:19:10,520 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:19:10,521 EPOCH 12 done: loss 7.3829 - lr 0.5000000 +2021-03-26 07:19:11,299 DEV : loss 7.321202754974365 - score 0.8794 +2021-03-26 07:19:11,328 BAD EPOCHS (no improvement): 0 +2021-03-26 07:19:20,919 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:19:22,829 epoch 13 - iter 5/50 - loss 6.82536354 - samples/sec: 83.90 - lr: 0.500000 +2021-03-26 07:19:24,718 epoch 13 - iter 10/50 - loss 6.86277728 - samples/sec: 84.80 - lr: 0.500000 +2021-03-26 07:19:26,660 epoch 13 - iter 15/50 - loss 6.89155194 - samples/sec: 82.45 - lr: 0.500000 +2021-03-26 07:19:28,433 epoch 13 - iter 20/50 - loss 6.78971924 - samples/sec: 90.29 - lr: 0.500000 +2021-03-26 07:19:30,261 epoch 13 - iter 25/50 - loss 6.97617368 - samples/sec: 87.62 - lr: 0.500000 +2021-03-26 07:19:32,017 epoch 13 - iter 30/50 - loss 6.93531903 - samples/sec: 91.21 - lr: 0.500000 +2021-03-26 07:19:33,717 epoch 13 - iter 35/50 - loss 6.90266201 - samples/sec: 94.22 - lr: 0.500000 +2021-03-26 07:19:35,568 epoch 13 - iter 40/50 - loss 6.96388810 - samples/sec: 86.53 - lr: 0.500000 +2021-03-26 07:19:37,534 epoch 13 - iter 45/50 - loss 6.98509439 - samples/sec: 81.45 - lr: 0.500000 +2021-03-26 07:19:39,373 epoch 13 - iter 50/50 - loss 7.03323493 - samples/sec: 87.09 - lr: 0.500000 +2021-03-26 07:19:39,374 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:19:39,374 EPOCH 13 done: loss 7.0332 - lr 0.5000000 +2021-03-26 07:19:40,117 DEV : loss 7.746276378631592 - score 0.8679 +2021-03-26 07:19:40,141 BAD EPOCHS (no improvement): 1 +2021-03-26 07:19:40,142 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:19:42,097 epoch 14 - iter 5/50 - loss 8.29018412 - samples/sec: 81.96 - lr: 0.500000 +2021-03-26 07:19:43,923 epoch 14 - iter 10/50 - loss 7.02287879 - samples/sec: 87.72 - lr: 0.500000 +2021-03-26 07:19:45,874 epoch 14 - iter 15/50 - loss 6.64712122 - samples/sec: 82.06 - lr: 0.500000 +2021-03-26 07:19:47,724 epoch 14 - iter 20/50 - loss 6.63326148 - samples/sec: 86.57 - lr: 0.500000 +2021-03-26 07:19:49,762 epoch 14 - iter 25/50 - loss 6.82262151 - samples/sec: 78.58 - lr: 0.500000 +2021-03-26 07:19:51,767 epoch 14 - iter 30/50 - loss 6.73005801 - samples/sec: 79.86 - lr: 0.500000 +2021-03-26 07:19:53,622 epoch 14 - iter 35/50 - loss 6.68655214 - samples/sec: 86.36 - lr: 0.500000 +2021-03-26 07:19:55,413 epoch 14 - iter 40/50 - loss 6.68068996 - samples/sec: 89.45 - lr: 0.500000 +2021-03-26 07:19:57,250 epoch 14 - iter 45/50 - loss 6.69404273 - samples/sec: 87.18 - lr: 0.500000 +2021-03-26 07:19:59,327 epoch 14 - iter 50/50 - loss 6.89570993 - samples/sec: 77.09 - lr: 0.500000 +2021-03-26 07:19:59,328 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:19:59,328 EPOCH 14 done: loss 6.8957 - lr 0.5000000 +2021-03-26 07:20:00,100 DEV : loss 7.810580253601074 - score 0.8754 +2021-03-26 07:20:00,124 BAD EPOCHS (no improvement): 2 +2021-03-26 07:20:00,125 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:20:02,456 epoch 15 - iter 5/50 - loss 5.86052933 - samples/sec: 68.73 - lr: 0.500000 +2021-03-26 07:20:04,317 epoch 15 - iter 10/50 - loss 5.63950758 - samples/sec: 86.02 - lr: 0.500000 +2021-03-26 07:20:06,292 epoch 15 - iter 15/50 - loss 5.85248785 - samples/sec: 81.10 - lr: 0.500000 +2021-03-26 07:20:08,170 epoch 15 - iter 20/50 - loss 6.19100833 - samples/sec: 85.29 - lr: 0.500000 +2021-03-26 07:20:09,969 epoch 15 - iter 25/50 - loss 6.32947805 - samples/sec: 89.01 - lr: 0.500000 +2021-03-26 07:20:11,895 epoch 15 - iter 30/50 - loss 6.30526145 - samples/sec: 83.20 - lr: 0.500000 +2021-03-26 07:20:14,361 epoch 15 - iter 35/50 - loss 6.28907063 - samples/sec: 64.93 - lr: 0.500000 +2021-03-26 07:20:16,378 epoch 15 - iter 40/50 - loss 6.33453875 - samples/sec: 79.40 - lr: 0.500000 +2021-03-26 07:20:18,273 epoch 15 - iter 45/50 - loss 6.43837033 - samples/sec: 84.46 - lr: 0.500000 +2021-03-26 07:20:19,958 epoch 15 - iter 50/50 - loss 6.35599067 - samples/sec: 95.07 - lr: 0.500000 +2021-03-26 07:20:19,959 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:20:19,960 EPOCH 15 done: loss 6.3560 - lr 0.5000000 +2021-03-26 07:20:20,732 DEV : loss 7.69682502746582 - score 0.8702 +2021-03-26 07:20:20,756 BAD EPOCHS (no improvement): 3 +2021-03-26 07:20:20,757 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:20:22,535 epoch 16 - iter 5/50 - loss 5.94938555 - samples/sec: 90.08 - lr: 0.500000 +2021-03-26 07:20:24,532 epoch 16 - iter 10/50 - loss 5.70883789 - samples/sec: 80.19 - lr: 0.500000 +2021-03-26 07:20:26,548 epoch 16 - iter 15/50 - loss 5.92653176 - samples/sec: 79.43 - lr: 0.500000 +2021-03-26 07:20:28,477 epoch 16 - iter 20/50 - loss 6.07018437 - samples/sec: 83.04 - lr: 0.500000 +2021-03-26 07:20:30,356 epoch 16 - iter 25/50 - loss 5.94514559 - samples/sec: 85.19 - lr: 0.500000 +2021-03-26 07:20:32,259 epoch 16 - iter 30/50 - loss 6.15654551 - samples/sec: 84.17 - lr: 0.500000 +2021-03-26 07:20:34,121 epoch 16 - iter 35/50 - loss 6.13194075 - samples/sec: 86.02 - lr: 0.500000 +2021-03-26 07:20:35,925 epoch 16 - iter 40/50 - loss 6.17352308 - samples/sec: 88.76 - lr: 0.500000 +2021-03-26 07:20:37,712 epoch 16 - iter 45/50 - loss 6.20834807 - samples/sec: 89.65 - lr: 0.500000 +2021-03-26 07:20:39,459 epoch 16 - iter 50/50 - loss 6.29687656 - samples/sec: 91.73 - lr: 0.500000 +2021-03-26 07:20:39,460 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:20:39,460 EPOCH 16 done: loss 6.2969 - lr 0.5000000 +2021-03-26 07:20:40,223 DEV : loss 7.245139122009277 - score 0.8862 +2021-03-26 07:20:40,247 BAD EPOCHS (no improvement): 0 +2021-03-26 07:20:49,704 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:20:51,540 epoch 17 - iter 5/50 - loss 5.49544411 - samples/sec: 87.28 - lr: 0.500000 +2021-03-26 07:20:53,435 epoch 17 - iter 10/50 - loss 5.53276272 - samples/sec: 84.53 - lr: 0.500000 +2021-03-26 07:20:55,254 epoch 17 - iter 15/50 - loss 5.87554884 - samples/sec: 88.05 - lr: 0.500000 +2021-03-26 07:20:57,513 epoch 17 - iter 20/50 - loss 5.90366483 - samples/sec: 70.88 - lr: 0.500000 +2021-03-26 07:21:00,040 epoch 17 - iter 25/50 - loss 5.88849455 - samples/sec: 63.38 - lr: 0.500000 +2021-03-26 07:21:01,932 epoch 17 - iter 30/50 - loss 5.99359461 - samples/sec: 84.60 - lr: 0.500000 +2021-03-26 07:21:03,962 epoch 17 - iter 35/50 - loss 6.00984791 - samples/sec: 78.89 - lr: 0.500000 +2021-03-26 07:21:05,843 epoch 17 - iter 40/50 - loss 6.00084951 - samples/sec: 85.14 - lr: 0.500000 +2021-03-26 07:21:07,641 epoch 17 - iter 45/50 - loss 5.98434572 - samples/sec: 89.10 - lr: 0.500000 +2021-03-26 07:21:09,476 epoch 17 - iter 50/50 - loss 5.94449944 - samples/sec: 87.28 - lr: 0.500000 +2021-03-26 07:21:09,477 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:21:09,478 EPOCH 17 done: loss 5.9445 - lr 0.5000000 +2021-03-26 07:21:10,240 DEV : loss 7.041107177734375 - score 0.8888 +2021-03-26 07:21:10,263 BAD EPOCHS (no improvement): 0 +2021-03-26 07:21:19,640 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:21:21,345 epoch 18 - iter 5/50 - loss 5.46243830 - samples/sec: 93.98 - lr: 0.500000 +2021-03-26 07:21:23,270 epoch 18 - iter 10/50 - loss 5.42810898 - samples/sec: 83.23 - lr: 0.500000 +2021-03-26 07:21:25,252 epoch 18 - iter 15/50 - loss 5.50631167 - samples/sec: 80.78 - lr: 0.500000 +2021-03-26 07:21:27,184 epoch 18 - iter 20/50 - loss 5.40645225 - samples/sec: 82.90 - lr: 0.500000 +2021-03-26 07:21:29,157 epoch 18 - iter 25/50 - loss 5.53953705 - samples/sec: 81.15 - lr: 0.500000 +2021-03-26 07:21:31,044 epoch 18 - iter 30/50 - loss 5.67473930 - samples/sec: 84.85 - lr: 0.500000 +2021-03-26 07:21:33,113 epoch 18 - iter 35/50 - loss 5.69194521 - samples/sec: 77.40 - lr: 0.500000 +2021-03-26 07:21:35,063 epoch 18 - iter 40/50 - loss 5.79437474 - samples/sec: 82.13 - lr: 0.500000 +2021-03-26 07:21:36,880 epoch 18 - iter 45/50 - loss 5.81149050 - samples/sec: 88.11 - lr: 0.500000 +2021-03-26 07:21:38,666 epoch 18 - iter 50/50 - loss 5.94897350 - samples/sec: 89.70 - lr: 0.500000 +2021-03-26 07:21:38,666 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:21:38,667 EPOCH 18 done: loss 5.9490 - lr 0.5000000 +2021-03-26 07:21:39,405 DEV : loss 7.531728267669678 - score 0.8795 +2021-03-26 07:21:39,429 BAD EPOCHS (no improvement): 1 +2021-03-26 07:21:39,429 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:21:41,309 epoch 19 - iter 5/50 - loss 6.17610245 - samples/sec: 85.22 - lr: 0.500000 +2021-03-26 07:21:43,200 epoch 19 - iter 10/50 - loss 5.49422398 - samples/sec: 84.68 - lr: 0.500000 +2021-03-26 07:21:45,021 epoch 19 - iter 15/50 - loss 5.62345748 - samples/sec: 87.99 - lr: 0.500000 +2021-03-26 07:21:46,927 epoch 19 - iter 20/50 - loss 5.61114949 - samples/sec: 84.00 - lr: 0.500000 +2021-03-26 07:21:48,793 epoch 19 - iter 25/50 - loss 5.58432162 - samples/sec: 85.85 - lr: 0.500000 +2021-03-26 07:21:50,738 epoch 19 - iter 30/50 - loss 5.60729048 - samples/sec: 82.30 - lr: 0.500000 +2021-03-26 07:21:52,627 epoch 19 - iter 35/50 - loss 5.65039468 - samples/sec: 84.80 - lr: 0.500000 +2021-03-26 07:21:54,402 epoch 19 - iter 40/50 - loss 5.60913602 - samples/sec: 90.27 - lr: 0.500000 +2021-03-26 07:21:56,310 epoch 19 - iter 45/50 - loss 5.58604383 - samples/sec: 83.97 - lr: 0.500000 +2021-03-26 07:21:58,069 epoch 19 - iter 50/50 - loss 5.60817661 - samples/sec: 91.07 - lr: 0.500000 +2021-03-26 07:21:58,070 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:21:58,070 EPOCH 19 done: loss 5.6082 - lr 0.5000000 +2021-03-26 07:21:58,818 DEV : loss 7.617198467254639 - score 0.8725 +2021-03-26 07:21:58,842 BAD EPOCHS (no improvement): 2 +2021-03-26 07:21:58,842 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:22:01,088 epoch 20 - iter 5/50 - loss 4.85406322 - samples/sec: 71.30 - lr: 0.500000 +2021-03-26 07:22:02,889 epoch 20 - iter 10/50 - loss 5.31827521 - samples/sec: 88.93 - lr: 0.500000 +2021-03-26 07:22:04,776 epoch 20 - iter 15/50 - loss 5.31004019 - samples/sec: 84.86 - lr: 0.500000 +2021-03-26 07:22:06,555 epoch 20 - iter 20/50 - loss 5.45654291 - samples/sec: 90.04 - lr: 0.500000 +2021-03-26 07:22:08,464 epoch 20 - iter 25/50 - loss 5.38042480 - samples/sec: 83.86 - lr: 0.500000 +2021-03-26 07:22:10,340 epoch 20 - iter 30/50 - loss 5.55804290 - samples/sec: 85.39 - lr: 0.500000 +2021-03-26 07:22:12,087 epoch 20 - iter 35/50 - loss 5.47529270 - samples/sec: 91.63 - lr: 0.500000 +2021-03-26 07:22:13,934 epoch 20 - iter 40/50 - loss 5.46405497 - samples/sec: 86.72 - lr: 0.500000 +2021-03-26 07:22:16,049 epoch 20 - iter 45/50 - loss 5.52897902 - samples/sec: 75.70 - lr: 0.500000 +2021-03-26 07:22:17,768 epoch 20 - iter 50/50 - loss 5.54975753 - samples/sec: 93.19 - lr: 0.500000 +2021-03-26 07:22:17,769 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:22:17,769 EPOCH 20 done: loss 5.5498 - lr 0.5000000 +2021-03-26 07:22:18,532 DEV : loss 7.108177185058594 - score 0.8819 +2021-03-26 07:22:18,556 BAD EPOCHS (no improvement): 3 +2021-03-26 07:22:18,557 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:22:20,513 epoch 21 - iter 5/50 - loss 5.16098585 - samples/sec: 81.87 - lr: 0.500000 +2021-03-26 07:22:22,461 epoch 21 - iter 10/50 - loss 5.18648055 - samples/sec: 82.22 - lr: 0.500000 +2021-03-26 07:22:24,278 epoch 21 - iter 15/50 - loss 4.95983901 - samples/sec: 88.16 - lr: 0.500000 +2021-03-26 07:22:26,051 epoch 21 - iter 20/50 - loss 4.78192036 - samples/sec: 90.29 - lr: 0.500000 +2021-03-26 07:22:27,861 epoch 21 - iter 25/50 - loss 4.84919130 - samples/sec: 88.48 - lr: 0.500000 +2021-03-26 07:22:29,806 epoch 21 - iter 30/50 - loss 4.90508842 - samples/sec: 82.35 - lr: 0.500000 +2021-03-26 07:22:31,748 epoch 21 - iter 35/50 - loss 4.91921381 - samples/sec: 82.45 - lr: 0.500000 +2021-03-26 07:22:33,575 epoch 21 - iter 40/50 - loss 4.94709951 - samples/sec: 87.67 - lr: 0.500000 +2021-03-26 07:22:35,497 epoch 21 - iter 45/50 - loss 5.04448750 - samples/sec: 83.36 - lr: 0.500000 +2021-03-26 07:22:37,206 epoch 21 - iter 50/50 - loss 5.09773973 - samples/sec: 93.72 - lr: 0.500000 +2021-03-26 07:22:37,207 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:22:37,207 EPOCH 21 done: loss 5.0977 - lr 0.5000000 +2021-03-26 07:22:37,955 DEV : loss 7.056064128875732 - score 0.8867 +2021-03-26 07:22:37,978 BAD EPOCHS (no improvement): 4 +2021-03-26 07:22:37,978 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:22:40,485 epoch 22 - iter 5/50 - loss 4.97422800 - samples/sec: 63.88 - lr: 0.250000 +2021-03-26 07:22:42,424 epoch 22 - iter 10/50 - loss 5.04048371 - samples/sec: 82.56 - lr: 0.250000 +2021-03-26 07:22:44,332 epoch 22 - iter 15/50 - loss 5.08245630 - samples/sec: 83.96 - lr: 0.250000 +2021-03-26 07:22:46,359 epoch 22 - iter 20/50 - loss 4.87549429 - samples/sec: 78.98 - lr: 0.250000 +2021-03-26 07:22:48,277 epoch 22 - iter 25/50 - loss 4.77376683 - samples/sec: 83.53 - lr: 0.250000 +2021-03-26 07:22:49,957 epoch 22 - iter 30/50 - loss 4.53706872 - samples/sec: 95.33 - lr: 0.250000 +2021-03-26 07:22:51,797 epoch 22 - iter 35/50 - loss 4.60300268 - samples/sec: 87.03 - lr: 0.250000 +2021-03-26 07:22:54,177 epoch 22 - iter 40/50 - loss 4.57214971 - samples/sec: 67.27 - lr: 0.250000 +2021-03-26 07:22:55,995 epoch 22 - iter 45/50 - loss 4.56367335 - samples/sec: 88.10 - lr: 0.250000 +2021-03-26 07:22:57,760 epoch 22 - iter 50/50 - loss 4.46984960 - samples/sec: 90.70 - lr: 0.250000 +2021-03-26 07:22:57,761 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:22:57,762 EPOCH 22 done: loss 4.4698 - lr 0.2500000 +2021-03-26 07:22:58,540 DEV : loss 6.589443206787109 - score 0.8909 +2021-03-26 07:22:58,563 BAD EPOCHS (no improvement): 0 +2021-03-26 07:23:07,888 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:23:10,013 epoch 23 - iter 5/50 - loss 4.35152192 - samples/sec: 75.40 - lr: 0.250000 +2021-03-26 07:23:12,196 epoch 23 - iter 10/50 - loss 4.12232957 - samples/sec: 73.35 - lr: 0.250000 +2021-03-26 07:23:14,091 epoch 23 - iter 15/50 - loss 4.29600732 - samples/sec: 84.52 - lr: 0.250000 +2021-03-26 07:23:16,476 epoch 23 - iter 20/50 - loss 4.32097278 - samples/sec: 67.15 - lr: 0.250000 +2021-03-26 07:23:19,105 epoch 23 - iter 25/50 - loss 4.47653557 - samples/sec: 60.90 - lr: 0.250000 +2021-03-26 07:23:21,677 epoch 23 - iter 30/50 - loss 4.30746055 - samples/sec: 62.25 - lr: 0.250000 +2021-03-26 07:23:23,532 epoch 23 - iter 35/50 - loss 4.25856673 - samples/sec: 86.33 - lr: 0.250000 +2021-03-26 07:23:25,373 epoch 23 - iter 40/50 - loss 4.24913441 - samples/sec: 87.04 - lr: 0.250000 +2021-03-26 07:23:27,457 epoch 23 - iter 45/50 - loss 4.21190177 - samples/sec: 76.81 - lr: 0.250000 +2021-03-26 07:23:29,194 epoch 23 - iter 50/50 - loss 4.23462809 - samples/sec: 92.24 - lr: 0.250000 +2021-03-26 07:23:29,195 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:23:29,195 EPOCH 23 done: loss 4.2346 - lr 0.2500000 +2021-03-26 07:23:29,932 DEV : loss 6.665990352630615 - score 0.8944 +2021-03-26 07:23:29,955 BAD EPOCHS (no improvement): 0 +2021-03-26 07:23:39,427 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:23:42,038 epoch 24 - iter 5/50 - loss 3.49912410 - samples/sec: 61.35 - lr: 0.250000 +2021-03-26 07:23:44,305 epoch 24 - iter 10/50 - loss 3.82160933 - samples/sec: 70.64 - lr: 0.250000 +2021-03-26 07:23:46,197 epoch 24 - iter 15/50 - loss 4.00115436 - samples/sec: 84.65 - lr: 0.250000 +2021-03-26 07:23:48,026 epoch 24 - iter 20/50 - loss 3.97462704 - samples/sec: 87.56 - lr: 0.250000 +2021-03-26 07:23:49,968 epoch 24 - iter 25/50 - loss 4.03846642 - samples/sec: 82.48 - lr: 0.250000 +2021-03-26 07:23:51,808 epoch 24 - iter 30/50 - loss 4.01450043 - samples/sec: 87.06 - lr: 0.250000 +2021-03-26 07:23:53,705 epoch 24 - iter 35/50 - loss 4.03496842 - samples/sec: 84.44 - lr: 0.250000 +2021-03-26 07:23:55,830 epoch 24 - iter 40/50 - loss 4.01507310 - samples/sec: 75.35 - lr: 0.250000 +2021-03-26 07:23:57,902 epoch 24 - iter 45/50 - loss 4.05896441 - samples/sec: 77.29 - lr: 0.250000 +2021-03-26 07:23:59,648 epoch 24 - iter 50/50 - loss 4.00071608 - samples/sec: 91.72 - lr: 0.250000 +2021-03-26 07:23:59,649 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:23:59,649 EPOCH 24 done: loss 4.0007 - lr 0.2500000 +2021-03-26 07:24:00,406 DEV : loss 6.711074352264404 - score 0.8923 +2021-03-26 07:24:00,429 BAD EPOCHS (no improvement): 1 +2021-03-26 07:24:00,430 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:24:02,724 epoch 25 - iter 5/50 - loss 4.26815948 - samples/sec: 69.81 - lr: 0.250000 +2021-03-26 07:24:04,567 epoch 25 - iter 10/50 - loss 4.09609385 - samples/sec: 86.90 - lr: 0.250000 +2021-03-26 07:24:06,381 epoch 25 - iter 15/50 - loss 3.88939308 - samples/sec: 88.31 - lr: 0.250000 +2021-03-26 07:24:08,402 epoch 25 - iter 20/50 - loss 3.84197943 - samples/sec: 79.21 - lr: 0.250000 +2021-03-26 07:24:10,315 epoch 25 - iter 25/50 - loss 3.85339655 - samples/sec: 83.79 - lr: 0.250000 +2021-03-26 07:24:12,193 epoch 25 - iter 30/50 - loss 4.03538799 - samples/sec: 85.27 - lr: 0.250000 +2021-03-26 07:24:13,909 epoch 25 - iter 35/50 - loss 4.00416953 - samples/sec: 93.33 - lr: 0.250000 +2021-03-26 07:24:15,730 epoch 25 - iter 40/50 - loss 3.97750949 - samples/sec: 87.96 - lr: 0.250000 +2021-03-26 07:24:17,652 epoch 25 - iter 45/50 - loss 4.02689092 - samples/sec: 83.33 - lr: 0.250000 +2021-03-26 07:24:19,436 epoch 25 - iter 50/50 - loss 4.01936452 - samples/sec: 89.76 - lr: 0.250000 +2021-03-26 07:24:19,437 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:24:19,437 EPOCH 25 done: loss 4.0194 - lr 0.2500000 +2021-03-26 07:24:20,202 DEV : loss 6.78806209564209 - score 0.8905 +2021-03-26 07:24:20,226 BAD EPOCHS (no improvement): 2 +2021-03-26 07:24:20,227 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:24:22,040 epoch 26 - iter 5/50 - loss 3.97351761 - samples/sec: 88.37 - lr: 0.250000 +2021-03-26 07:24:24,191 epoch 26 - iter 10/50 - loss 3.95412531 - samples/sec: 74.41 - lr: 0.250000 +2021-03-26 07:24:26,350 epoch 26 - iter 15/50 - loss 3.84625311 - samples/sec: 74.16 - lr: 0.250000 +2021-03-26 07:24:28,195 epoch 26 - iter 20/50 - loss 3.71876276 - samples/sec: 86.83 - lr: 0.250000 +2021-03-26 07:24:29,948 epoch 26 - iter 25/50 - loss 3.80736031 - samples/sec: 91.35 - lr: 0.250000 +2021-03-26 07:24:31,884 epoch 26 - iter 30/50 - loss 3.73687125 - samples/sec: 82.70 - lr: 0.250000 +2021-03-26 07:24:33,803 epoch 26 - iter 35/50 - loss 3.74322898 - samples/sec: 83.44 - lr: 0.250000 +2021-03-26 07:24:35,661 epoch 26 - iter 40/50 - loss 3.73008016 - samples/sec: 86.21 - lr: 0.250000 +2021-03-26 07:24:37,550 epoch 26 - iter 45/50 - loss 3.72723302 - samples/sec: 84.78 - lr: 0.250000 +2021-03-26 07:24:39,196 epoch 26 - iter 50/50 - loss 3.73614890 - samples/sec: 97.34 - lr: 0.250000 +2021-03-26 07:24:39,197 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:24:39,197 EPOCH 26 done: loss 3.7361 - lr 0.2500000 +2021-03-26 07:24:39,926 DEV : loss 6.780250549316406 - score 0.8948 +2021-03-26 07:24:39,949 BAD EPOCHS (no improvement): 0 +2021-03-26 07:24:49,179 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:24:51,143 epoch 27 - iter 5/50 - loss 3.09613357 - samples/sec: 81.57 - lr: 0.250000 +2021-03-26 07:24:53,307 epoch 27 - iter 10/50 - loss 3.32715802 - samples/sec: 74.02 - lr: 0.250000 +2021-03-26 07:24:55,195 epoch 27 - iter 15/50 - loss 3.44630082 - samples/sec: 84.85 - lr: 0.250000 +2021-03-26 07:24:57,059 epoch 27 - iter 20/50 - loss 3.56317025 - samples/sec: 85.90 - lr: 0.250000 +2021-03-26 07:24:59,006 epoch 27 - iter 25/50 - loss 3.52153577 - samples/sec: 82.26 - lr: 0.250000 +2021-03-26 07:25:00,851 epoch 27 - iter 30/50 - loss 3.51912872 - samples/sec: 86.79 - lr: 0.250000 +2021-03-26 07:25:02,735 epoch 27 - iter 35/50 - loss 3.56115228 - samples/sec: 85.03 - lr: 0.250000 +2021-03-26 07:25:04,512 epoch 27 - iter 40/50 - loss 3.63291188 - samples/sec: 90.13 - lr: 0.250000 +2021-03-26 07:25:06,374 epoch 27 - iter 45/50 - loss 3.60862633 - samples/sec: 86.03 - lr: 0.250000 +2021-03-26 07:25:08,332 epoch 27 - iter 50/50 - loss 3.70079799 - samples/sec: 81.77 - lr: 0.250000 +2021-03-26 07:25:08,333 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:25:08,334 EPOCH 27 done: loss 3.7008 - lr 0.2500000 +2021-03-26 07:25:09,093 DEV : loss 6.70261287689209 - score 0.8979 +2021-03-26 07:25:09,116 BAD EPOCHS (no improvement): 0 +2021-03-26 07:25:18,299 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:25:20,119 epoch 28 - iter 5/50 - loss 3.45658116 - samples/sec: 88.05 - lr: 0.250000 +2021-03-26 07:25:21,814 epoch 28 - iter 10/50 - loss 3.54790840 - samples/sec: 94.48 - lr: 0.250000 +2021-03-26 07:25:23,590 epoch 28 - iter 15/50 - loss 3.70695179 - samples/sec: 90.16 - lr: 0.250000 +2021-03-26 07:25:25,547 epoch 28 - iter 20/50 - loss 3.64068614 - samples/sec: 81.83 - lr: 0.250000 +2021-03-26 07:25:27,362 epoch 28 - iter 25/50 - loss 3.64049382 - samples/sec: 88.26 - lr: 0.250000 +2021-03-26 07:25:29,297 epoch 28 - iter 30/50 - loss 3.62528403 - samples/sec: 82.75 - lr: 0.250000 +2021-03-26 07:25:31,147 epoch 28 - iter 35/50 - loss 3.69814000 - samples/sec: 86.61 - lr: 0.250000 +2021-03-26 07:25:32,902 epoch 28 - iter 40/50 - loss 3.70481655 - samples/sec: 91.25 - lr: 0.250000 +2021-03-26 07:25:34,654 epoch 28 - iter 45/50 - loss 3.72054102 - samples/sec: 91.42 - lr: 0.250000 +2021-03-26 07:25:36,296 epoch 28 - iter 50/50 - loss 3.62931919 - samples/sec: 97.51 - lr: 0.250000 +2021-03-26 07:25:36,297 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:25:36,297 EPOCH 28 done: loss 3.6293 - lr 0.2500000 +2021-03-26 07:25:37,035 DEV : loss 6.870364189147949 - score 0.8938 +2021-03-26 07:25:37,058 BAD EPOCHS (no improvement): 1 +2021-03-26 07:25:37,059 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:25:38,852 epoch 29 - iter 5/50 - loss 3.84682484 - samples/sec: 89.36 - lr: 0.250000 +2021-03-26 07:25:40,763 epoch 29 - iter 10/50 - loss 3.43832564 - samples/sec: 83.80 - lr: 0.250000 +2021-03-26 07:25:42,686 epoch 29 - iter 15/50 - loss 3.30578225 - samples/sec: 83.29 - lr: 0.250000 +2021-03-26 07:25:44,467 epoch 29 - iter 20/50 - loss 3.17273824 - samples/sec: 89.91 - lr: 0.250000 +2021-03-26 07:25:46,512 epoch 29 - iter 25/50 - loss 3.26868323 - samples/sec: 78.30 - lr: 0.250000 +2021-03-26 07:25:48,416 epoch 29 - iter 30/50 - loss 3.36351695 - samples/sec: 84.08 - lr: 0.250000 +2021-03-26 07:25:50,494 epoch 29 - iter 35/50 - loss 3.44216044 - samples/sec: 77.09 - lr: 0.250000 +2021-03-26 07:25:52,408 epoch 29 - iter 40/50 - loss 3.45896108 - samples/sec: 83.64 - lr: 0.250000 +2021-03-26 07:25:54,167 epoch 29 - iter 45/50 - loss 3.45697110 - samples/sec: 91.05 - lr: 0.250000 +2021-03-26 07:25:56,013 epoch 29 - iter 50/50 - loss 3.46739976 - samples/sec: 86.79 - lr: 0.250000 +2021-03-26 07:25:56,014 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:25:56,014 EPOCH 29 done: loss 3.4674 - lr 0.2500000 +2021-03-26 07:25:56,785 DEV : loss 6.876599311828613 - score 0.8942 +2021-03-26 07:25:56,808 BAD EPOCHS (no improvement): 2 +2021-03-26 07:25:56,813 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:25:58,709 epoch 30 - iter 5/50 - loss 3.38032241 - samples/sec: 84.46 - lr: 0.250000 +2021-03-26 07:26:00,609 epoch 30 - iter 10/50 - loss 3.35932336 - samples/sec: 84.33 - lr: 0.250000 +2021-03-26 07:26:02,705 epoch 30 - iter 15/50 - loss 3.36644726 - samples/sec: 76.38 - lr: 0.250000 +2021-03-26 07:26:04,522 epoch 30 - iter 20/50 - loss 3.44605324 - samples/sec: 88.18 - lr: 0.250000 +2021-03-26 07:26:06,409 epoch 30 - iter 25/50 - loss 3.31322743 - samples/sec: 84.86 - lr: 0.250000 +2021-03-26 07:26:08,375 epoch 30 - iter 30/50 - loss 3.30118415 - samples/sec: 81.46 - lr: 0.250000 +2021-03-26 07:26:10,268 epoch 30 - iter 35/50 - loss 3.35088294 - samples/sec: 84.60 - lr: 0.250000 +2021-03-26 07:26:12,346 epoch 30 - iter 40/50 - loss 3.35634041 - samples/sec: 77.06 - lr: 0.250000 +2021-03-26 07:26:14,373 epoch 30 - iter 45/50 - loss 3.32806790 - samples/sec: 79.05 - lr: 0.250000 +2021-03-26 07:26:16,372 epoch 30 - iter 50/50 - loss 3.35867829 - samples/sec: 80.10 - lr: 0.250000 +2021-03-26 07:26:16,373 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:26:16,374 EPOCH 30 done: loss 3.3587 - lr 0.2500000 +2021-03-26 07:26:17,119 DEV : loss 6.797982692718506 - score 0.8985 +2021-03-26 07:26:17,142 BAD EPOCHS (no improvement): 0 +2021-03-26 07:26:26,559 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:26:28,668 epoch 31 - iter 5/50 - loss 3.23683105 - samples/sec: 75.97 - lr: 0.250000 +2021-03-26 07:26:30,578 epoch 31 - iter 10/50 - loss 3.41368830 - samples/sec: 83.83 - lr: 0.250000 +2021-03-26 07:26:32,372 epoch 31 - iter 15/50 - loss 3.27620610 - samples/sec: 89.30 - lr: 0.250000 +2021-03-26 07:26:34,304 epoch 31 - iter 20/50 - loss 3.27124422 - samples/sec: 82.89 - lr: 0.250000 +2021-03-26 07:26:36,180 epoch 31 - iter 25/50 - loss 3.34331117 - samples/sec: 85.38 - lr: 0.250000 +2021-03-26 07:26:38,171 epoch 31 - iter 30/50 - loss 3.37468915 - samples/sec: 80.41 - lr: 0.250000 +2021-03-26 07:26:40,132 epoch 31 - iter 35/50 - loss 3.39782465 - samples/sec: 81.66 - lr: 0.250000 +2021-03-26 07:26:41,928 epoch 31 - iter 40/50 - loss 3.44018836 - samples/sec: 89.17 - lr: 0.250000 +2021-03-26 07:26:43,705 epoch 31 - iter 45/50 - loss 3.43161379 - samples/sec: 90.12 - lr: 0.250000 +2021-03-26 07:26:45,676 epoch 31 - iter 50/50 - loss 3.38064914 - samples/sec: 81.24 - lr: 0.250000 +2021-03-26 07:26:45,677 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:26:45,677 EPOCH 31 done: loss 3.3806 - lr 0.2500000 +2021-03-26 07:26:46,427 DEV : loss 6.567054748535156 - score 0.8995 +2021-03-26 07:26:46,451 BAD EPOCHS (no improvement): 0 +2021-03-26 07:26:55,784 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:26:57,725 epoch 32 - iter 5/50 - loss 3.01723185 - samples/sec: 82.52 - lr: 0.250000 +2021-03-26 07:26:59,668 epoch 32 - iter 10/50 - loss 2.86918635 - samples/sec: 82.41 - lr: 0.250000 +2021-03-26 07:27:01,524 epoch 32 - iter 15/50 - loss 2.91769945 - samples/sec: 86.28 - lr: 0.250000 +2021-03-26 07:27:03,487 epoch 32 - iter 20/50 - loss 3.17702067 - samples/sec: 81.61 - lr: 0.250000 +2021-03-26 07:27:05,403 epoch 32 - iter 25/50 - loss 3.25882262 - samples/sec: 83.56 - lr: 0.250000 +2021-03-26 07:27:07,277 epoch 32 - iter 30/50 - loss 3.21344744 - samples/sec: 85.48 - lr: 0.250000 +2021-03-26 07:27:09,152 epoch 32 - iter 35/50 - loss 3.31935992 - samples/sec: 85.39 - lr: 0.250000 +2021-03-26 07:27:10,972 epoch 32 - iter 40/50 - loss 3.29181318 - samples/sec: 87.99 - lr: 0.250000 +2021-03-26 07:27:12,972 epoch 32 - iter 45/50 - loss 3.27006182 - samples/sec: 80.11 - lr: 0.250000 +2021-03-26 07:27:14,708 epoch 32 - iter 50/50 - loss 3.22836397 - samples/sec: 92.22 - lr: 0.250000 +2021-03-26 07:27:14,709 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:27:14,709 EPOCH 32 done: loss 3.2284 - lr 0.2500000 +2021-03-26 07:27:15,478 DEV : loss 6.905445098876953 - score 0.8964 +2021-03-26 07:27:15,495 BAD EPOCHS (no improvement): 1 +2021-03-26 07:27:15,496 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:27:17,361 epoch 33 - iter 5/50 - loss 2.92829876 - samples/sec: 85.90 - lr: 0.250000 +2021-03-26 07:27:19,260 epoch 33 - iter 10/50 - loss 3.38583150 - samples/sec: 84.31 - lr: 0.250000 +2021-03-26 07:27:20,997 epoch 33 - iter 15/50 - loss 3.30913531 - samples/sec: 92.19 - lr: 0.250000 +2021-03-26 07:27:22,878 epoch 33 - iter 20/50 - loss 3.23969779 - samples/sec: 85.15 - lr: 0.250000 +2021-03-26 07:27:24,908 epoch 33 - iter 25/50 - loss 3.33625289 - samples/sec: 78.86 - lr: 0.250000 +2021-03-26 07:27:26,825 epoch 33 - iter 30/50 - loss 3.32055628 - samples/sec: 83.54 - lr: 0.250000 +2021-03-26 07:27:28,752 epoch 33 - iter 35/50 - loss 3.35262194 - samples/sec: 83.14 - lr: 0.250000 +2021-03-26 07:27:30,695 epoch 33 - iter 40/50 - loss 3.32879865 - samples/sec: 82.45 - lr: 0.250000 +2021-03-26 07:27:32,552 epoch 33 - iter 45/50 - loss 3.29292493 - samples/sec: 86.22 - lr: 0.250000 +2021-03-26 07:27:34,441 epoch 33 - iter 50/50 - loss 3.29516871 - samples/sec: 84.80 - lr: 0.250000 +2021-03-26 07:27:34,442 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:27:34,442 EPOCH 33 done: loss 3.2952 - lr 0.2500000 +2021-03-26 07:27:35,209 DEV : loss 7.081844329833984 - score 0.8922 +2021-03-26 07:27:35,232 BAD EPOCHS (no improvement): 2 +2021-03-26 07:27:35,233 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:27:37,298 epoch 34 - iter 5/50 - loss 3.31536870 - samples/sec: 77.57 - lr: 0.250000 +2021-03-26 07:27:39,187 epoch 34 - iter 10/50 - loss 3.41087134 - samples/sec: 84.78 - lr: 0.250000 +2021-03-26 07:27:40,988 epoch 34 - iter 15/50 - loss 3.22168013 - samples/sec: 88.92 - lr: 0.250000 +2021-03-26 07:27:43,080 epoch 34 - iter 20/50 - loss 3.27023110 - samples/sec: 76.54 - lr: 0.250000 +2021-03-26 07:27:44,884 epoch 34 - iter 25/50 - loss 3.25223615 - samples/sec: 88.78 - lr: 0.250000 +2021-03-26 07:27:46,779 epoch 34 - iter 30/50 - loss 3.19871631 - samples/sec: 84.50 - lr: 0.250000 +2021-03-26 07:27:48,725 epoch 34 - iter 35/50 - loss 3.21700798 - samples/sec: 82.27 - lr: 0.250000 +2021-03-26 07:27:50,559 epoch 34 - iter 40/50 - loss 3.25193202 - samples/sec: 87.36 - lr: 0.250000 +2021-03-26 07:27:52,276 epoch 34 - iter 45/50 - loss 3.17302873 - samples/sec: 93.24 - lr: 0.250000 +2021-03-26 07:27:54,008 epoch 34 - iter 50/50 - loss 3.21401441 - samples/sec: 92.47 - lr: 0.250000 +2021-03-26 07:27:54,009 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:27:54,009 EPOCH 34 done: loss 3.2140 - lr 0.2500000 +2021-03-26 07:27:54,749 DEV : loss 6.83712911605835 - score 0.8979 +2021-03-26 07:27:54,773 BAD EPOCHS (no improvement): 3 +2021-03-26 07:27:54,773 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:27:56,701 epoch 35 - iter 5/50 - loss 3.56807122 - samples/sec: 83.11 - lr: 0.250000 +2021-03-26 07:27:58,584 epoch 35 - iter 10/50 - loss 3.23114133 - samples/sec: 85.02 - lr: 0.250000 +2021-03-26 07:28:00,492 epoch 35 - iter 15/50 - loss 3.04977350 - samples/sec: 83.96 - lr: 0.250000 +2021-03-26 07:28:02,232 epoch 35 - iter 20/50 - loss 3.10344130 - samples/sec: 92.01 - lr: 0.250000 +2021-03-26 07:28:04,194 epoch 35 - iter 25/50 - loss 3.07100470 - samples/sec: 81.63 - lr: 0.250000 +2021-03-26 07:28:06,060 epoch 35 - iter 30/50 - loss 3.07075768 - samples/sec: 85.82 - lr: 0.250000 +2021-03-26 07:28:07,840 epoch 35 - iter 35/50 - loss 3.09712845 - samples/sec: 89.97 - lr: 0.250000 +2021-03-26 07:28:09,991 epoch 35 - iter 40/50 - loss 3.11089132 - samples/sec: 74.44 - lr: 0.250000 +2021-03-26 07:28:11,928 epoch 35 - iter 45/50 - loss 3.07642374 - samples/sec: 82.69 - lr: 0.250000 +2021-03-26 07:28:13,727 epoch 35 - iter 50/50 - loss 3.12534660 - samples/sec: 89.01 - lr: 0.250000 +2021-03-26 07:28:13,728 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:28:13,728 EPOCH 35 done: loss 3.1253 - lr 0.2500000 +2021-03-26 07:28:14,471 DEV : loss 7.0124664306640625 - score 0.9001 +2021-03-26 07:28:14,487 BAD EPOCHS (no improvement): 0 +2021-03-26 07:28:23,704 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:28:25,601 epoch 36 - iter 5/50 - loss 3.38841424 - samples/sec: 84.46 - lr: 0.250000 +2021-03-26 07:28:27,763 epoch 36 - iter 10/50 - loss 3.24558442 - samples/sec: 74.06 - lr: 0.250000 +2021-03-26 07:28:29,859 epoch 36 - iter 15/50 - loss 3.03712088 - samples/sec: 76.40 - lr: 0.250000 +2021-03-26 07:28:31,779 epoch 36 - iter 20/50 - loss 2.96634012 - samples/sec: 83.44 - lr: 0.250000 +2021-03-26 07:28:33,656 epoch 36 - iter 25/50 - loss 3.01913111 - samples/sec: 85.29 - lr: 0.250000 +2021-03-26 07:28:35,623 epoch 36 - iter 30/50 - loss 3.01572754 - samples/sec: 81.41 - lr: 0.250000 +2021-03-26 07:28:37,505 epoch 36 - iter 35/50 - loss 2.99192525 - samples/sec: 85.09 - lr: 0.250000 +2021-03-26 07:28:39,644 epoch 36 - iter 40/50 - loss 2.98679020 - samples/sec: 74.87 - lr: 0.250000 +2021-03-26 07:28:41,608 epoch 36 - iter 45/50 - loss 3.01203941 - samples/sec: 81.54 - lr: 0.250000 +2021-03-26 07:28:43,396 epoch 36 - iter 50/50 - loss 3.01612416 - samples/sec: 89.57 - lr: 0.250000 +2021-03-26 07:28:43,397 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:28:43,397 EPOCH 36 done: loss 3.0161 - lr 0.2500000 +2021-03-26 07:28:44,140 DEV : loss 6.748878479003906 - score 0.9009 +2021-03-26 07:28:44,163 BAD EPOCHS (no improvement): 0 +2021-03-26 07:28:53,279 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:28:55,230 epoch 37 - iter 5/50 - loss 3.26688495 - samples/sec: 82.06 - lr: 0.250000 +2021-03-26 07:28:57,277 epoch 37 - iter 10/50 - loss 3.48513596 - samples/sec: 78.25 - lr: 0.250000 +2021-03-26 07:28:59,254 epoch 37 - iter 15/50 - loss 3.37037697 - samples/sec: 80.98 - lr: 0.250000 +2021-03-26 07:29:01,033 epoch 37 - iter 20/50 - loss 3.33579125 - samples/sec: 90.00 - lr: 0.250000 +2021-03-26 07:29:03,090 epoch 37 - iter 25/50 - loss 3.29973496 - samples/sec: 77.85 - lr: 0.250000 +2021-03-26 07:29:05,378 epoch 37 - iter 30/50 - loss 3.26574783 - samples/sec: 69.98 - lr: 0.250000 +2021-03-26 07:29:07,272 epoch 37 - iter 35/50 - loss 3.18129197 - samples/sec: 84.55 - lr: 0.250000 +2021-03-26 07:29:09,191 epoch 37 - iter 40/50 - loss 3.16513637 - samples/sec: 83.47 - lr: 0.250000 +2021-03-26 07:29:11,029 epoch 37 - iter 45/50 - loss 3.19896053 - samples/sec: 87.12 - lr: 0.250000 +2021-03-26 07:29:12,624 epoch 37 - iter 50/50 - loss 3.09600536 - samples/sec: 100.38 - lr: 0.250000 +2021-03-26 07:29:12,625 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:29:12,625 EPOCH 37 done: loss 3.0960 - lr 0.2500000 +2021-03-26 07:29:13,380 DEV : loss 7.075430870056152 - score 0.894 +2021-03-26 07:29:13,399 BAD EPOCHS (no improvement): 1 +2021-03-26 07:29:13,399 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:29:15,379 epoch 38 - iter 5/50 - loss 2.92954760 - samples/sec: 80.92 - lr: 0.250000 +2021-03-26 07:29:17,270 epoch 38 - iter 10/50 - loss 3.19841943 - samples/sec: 84.68 - lr: 0.250000 +2021-03-26 07:29:19,069 epoch 38 - iter 15/50 - loss 3.19953322 - samples/sec: 89.00 - lr: 0.250000 +2021-03-26 07:29:21,011 epoch 38 - iter 20/50 - loss 3.19118745 - samples/sec: 82.46 - lr: 0.250000 +2021-03-26 07:29:23,093 epoch 38 - iter 25/50 - loss 3.15121227 - samples/sec: 76.91 - lr: 0.250000 +2021-03-26 07:29:24,945 epoch 38 - iter 30/50 - loss 3.11521289 - samples/sec: 86.47 - lr: 0.250000 +2021-03-26 07:29:26,874 epoch 38 - iter 35/50 - loss 3.17518789 - samples/sec: 83.04 - lr: 0.250000 +2021-03-26 07:29:28,737 epoch 38 - iter 40/50 - loss 3.10305169 - samples/sec: 85.96 - lr: 0.250000 +2021-03-26 07:29:30,716 epoch 38 - iter 45/50 - loss 3.10097717 - samples/sec: 80.92 - lr: 0.250000 +2021-03-26 07:29:32,369 epoch 38 - iter 50/50 - loss 3.10808053 - samples/sec: 96.92 - lr: 0.250000 +2021-03-26 07:29:32,369 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:29:32,370 EPOCH 38 done: loss 3.1081 - lr 0.2500000 +2021-03-26 07:29:33,114 DEV : loss 7.046388149261475 - score 0.8964 +2021-03-26 07:29:33,137 BAD EPOCHS (no improvement): 2 +2021-03-26 07:29:33,138 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:29:34,929 epoch 39 - iter 5/50 - loss 2.50283210 - samples/sec: 89.45 - lr: 0.250000 +2021-03-26 07:29:36,718 epoch 39 - iter 10/50 - loss 2.79771951 - samples/sec: 89.50 - lr: 0.250000 +2021-03-26 07:29:38,533 epoch 39 - iter 15/50 - loss 2.75515049 - samples/sec: 88.24 - lr: 0.250000 +2021-03-26 07:29:40,377 epoch 39 - iter 20/50 - loss 2.72164462 - samples/sec: 86.86 - lr: 0.250000 +2021-03-26 07:29:42,317 epoch 39 - iter 25/50 - loss 2.82088650 - samples/sec: 82.52 - lr: 0.250000 +2021-03-26 07:29:44,130 epoch 39 - iter 30/50 - loss 2.86356262 - samples/sec: 88.37 - lr: 0.250000 +2021-03-26 07:29:46,074 epoch 39 - iter 35/50 - loss 2.78411827 - samples/sec: 82.34 - lr: 0.250000 +2021-03-26 07:29:48,047 epoch 39 - iter 40/50 - loss 2.78368353 - samples/sec: 81.18 - lr: 0.250000 +2021-03-26 07:29:50,249 epoch 39 - iter 45/50 - loss 2.78644994 - samples/sec: 72.71 - lr: 0.250000 +2021-03-26 07:29:52,012 epoch 39 - iter 50/50 - loss 2.91670645 - samples/sec: 90.85 - lr: 0.250000 +2021-03-26 07:29:52,013 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:29:52,014 EPOCH 39 done: loss 2.9167 - lr 0.2500000 +2021-03-26 07:29:52,784 DEV : loss 6.61793851852417 - score 0.8997 +2021-03-26 07:29:52,806 BAD EPOCHS (no improvement): 3 +2021-03-26 07:29:52,807 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:29:54,705 epoch 40 - iter 5/50 - loss 2.90698042 - samples/sec: 84.39 - lr: 0.250000 +2021-03-26 07:29:56,712 epoch 40 - iter 10/50 - loss 3.07795951 - samples/sec: 79.80 - lr: 0.250000 +2021-03-26 07:29:58,621 epoch 40 - iter 15/50 - loss 3.00809290 - samples/sec: 83.88 - lr: 0.250000 +2021-03-26 07:30:00,450 epoch 40 - iter 20/50 - loss 2.98251244 - samples/sec: 87.58 - lr: 0.250000 +2021-03-26 07:30:02,370 epoch 40 - iter 25/50 - loss 3.01671128 - samples/sec: 83.44 - lr: 0.250000 +2021-03-26 07:30:04,182 epoch 40 - iter 30/50 - loss 2.96408661 - samples/sec: 88.39 - lr: 0.250000 +2021-03-26 07:30:06,157 epoch 40 - iter 35/50 - loss 2.98403021 - samples/sec: 81.05 - lr: 0.250000 +2021-03-26 07:30:08,207 epoch 40 - iter 40/50 - loss 2.96908256 - samples/sec: 78.14 - lr: 0.250000 +2021-03-26 07:30:10,113 epoch 40 - iter 45/50 - loss 2.96535072 - samples/sec: 84.02 - lr: 0.250000 +2021-03-26 07:30:11,796 epoch 40 - iter 50/50 - loss 2.92278719 - samples/sec: 95.15 - lr: 0.250000 +2021-03-26 07:30:11,797 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:30:11,797 EPOCH 40 done: loss 2.9228 - lr 0.2500000 +2021-03-26 07:30:12,573 DEV : loss 6.909717559814453 - score 0.8993 +2021-03-26 07:30:12,589 BAD EPOCHS (no improvement): 4 +2021-03-26 07:30:12,589 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:30:14,528 epoch 41 - iter 5/50 - loss 2.69951725 - samples/sec: 82.61 - lr: 0.125000 +2021-03-26 07:30:16,345 epoch 41 - iter 10/50 - loss 2.91924675 - samples/sec: 88.13 - lr: 0.125000 +2021-03-26 07:30:18,195 epoch 41 - iter 15/50 - loss 2.98765427 - samples/sec: 86.59 - lr: 0.125000 +2021-03-26 07:30:20,152 epoch 41 - iter 20/50 - loss 2.92043756 - samples/sec: 81.82 - lr: 0.125000 +2021-03-26 07:30:22,045 epoch 41 - iter 25/50 - loss 2.89524738 - samples/sec: 84.59 - lr: 0.125000 +2021-03-26 07:30:23,910 epoch 41 - iter 30/50 - loss 2.76787079 - samples/sec: 85.93 - lr: 0.125000 +2021-03-26 07:30:25,981 epoch 41 - iter 35/50 - loss 2.83275717 - samples/sec: 77.31 - lr: 0.125000 +2021-03-26 07:30:27,874 epoch 41 - iter 40/50 - loss 2.84688303 - samples/sec: 84.62 - lr: 0.125000 +2021-03-26 07:30:29,934 epoch 41 - iter 45/50 - loss 2.83739944 - samples/sec: 77.73 - lr: 0.125000 +2021-03-26 07:30:31,677 epoch 41 - iter 50/50 - loss 2.75893023 - samples/sec: 91.90 - lr: 0.125000 +2021-03-26 07:30:31,678 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:30:31,678 EPOCH 41 done: loss 2.7589 - lr 0.1250000 +2021-03-26 07:30:32,451 DEV : loss 7.06007194519043 - score 0.9005 +2021-03-26 07:30:32,474 BAD EPOCHS (no improvement): 1 +2021-03-26 07:30:32,475 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:30:34,232 epoch 42 - iter 5/50 - loss 2.20300517 - samples/sec: 91.14 - lr: 0.125000 +2021-03-26 07:30:36,209 epoch 42 - iter 10/50 - loss 2.60058794 - samples/sec: 81.01 - lr: 0.125000 +2021-03-26 07:30:38,096 epoch 42 - iter 15/50 - loss 2.56362259 - samples/sec: 84.89 - lr: 0.125000 +2021-03-26 07:30:40,114 epoch 42 - iter 20/50 - loss 2.60823840 - samples/sec: 79.32 - lr: 0.125000 +2021-03-26 07:30:41,981 epoch 42 - iter 25/50 - loss 2.54176937 - samples/sec: 85.79 - lr: 0.125000 +2021-03-26 07:30:43,878 epoch 42 - iter 30/50 - loss 2.59154145 - samples/sec: 84.39 - lr: 0.125000 +2021-03-26 07:30:45,792 epoch 42 - iter 35/50 - loss 2.60671992 - samples/sec: 83.68 - lr: 0.125000 +2021-03-26 07:30:47,760 epoch 42 - iter 40/50 - loss 2.60145665 - samples/sec: 81.36 - lr: 0.125000 +2021-03-26 07:30:49,630 epoch 42 - iter 45/50 - loss 2.62242535 - samples/sec: 85.65 - lr: 0.125000 +2021-03-26 07:30:51,461 epoch 42 - iter 50/50 - loss 2.64225022 - samples/sec: 87.44 - lr: 0.125000 +2021-03-26 07:30:51,462 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:30:51,462 EPOCH 42 done: loss 2.6423 - lr 0.1250000 +2021-03-26 07:30:52,213 DEV : loss 6.793901443481445 - score 0.8985 +2021-03-26 07:30:52,237 BAD EPOCHS (no improvement): 2 +2021-03-26 07:30:52,237 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:30:54,094 epoch 43 - iter 5/50 - loss 2.99211526 - samples/sec: 86.26 - lr: 0.125000 +2021-03-26 07:30:55,996 epoch 43 - iter 10/50 - loss 3.00851283 - samples/sec: 84.22 - lr: 0.125000 +2021-03-26 07:30:57,841 epoch 43 - iter 15/50 - loss 2.80074444 - samples/sec: 86.83 - lr: 0.125000 +2021-03-26 07:30:59,641 epoch 43 - iter 20/50 - loss 2.79492873 - samples/sec: 88.95 - lr: 0.125000 +2021-03-26 07:31:01,367 epoch 43 - iter 25/50 - loss 2.69082284 - samples/sec: 92.77 - lr: 0.125000 +2021-03-26 07:31:03,222 epoch 43 - iter 30/50 - loss 2.71694736 - samples/sec: 86.36 - lr: 0.125000 +2021-03-26 07:31:04,991 epoch 43 - iter 35/50 - loss 2.74828994 - samples/sec: 90.51 - lr: 0.125000 +2021-03-26 07:31:06,992 epoch 43 - iter 40/50 - loss 2.68706407 - samples/sec: 80.06 - lr: 0.125000 +2021-03-26 07:31:08,941 epoch 43 - iter 45/50 - loss 2.67314792 - samples/sec: 82.15 - lr: 0.125000 +2021-03-26 07:31:10,759 epoch 43 - iter 50/50 - loss 2.70004877 - samples/sec: 88.09 - lr: 0.125000 +2021-03-26 07:31:10,760 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:31:10,760 EPOCH 43 done: loss 2.7000 - lr 0.1250000 +2021-03-26 07:31:12,603 DEV : loss 6.907846450805664 - score 0.8958 +2021-03-26 07:31:12,627 BAD EPOCHS (no improvement): 3 +2021-03-26 07:31:12,628 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:31:14,573 epoch 44 - iter 5/50 - loss 2.65794680 - samples/sec: 82.31 - lr: 0.125000 +2021-03-26 07:31:16,429 epoch 44 - iter 10/50 - loss 2.65035695 - samples/sec: 86.29 - lr: 0.125000 +2021-03-26 07:31:18,413 epoch 44 - iter 15/50 - loss 2.43190594 - samples/sec: 80.73 - lr: 0.125000 +2021-03-26 07:31:20,506 epoch 44 - iter 20/50 - loss 2.35712910 - samples/sec: 76.49 - lr: 0.125000 +2021-03-26 07:31:22,407 epoch 44 - iter 25/50 - loss 2.42408473 - samples/sec: 84.25 - lr: 0.125000 +2021-03-26 07:31:24,360 epoch 44 - iter 30/50 - loss 2.43956635 - samples/sec: 82.00 - lr: 0.125000 +2021-03-26 07:31:26,108 epoch 44 - iter 35/50 - loss 2.39717156 - samples/sec: 91.59 - lr: 0.125000 +2021-03-26 07:31:27,983 epoch 44 - iter 40/50 - loss 2.40287006 - samples/sec: 85.42 - lr: 0.125000 +2021-03-26 07:31:29,855 epoch 44 - iter 45/50 - loss 2.48455736 - samples/sec: 85.52 - lr: 0.125000 +2021-03-26 07:31:31,687 epoch 44 - iter 50/50 - loss 2.45833510 - samples/sec: 87.43 - lr: 0.125000 +2021-03-26 07:31:31,687 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:31:31,688 EPOCH 44 done: loss 2.4583 - lr 0.1250000 +2021-03-26 07:31:32,430 DEV : loss 6.795278072357178 - score 0.9013 +2021-03-26 07:31:32,453 BAD EPOCHS (no improvement): 0 +2021-03-26 07:31:41,582 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:31:43,686 epoch 45 - iter 5/50 - loss 2.66865928 - samples/sec: 76.12 - lr: 0.125000 +2021-03-26 07:31:45,682 epoch 45 - iter 10/50 - loss 2.62356399 - samples/sec: 80.21 - lr: 0.125000 +2021-03-26 07:31:47,632 epoch 45 - iter 15/50 - loss 2.62658451 - samples/sec: 82.13 - lr: 0.125000 +2021-03-26 07:31:49,495 epoch 45 - iter 20/50 - loss 2.51406005 - samples/sec: 85.93 - lr: 0.125000 +2021-03-26 07:31:51,298 epoch 45 - iter 25/50 - loss 2.43753824 - samples/sec: 88.83 - lr: 0.125000 +2021-03-26 07:31:53,130 epoch 45 - iter 30/50 - loss 2.41095030 - samples/sec: 87.46 - lr: 0.125000 +2021-03-26 07:31:55,089 epoch 45 - iter 35/50 - loss 2.47412571 - samples/sec: 81.72 - lr: 0.125000 +2021-03-26 07:31:57,064 epoch 45 - iter 40/50 - loss 2.51294028 - samples/sec: 81.11 - lr: 0.125000 +2021-03-26 07:31:59,012 epoch 45 - iter 45/50 - loss 2.51155817 - samples/sec: 82.21 - lr: 0.125000 +2021-03-26 07:32:00,785 epoch 45 - iter 50/50 - loss 2.48287714 - samples/sec: 90.32 - lr: 0.125000 +2021-03-26 07:32:00,786 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:32:00,786 EPOCH 45 done: loss 2.4829 - lr 0.1250000 +2021-03-26 07:32:01,572 DEV : loss 6.7710466384887695 - score 0.9001 +2021-03-26 07:32:01,587 BAD EPOCHS (no improvement): 1 +2021-03-26 07:32:01,588 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:32:03,478 epoch 46 - iter 5/50 - loss 2.60788822 - samples/sec: 84.73 - lr: 0.125000 +2021-03-26 07:32:05,434 epoch 46 - iter 10/50 - loss 2.64636929 - samples/sec: 81.88 - lr: 0.125000 +2021-03-26 07:32:07,406 epoch 46 - iter 15/50 - loss 2.56494276 - samples/sec: 81.19 - lr: 0.125000 +2021-03-26 07:32:09,479 epoch 46 - iter 20/50 - loss 2.48922122 - samples/sec: 77.25 - lr: 0.125000 +2021-03-26 07:32:11,354 epoch 46 - iter 25/50 - loss 2.39800469 - samples/sec: 85.41 - lr: 0.125000 +2021-03-26 07:32:13,496 epoch 46 - iter 30/50 - loss 2.46986194 - samples/sec: 74.73 - lr: 0.125000 +2021-03-26 07:32:15,438 epoch 46 - iter 35/50 - loss 2.52575457 - samples/sec: 82.46 - lr: 0.125000 +2021-03-26 07:32:17,230 epoch 46 - iter 40/50 - loss 2.51320742 - samples/sec: 89.42 - lr: 0.125000 +2021-03-26 07:32:19,111 epoch 46 - iter 45/50 - loss 2.50932122 - samples/sec: 85.10 - lr: 0.125000 +2021-03-26 07:32:20,931 epoch 46 - iter 50/50 - loss 2.51372438 - samples/sec: 87.99 - lr: 0.125000 +2021-03-26 07:32:20,932 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:32:20,933 EPOCH 46 done: loss 2.5137 - lr 0.1250000 +2021-03-26 07:32:21,761 DEV : loss 6.900432109832764 - score 0.9009 +2021-03-26 07:32:21,784 BAD EPOCHS (no improvement): 2 +2021-03-26 07:32:21,785 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:32:23,828 epoch 47 - iter 5/50 - loss 2.17711518 - samples/sec: 78.39 - lr: 0.125000 +2021-03-26 07:32:25,751 epoch 47 - iter 10/50 - loss 2.16294295 - samples/sec: 83.29 - lr: 0.125000 +2021-03-26 07:32:27,646 epoch 47 - iter 15/50 - loss 2.39153829 - samples/sec: 84.51 - lr: 0.125000 +2021-03-26 07:32:29,583 epoch 47 - iter 20/50 - loss 2.47488337 - samples/sec: 82.68 - lr: 0.125000 +2021-03-26 07:32:31,383 epoch 47 - iter 25/50 - loss 2.43727724 - samples/sec: 88.96 - lr: 0.125000 +2021-03-26 07:32:33,312 epoch 47 - iter 30/50 - loss 2.39282555 - samples/sec: 83.04 - lr: 0.125000 +2021-03-26 07:32:35,297 epoch 47 - iter 35/50 - loss 2.37608892 - samples/sec: 80.67 - lr: 0.125000 +2021-03-26 07:32:37,316 epoch 47 - iter 40/50 - loss 2.43819055 - samples/sec: 79.31 - lr: 0.125000 +2021-03-26 07:32:39,249 epoch 47 - iter 45/50 - loss 2.43623858 - samples/sec: 82.85 - lr: 0.125000 +2021-03-26 07:32:41,043 epoch 47 - iter 50/50 - loss 2.43931659 - samples/sec: 89.28 - lr: 0.125000 +2021-03-26 07:32:41,044 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:32:41,044 EPOCH 47 done: loss 2.4393 - lr 0.1250000 +2021-03-26 07:32:41,829 DEV : loss 6.888872146606445 - score 0.9058 +2021-03-26 07:32:41,853 BAD EPOCHS (no improvement): 0 +2021-03-26 07:32:51,326 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:32:53,272 epoch 48 - iter 5/50 - loss 2.19716325 - samples/sec: 82.35 - lr: 0.125000 +2021-03-26 07:32:55,250 epoch 48 - iter 10/50 - loss 2.12402178 - samples/sec: 80.97 - lr: 0.125000 +2021-03-26 07:32:57,196 epoch 48 - iter 15/50 - loss 2.37712937 - samples/sec: 82.29 - lr: 0.125000 +2021-03-26 07:32:59,158 epoch 48 - iter 20/50 - loss 2.46164612 - samples/sec: 81.59 - lr: 0.125000 +2021-03-26 07:33:01,030 epoch 48 - iter 25/50 - loss 2.40853381 - samples/sec: 85.58 - lr: 0.125000 +2021-03-26 07:33:02,830 epoch 48 - iter 30/50 - loss 2.40115452 - samples/sec: 88.98 - lr: 0.125000 +2021-03-26 07:33:04,636 epoch 48 - iter 35/50 - loss 2.41172790 - samples/sec: 88.72 - lr: 0.125000 +2021-03-26 07:33:06,417 epoch 48 - iter 40/50 - loss 2.43344751 - samples/sec: 89.90 - lr: 0.125000 +2021-03-26 07:33:08,403 epoch 48 - iter 45/50 - loss 2.36635066 - samples/sec: 80.63 - lr: 0.125000 +2021-03-26 07:33:10,071 epoch 48 - iter 50/50 - loss 2.31805849 - samples/sec: 96.03 - lr: 0.125000 +2021-03-26 07:33:10,072 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:33:10,072 EPOCH 48 done: loss 2.3181 - lr 0.1250000 +2021-03-26 07:33:10,825 DEV : loss 6.771639823913574 - score 0.9038 +2021-03-26 07:33:10,849 BAD EPOCHS (no improvement): 1 +2021-03-26 07:33:10,850 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:33:12,763 epoch 49 - iter 5/50 - loss 2.02018476 - samples/sec: 83.75 - lr: 0.125000 +2021-03-26 07:33:14,858 epoch 49 - iter 10/50 - loss 2.49009266 - samples/sec: 76.47 - lr: 0.125000 +2021-03-26 07:33:16,829 epoch 49 - iter 15/50 - loss 2.47853305 - samples/sec: 81.24 - lr: 0.125000 +2021-03-26 07:33:18,545 epoch 49 - iter 20/50 - loss 2.35995474 - samples/sec: 93.34 - lr: 0.125000 +2021-03-26 07:33:20,647 epoch 49 - iter 25/50 - loss 2.44599000 - samples/sec: 76.17 - lr: 0.125000 +2021-03-26 07:33:22,585 epoch 49 - iter 30/50 - loss 2.42486334 - samples/sec: 82.63 - lr: 0.125000 +2021-03-26 07:33:24,695 epoch 49 - iter 35/50 - loss 2.40869852 - samples/sec: 75.91 - lr: 0.125000 +2021-03-26 07:33:26,614 epoch 49 - iter 40/50 - loss 2.40710014 - samples/sec: 83.44 - lr: 0.125000 +2021-03-26 07:33:28,519 epoch 49 - iter 45/50 - loss 2.39904715 - samples/sec: 84.05 - lr: 0.125000 +2021-03-26 07:33:30,267 epoch 49 - iter 50/50 - loss 2.49603957 - samples/sec: 91.62 - lr: 0.125000 +2021-03-26 07:33:30,268 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:33:30,269 EPOCH 49 done: loss 2.4960 - lr 0.1250000 +2021-03-26 07:33:31,042 DEV : loss 6.907740592956543 - score 0.8997 +2021-03-26 07:33:31,066 BAD EPOCHS (no improvement): 2 +2021-03-26 07:33:31,067 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:33:33,085 epoch 50 - iter 5/50 - loss 2.59914744 - samples/sec: 79.35 - lr: 0.125000 +2021-03-26 07:33:35,070 epoch 50 - iter 10/50 - loss 2.43452330 - samples/sec: 80.65 - lr: 0.125000 +2021-03-26 07:33:37,278 epoch 50 - iter 15/50 - loss 2.46958216 - samples/sec: 72.53 - lr: 0.125000 +2021-03-26 07:33:39,115 epoch 50 - iter 20/50 - loss 2.50137873 - samples/sec: 87.18 - lr: 0.125000 +2021-03-26 07:33:40,963 epoch 50 - iter 25/50 - loss 2.44457360 - samples/sec: 86.64 - lr: 0.125000 +2021-03-26 07:33:42,959 epoch 50 - iter 30/50 - loss 2.38336607 - samples/sec: 80.24 - lr: 0.125000 +2021-03-26 07:33:44,884 epoch 50 - iter 35/50 - loss 2.34478426 - samples/sec: 83.19 - lr: 0.125000 +2021-03-26 07:33:46,766 epoch 50 - iter 40/50 - loss 2.36545067 - samples/sec: 85.06 - lr: 0.125000 +2021-03-26 07:33:48,718 epoch 50 - iter 45/50 - loss 2.32171084 - samples/sec: 82.04 - lr: 0.125000 +2021-03-26 07:33:50,552 epoch 50 - iter 50/50 - loss 2.32172669 - samples/sec: 87.32 - lr: 0.125000 +2021-03-26 07:33:50,553 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:33:50,554 EPOCH 50 done: loss 2.3217 - lr 0.1250000 +2021-03-26 07:33:51,325 DEV : loss 6.9584059715271 - score 0.9001 +2021-03-26 07:33:51,348 BAD EPOCHS (no improvement): 3 +2021-03-26 07:33:51,349 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:33:53,245 epoch 51 - iter 5/50 - loss 2.67340035 - samples/sec: 84.46 - lr: 0.125000 +2021-03-26 07:33:55,107 epoch 51 - iter 10/50 - loss 2.55235028 - samples/sec: 86.03 - lr: 0.125000 +2021-03-26 07:33:57,007 epoch 51 - iter 15/50 - loss 2.41178315 - samples/sec: 84.26 - lr: 0.125000 +2021-03-26 07:33:59,093 epoch 51 - iter 20/50 - loss 2.43315674 - samples/sec: 76.78 - lr: 0.125000 +2021-03-26 07:34:00,902 epoch 51 - iter 25/50 - loss 2.43270576 - samples/sec: 88.55 - lr: 0.125000 +2021-03-26 07:34:02,906 epoch 51 - iter 30/50 - loss 2.46247502 - samples/sec: 79.89 - lr: 0.125000 +2021-03-26 07:34:04,810 epoch 51 - iter 35/50 - loss 2.42736442 - samples/sec: 84.15 - lr: 0.125000 +2021-03-26 07:34:06,616 epoch 51 - iter 40/50 - loss 2.43767639 - samples/sec: 88.66 - lr: 0.125000 +2021-03-26 07:34:08,503 epoch 51 - iter 45/50 - loss 2.43689898 - samples/sec: 84.89 - lr: 0.125000 +2021-03-26 07:34:10,323 epoch 51 - iter 50/50 - loss 2.46275803 - samples/sec: 88.00 - lr: 0.125000 +2021-03-26 07:34:10,324 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:34:10,325 EPOCH 51 done: loss 2.4628 - lr 0.1250000 +2021-03-26 07:34:11,097 DEV : loss 7.135655879974365 - score 0.8993 +2021-03-26 07:34:11,120 BAD EPOCHS (no improvement): 4 +2021-03-26 07:34:11,121 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:34:13,126 epoch 52 - iter 5/50 - loss 2.37440939 - samples/sec: 79.89 - lr: 0.062500 +2021-03-26 07:34:15,203 epoch 52 - iter 10/50 - loss 2.19602715 - samples/sec: 77.09 - lr: 0.062500 +2021-03-26 07:34:17,264 epoch 52 - iter 15/50 - loss 2.14743586 - samples/sec: 77.68 - lr: 0.062500 +2021-03-26 07:34:19,074 epoch 52 - iter 20/50 - loss 2.16298716 - samples/sec: 88.50 - lr: 0.062500 +2021-03-26 07:34:20,918 epoch 52 - iter 25/50 - loss 2.23282931 - samples/sec: 86.83 - lr: 0.062500 +2021-03-26 07:34:22,807 epoch 52 - iter 30/50 - loss 2.24415377 - samples/sec: 84.77 - lr: 0.062500 +2021-03-26 07:34:24,701 epoch 52 - iter 35/50 - loss 2.22840363 - samples/sec: 84.57 - lr: 0.062500 +2021-03-26 07:34:26,556 epoch 52 - iter 40/50 - loss 2.27091244 - samples/sec: 86.31 - lr: 0.062500 +2021-03-26 07:34:28,460 epoch 52 - iter 45/50 - loss 2.29454314 - samples/sec: 84.17 - lr: 0.062500 +2021-03-26 07:34:30,182 epoch 52 - iter 50/50 - loss 2.28486887 - samples/sec: 92.97 - lr: 0.062500 +2021-03-26 07:34:30,183 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:34:30,183 EPOCH 52 done: loss 2.2849 - lr 0.0625000 +2021-03-26 07:34:31,000 DEV : loss 6.901858329772949 - score 0.9021 +2021-03-26 07:34:31,024 BAD EPOCHS (no improvement): 1 +2021-03-26 07:34:31,025 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:34:32,894 epoch 53 - iter 5/50 - loss 2.22966437 - samples/sec: 85.69 - lr: 0.062500 +2021-03-26 07:34:34,851 epoch 53 - iter 10/50 - loss 2.42526587 - samples/sec: 81.82 - lr: 0.062500 +2021-03-26 07:34:36,650 epoch 53 - iter 15/50 - loss 2.38672217 - samples/sec: 89.04 - lr: 0.062500 +2021-03-26 07:34:38,679 epoch 53 - iter 20/50 - loss 2.36199059 - samples/sec: 78.92 - lr: 0.062500 +2021-03-26 07:34:40,667 epoch 53 - iter 25/50 - loss 2.39168876 - samples/sec: 80.53 - lr: 0.062500 +2021-03-26 07:34:42,463 epoch 53 - iter 30/50 - loss 2.40719418 - samples/sec: 89.20 - lr: 0.062500 +2021-03-26 07:34:44,433 epoch 53 - iter 35/50 - loss 2.35297212 - samples/sec: 81.29 - lr: 0.062500 +2021-03-26 07:34:46,681 epoch 53 - iter 40/50 - loss 2.35524164 - samples/sec: 71.21 - lr: 0.062500 +2021-03-26 07:34:48,626 epoch 53 - iter 45/50 - loss 2.35598461 - samples/sec: 82.32 - lr: 0.062500 +2021-03-26 07:34:50,347 epoch 53 - iter 50/50 - loss 2.37433032 - samples/sec: 93.04 - lr: 0.062500 +2021-03-26 07:34:50,349 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:34:50,349 EPOCH 53 done: loss 2.3743 - lr 0.0625000 +2021-03-26 07:34:51,113 DEV : loss 6.839196681976318 - score 0.9046 +2021-03-26 07:34:51,136 BAD EPOCHS (no improvement): 2 +2021-03-26 07:34:51,137 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:34:53,007 epoch 54 - iter 5/50 - loss 2.27673917 - samples/sec: 85.66 - lr: 0.062500 +2021-03-26 07:34:55,023 epoch 54 - iter 10/50 - loss 2.14428871 - samples/sec: 79.43 - lr: 0.062500 +2021-03-26 07:34:56,831 epoch 54 - iter 15/50 - loss 2.28928282 - samples/sec: 88.58 - lr: 0.062500 +2021-03-26 07:34:58,829 epoch 54 - iter 20/50 - loss 2.22522359 - samples/sec: 80.16 - lr: 0.062500 +2021-03-26 07:35:00,861 epoch 54 - iter 25/50 - loss 2.26239754 - samples/sec: 78.81 - lr: 0.062500 +2021-03-26 07:35:02,671 epoch 54 - iter 30/50 - loss 2.30489551 - samples/sec: 88.46 - lr: 0.062500 +2021-03-26 07:35:04,581 epoch 54 - iter 35/50 - loss 2.31927896 - samples/sec: 83.85 - lr: 0.062500 +2021-03-26 07:35:06,708 epoch 54 - iter 40/50 - loss 2.34227904 - samples/sec: 75.29 - lr: 0.062500 +2021-03-26 07:35:08,487 epoch 54 - iter 45/50 - loss 2.33702100 - samples/sec: 90.05 - lr: 0.062500 +2021-03-26 07:35:10,331 epoch 54 - iter 50/50 - loss 2.34119186 - samples/sec: 86.83 - lr: 0.062500 +2021-03-26 07:35:10,332 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:35:10,332 EPOCH 54 done: loss 2.3412 - lr 0.0625000 +2021-03-26 07:35:11,131 DEV : loss 6.923906326293945 - score 0.9042 +2021-03-26 07:35:11,164 BAD EPOCHS (no improvement): 3 +2021-03-26 07:35:11,165 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:35:12,986 epoch 55 - iter 5/50 - loss 2.08973429 - samples/sec: 87.94 - lr: 0.062500 +2021-03-26 07:35:14,894 epoch 55 - iter 10/50 - loss 2.20089608 - samples/sec: 83.92 - lr: 0.062500 +2021-03-26 07:35:16,799 epoch 55 - iter 15/50 - loss 2.15231233 - samples/sec: 84.08 - lr: 0.062500 +2021-03-26 07:35:18,687 epoch 55 - iter 20/50 - loss 2.22952471 - samples/sec: 84.85 - lr: 0.062500 +2021-03-26 07:35:20,659 epoch 55 - iter 25/50 - loss 2.24563358 - samples/sec: 81.17 - lr: 0.062500 +2021-03-26 07:35:22,479 epoch 55 - iter 30/50 - loss 2.27382298 - samples/sec: 88.04 - lr: 0.062500 +2021-03-26 07:35:24,381 epoch 55 - iter 35/50 - loss 2.24468018 - samples/sec: 84.19 - lr: 0.062500 +2021-03-26 07:35:26,116 epoch 55 - iter 40/50 - loss 2.24657573 - samples/sec: 92.28 - lr: 0.062500 +2021-03-26 07:35:27,975 epoch 55 - iter 45/50 - loss 2.27305697 - samples/sec: 86.13 - lr: 0.062500 +2021-03-26 07:35:29,662 epoch 55 - iter 50/50 - loss 2.28477174 - samples/sec: 94.96 - lr: 0.062500 +2021-03-26 07:35:29,663 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:35:29,663 EPOCH 55 done: loss 2.2848 - lr 0.0625000 +2021-03-26 07:35:30,400 DEV : loss 6.859457969665527 - score 0.903 +2021-03-26 07:35:30,423 BAD EPOCHS (no improvement): 4 +2021-03-26 07:35:30,423 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:35:32,314 epoch 56 - iter 5/50 - loss 2.43909552 - samples/sec: 84.73 - lr: 0.031250 +2021-03-26 07:35:34,214 epoch 56 - iter 10/50 - loss 2.14584787 - samples/sec: 84.30 - lr: 0.031250 +2021-03-26 07:35:36,107 epoch 56 - iter 15/50 - loss 2.06615899 - samples/sec: 84.62 - lr: 0.031250 +2021-03-26 07:35:37,919 epoch 56 - iter 20/50 - loss 2.14122699 - samples/sec: 88.35 - lr: 0.031250 +2021-03-26 07:35:39,958 epoch 56 - iter 25/50 - loss 2.30763709 - samples/sec: 78.56 - lr: 0.031250 +2021-03-26 07:35:41,878 epoch 56 - iter 30/50 - loss 2.30141586 - samples/sec: 83.42 - lr: 0.031250 +2021-03-26 07:35:43,886 epoch 56 - iter 35/50 - loss 2.27650926 - samples/sec: 79.73 - lr: 0.031250 +2021-03-26 07:35:45,623 epoch 56 - iter 40/50 - loss 2.20432305 - samples/sec: 92.22 - lr: 0.031250 +2021-03-26 07:35:47,470 epoch 56 - iter 45/50 - loss 2.18452919 - samples/sec: 86.69 - lr: 0.031250 +2021-03-26 07:35:49,279 epoch 56 - iter 50/50 - loss 2.20786890 - samples/sec: 88.54 - lr: 0.031250 +2021-03-26 07:35:49,279 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:35:49,280 EPOCH 56 done: loss 2.2079 - lr 0.0312500 +2021-03-26 07:35:50,032 DEV : loss 6.856510162353516 - score 0.9034 +2021-03-26 07:35:50,056 BAD EPOCHS (no improvement): 1 +2021-03-26 07:35:50,057 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:35:51,895 epoch 57 - iter 5/50 - loss 2.20711362 - samples/sec: 87.11 - lr: 0.031250 +2021-03-26 07:35:53,859 epoch 57 - iter 10/50 - loss 2.09820828 - samples/sec: 81.53 - lr: 0.031250 +2021-03-26 07:35:55,852 epoch 57 - iter 15/50 - loss 2.17774463 - samples/sec: 80.39 - lr: 0.031250 +2021-03-26 07:35:57,847 epoch 57 - iter 20/50 - loss 2.19380504 - samples/sec: 80.25 - lr: 0.031250 +2021-03-26 07:35:59,793 epoch 57 - iter 25/50 - loss 2.25433336 - samples/sec: 82.30 - lr: 0.031250 +2021-03-26 07:36:01,485 epoch 57 - iter 30/50 - loss 2.18909953 - samples/sec: 94.70 - lr: 0.031250 +2021-03-26 07:36:03,414 epoch 57 - iter 35/50 - loss 2.21699893 - samples/sec: 83.00 - lr: 0.031250 +2021-03-26 07:36:05,258 epoch 57 - iter 40/50 - loss 2.20707081 - samples/sec: 86.82 - lr: 0.031250 +2021-03-26 07:36:07,154 epoch 57 - iter 45/50 - loss 2.18183962 - samples/sec: 84.49 - lr: 0.031250 +2021-03-26 07:36:08,967 epoch 57 - iter 50/50 - loss 2.14463658 - samples/sec: 88.33 - lr: 0.031250 +2021-03-26 07:36:08,968 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:36:08,968 EPOCH 57 done: loss 2.1446 - lr 0.0312500 +2021-03-26 07:36:09,739 DEV : loss 6.898594856262207 - score 0.9021 +2021-03-26 07:36:09,762 BAD EPOCHS (no improvement): 2 +2021-03-26 07:36:09,763 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:36:11,604 epoch 58 - iter 5/50 - loss 2.11004088 - samples/sec: 86.98 - lr: 0.031250 +2021-03-26 07:36:13,716 epoch 58 - iter 10/50 - loss 2.17785354 - samples/sec: 75.84 - lr: 0.031250 +2021-03-26 07:36:15,752 epoch 58 - iter 15/50 - loss 2.09577750 - samples/sec: 78.66 - lr: 0.031250 +2021-03-26 07:36:17,695 epoch 58 - iter 20/50 - loss 2.15892507 - samples/sec: 82.43 - lr: 0.031250 +2021-03-26 07:36:19,600 epoch 58 - iter 25/50 - loss 2.18243889 - samples/sec: 84.09 - lr: 0.031250 +2021-03-26 07:36:21,597 epoch 58 - iter 30/50 - loss 2.20849555 - samples/sec: 80.18 - lr: 0.031250 +2021-03-26 07:36:23,455 epoch 58 - iter 35/50 - loss 2.19579933 - samples/sec: 86.18 - lr: 0.031250 +2021-03-26 07:36:25,829 epoch 58 - iter 40/50 - loss 2.20347180 - samples/sec: 67.46 - lr: 0.031250 +2021-03-26 07:36:27,922 epoch 58 - iter 45/50 - loss 2.14917173 - samples/sec: 76.51 - lr: 0.031250 +2021-03-26 07:36:29,699 epoch 58 - iter 50/50 - loss 2.12325423 - samples/sec: 90.12 - lr: 0.031250 +2021-03-26 07:36:29,700 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:36:29,700 EPOCH 58 done: loss 2.1233 - lr 0.0312500 +2021-03-26 07:36:30,470 DEV : loss 6.878035545349121 - score 0.9021 +2021-03-26 07:36:30,486 BAD EPOCHS (no improvement): 3 +2021-03-26 07:36:30,487 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:36:32,199 epoch 59 - iter 5/50 - loss 1.79576149 - samples/sec: 93.55 - lr: 0.031250 +2021-03-26 07:36:34,316 epoch 59 - iter 10/50 - loss 1.99188758 - samples/sec: 75.64 - lr: 0.031250 +2021-03-26 07:36:36,503 epoch 59 - iter 15/50 - loss 1.93114869 - samples/sec: 73.23 - lr: 0.031250 +2021-03-26 07:36:38,470 epoch 59 - iter 20/50 - loss 2.04898582 - samples/sec: 81.41 - lr: 0.031250 +2021-03-26 07:36:40,451 epoch 59 - iter 25/50 - loss 2.05024611 - samples/sec: 80.83 - lr: 0.031250 +2021-03-26 07:36:42,341 epoch 59 - iter 30/50 - loss 2.11055830 - samples/sec: 84.71 - lr: 0.031250 +2021-03-26 07:36:44,205 epoch 59 - iter 35/50 - loss 2.07869013 - samples/sec: 85.90 - lr: 0.031250 +2021-03-26 07:36:46,054 epoch 59 - iter 40/50 - loss 2.07881030 - samples/sec: 86.63 - lr: 0.031250 +2021-03-26 07:36:47,989 epoch 59 - iter 45/50 - loss 2.10570982 - samples/sec: 82.75 - lr: 0.031250 +2021-03-26 07:36:49,789 epoch 59 - iter 50/50 - loss 2.15876318 - samples/sec: 88.98 - lr: 0.031250 +2021-03-26 07:36:49,790 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:36:49,790 EPOCH 59 done: loss 2.1588 - lr 0.0312500 +2021-03-26 07:36:50,538 DEV : loss 6.879229545593262 - score 0.9034 +2021-03-26 07:36:50,561 BAD EPOCHS (no improvement): 4 +2021-03-26 07:36:50,562 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:36:52,470 epoch 60 - iter 5/50 - loss 2.24569612 - samples/sec: 83.94 - lr: 0.015625 +2021-03-26 07:36:54,424 epoch 60 - iter 10/50 - loss 2.16484799 - samples/sec: 81.97 - lr: 0.015625 +2021-03-26 07:36:56,284 epoch 60 - iter 15/50 - loss 2.07707895 - samples/sec: 86.09 - lr: 0.015625 +2021-03-26 07:36:58,097 epoch 60 - iter 20/50 - loss 2.07254428 - samples/sec: 88.34 - lr: 0.015625 +2021-03-26 07:37:00,098 epoch 60 - iter 25/50 - loss 2.11141718 - samples/sec: 80.04 - lr: 0.015625 +2021-03-26 07:37:01,970 epoch 60 - iter 30/50 - loss 2.16300511 - samples/sec: 85.56 - lr: 0.015625 +2021-03-26 07:37:04,210 epoch 60 - iter 35/50 - loss 2.17970347 - samples/sec: 71.49 - lr: 0.015625 +2021-03-26 07:37:06,194 epoch 60 - iter 40/50 - loss 2.14677085 - samples/sec: 80.71 - lr: 0.015625 +2021-03-26 07:37:08,151 epoch 60 - iter 45/50 - loss 2.19213055 - samples/sec: 81.82 - lr: 0.015625 +2021-03-26 07:37:09,798 epoch 60 - iter 50/50 - loss 2.19411132 - samples/sec: 97.22 - lr: 0.015625 +2021-03-26 07:37:09,799 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:37:09,800 EPOCH 60 done: loss 2.1941 - lr 0.0156250 +2021-03-26 07:37:10,560 DEV : loss 6.902478218078613 - score 0.9013 +2021-03-26 07:37:10,583 BAD EPOCHS (no improvement): 1 +2021-03-26 07:37:10,584 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:37:12,558 epoch 61 - iter 5/50 - loss 1.71156173 - samples/sec: 81.11 - lr: 0.015625 +2021-03-26 07:37:14,379 epoch 61 - iter 10/50 - loss 2.01954519 - samples/sec: 87.97 - lr: 0.015625 +2021-03-26 07:37:16,305 epoch 61 - iter 15/50 - loss 2.12591212 - samples/sec: 83.17 - lr: 0.015625 +2021-03-26 07:37:18,326 epoch 61 - iter 20/50 - loss 2.02409774 - samples/sec: 79.24 - lr: 0.015625 +2021-03-26 07:37:20,245 epoch 61 - iter 25/50 - loss 2.01767517 - samples/sec: 83.43 - lr: 0.015625 +2021-03-26 07:37:22,231 epoch 61 - iter 30/50 - loss 2.13288812 - samples/sec: 80.66 - lr: 0.015625 +2021-03-26 07:37:24,110 epoch 61 - iter 35/50 - loss 2.08045435 - samples/sec: 85.21 - lr: 0.015625 +2021-03-26 07:37:26,075 epoch 61 - iter 40/50 - loss 2.11307848 - samples/sec: 81.51 - lr: 0.015625 +2021-03-26 07:37:28,083 epoch 61 - iter 45/50 - loss 2.10234797 - samples/sec: 79.74 - lr: 0.015625 +2021-03-26 07:37:29,939 epoch 61 - iter 50/50 - loss 2.15973810 - samples/sec: 86.28 - lr: 0.015625 +2021-03-26 07:37:29,940 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:37:29,940 EPOCH 61 done: loss 2.1597 - lr 0.0156250 +2021-03-26 07:37:30,696 DEV : loss 6.937427520751953 - score 0.9034 +2021-03-26 07:37:30,719 BAD EPOCHS (no improvement): 2 +2021-03-26 07:37:30,720 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:37:32,540 epoch 62 - iter 5/50 - loss 2.36723418 - samples/sec: 88.03 - lr: 0.015625 +2021-03-26 07:37:34,377 epoch 62 - iter 10/50 - loss 2.27173837 - samples/sec: 87.16 - lr: 0.015625 +2021-03-26 07:37:36,305 epoch 62 - iter 15/50 - loss 2.23157530 - samples/sec: 83.09 - lr: 0.015625 +2021-03-26 07:37:38,239 epoch 62 - iter 20/50 - loss 2.18523218 - samples/sec: 82.85 - lr: 0.015625 +2021-03-26 07:37:40,194 epoch 62 - iter 25/50 - loss 2.12496989 - samples/sec: 81.88 - lr: 0.015625 +2021-03-26 07:37:42,082 epoch 62 - iter 30/50 - loss 2.15814011 - samples/sec: 84.81 - lr: 0.015625 +2021-03-26 07:37:43,911 epoch 62 - iter 35/50 - loss 2.14331431 - samples/sec: 87.54 - lr: 0.015625 +2021-03-26 07:37:45,790 epoch 62 - iter 40/50 - loss 2.16459944 - samples/sec: 85.25 - lr: 0.015625 +2021-03-26 07:37:47,614 epoch 62 - iter 45/50 - loss 2.14495964 - samples/sec: 87.80 - lr: 0.015625 +2021-03-26 07:37:49,495 epoch 62 - iter 50/50 - loss 2.15630883 - samples/sec: 85.14 - lr: 0.015625 +2021-03-26 07:37:49,495 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:37:49,496 EPOCH 62 done: loss 2.1563 - lr 0.0156250 +2021-03-26 07:37:50,270 DEV : loss 6.966155052185059 - score 0.9026 +2021-03-26 07:37:50,289 BAD EPOCHS (no improvement): 3 +2021-03-26 07:37:50,289 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:37:52,372 epoch 63 - iter 5/50 - loss 2.65498366 - samples/sec: 76.90 - lr: 0.015625 +2021-03-26 07:37:54,246 epoch 63 - iter 10/50 - loss 2.41633086 - samples/sec: 85.45 - lr: 0.015625 +2021-03-26 07:37:56,141 epoch 63 - iter 15/50 - loss 2.24042091 - samples/sec: 84.55 - lr: 0.015625 +2021-03-26 07:37:57,922 epoch 63 - iter 20/50 - loss 2.24434867 - samples/sec: 89.89 - lr: 0.015625 +2021-03-26 07:38:00,010 epoch 63 - iter 25/50 - loss 2.21254596 - samples/sec: 76.71 - lr: 0.015625 +2021-03-26 07:38:01,846 epoch 63 - iter 30/50 - loss 2.23309576 - samples/sec: 87.21 - lr: 0.015625 +2021-03-26 07:38:03,664 epoch 63 - iter 35/50 - loss 2.25727123 - samples/sec: 88.13 - lr: 0.015625 +2021-03-26 07:38:05,727 epoch 63 - iter 40/50 - loss 2.29153742 - samples/sec: 77.60 - lr: 0.015625 +2021-03-26 07:38:07,670 epoch 63 - iter 45/50 - loss 2.27542347 - samples/sec: 82.43 - lr: 0.015625 +2021-03-26 07:38:09,298 epoch 63 - iter 50/50 - loss 2.20794976 - samples/sec: 98.42 - lr: 0.015625 +2021-03-26 07:38:09,298 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:38:09,299 EPOCH 63 done: loss 2.2079 - lr 0.0156250 +2021-03-26 07:38:10,041 DEV : loss 6.970630645751953 - score 0.9034 +2021-03-26 07:38:10,064 BAD EPOCHS (no improvement): 4 +2021-03-26 07:38:10,065 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:38:11,867 epoch 64 - iter 5/50 - loss 2.33771241 - samples/sec: 88.87 - lr: 0.007812 +2021-03-26 07:38:13,818 epoch 64 - iter 10/50 - loss 2.31490641 - samples/sec: 82.11 - lr: 0.007812 +2021-03-26 07:38:15,604 epoch 64 - iter 15/50 - loss 2.26052773 - samples/sec: 89.71 - lr: 0.007812 +2021-03-26 07:38:17,435 epoch 64 - iter 20/50 - loss 2.25423276 - samples/sec: 87.44 - lr: 0.007812 +2021-03-26 07:38:19,224 epoch 64 - iter 25/50 - loss 2.17361112 - samples/sec: 89.55 - lr: 0.007812 +2021-03-26 07:38:21,099 epoch 64 - iter 30/50 - loss 2.22481107 - samples/sec: 85.42 - lr: 0.007812 +2021-03-26 07:38:22,964 epoch 64 - iter 35/50 - loss 2.21949503 - samples/sec: 85.83 - lr: 0.007812 +2021-03-26 07:38:24,980 epoch 64 - iter 40/50 - loss 2.25171840 - samples/sec: 79.43 - lr: 0.007812 +2021-03-26 07:38:26,948 epoch 64 - iter 45/50 - loss 2.24363305 - samples/sec: 81.39 - lr: 0.007812 +2021-03-26 07:38:28,687 epoch 64 - iter 50/50 - loss 2.19226521 - samples/sec: 92.10 - lr: 0.007812 +2021-03-26 07:38:28,687 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:38:28,688 EPOCH 64 done: loss 2.1923 - lr 0.0078125 +2021-03-26 07:38:29,423 DEV : loss 6.981413841247559 - score 0.9038 +2021-03-26 07:38:29,445 BAD EPOCHS (no improvement): 1 +2021-03-26 07:38:29,446 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:38:31,416 epoch 65 - iter 5/50 - loss 2.65382171 - samples/sec: 81.32 - lr: 0.007812 +2021-03-26 07:38:33,343 epoch 65 - iter 10/50 - loss 2.49962165 - samples/sec: 83.10 - lr: 0.007812 +2021-03-26 07:38:35,096 epoch 65 - iter 15/50 - loss 2.24140796 - samples/sec: 91.39 - lr: 0.007812 +2021-03-26 07:38:37,073 epoch 65 - iter 20/50 - loss 2.36456291 - samples/sec: 80.97 - lr: 0.007812 +2021-03-26 07:38:38,881 epoch 65 - iter 25/50 - loss 2.31400333 - samples/sec: 88.57 - lr: 0.007812 +2021-03-26 07:38:40,631 epoch 65 - iter 30/50 - loss 2.31376214 - samples/sec: 91.55 - lr: 0.007812 +2021-03-26 07:38:42,559 epoch 65 - iter 35/50 - loss 2.36776146 - samples/sec: 83.05 - lr: 0.007812 +2021-03-26 07:38:44,738 epoch 65 - iter 40/50 - loss 2.31570167 - samples/sec: 73.49 - lr: 0.007812 +2021-03-26 07:38:46,633 epoch 65 - iter 45/50 - loss 2.27233456 - samples/sec: 84.51 - lr: 0.007812 +2021-03-26 07:38:48,435 epoch 65 - iter 50/50 - loss 2.19037542 - samples/sec: 88.87 - lr: 0.007812 +2021-03-26 07:38:48,435 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:38:48,436 EPOCH 65 done: loss 2.1904 - lr 0.0078125 +2021-03-26 07:38:49,231 DEV : loss 6.993993759155273 - score 0.903 +2021-03-26 07:38:49,254 BAD EPOCHS (no improvement): 2 +2021-03-26 07:38:49,255 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:38:51,251 epoch 66 - iter 5/50 - loss 2.00561562 - samples/sec: 80.23 - lr: 0.007812 +2021-03-26 07:38:53,199 epoch 66 - iter 10/50 - loss 2.08727456 - samples/sec: 82.21 - lr: 0.007812 +2021-03-26 07:38:54,977 epoch 66 - iter 15/50 - loss 2.09013340 - samples/sec: 90.07 - lr: 0.007812 +2021-03-26 07:38:56,739 epoch 66 - iter 20/50 - loss 2.02487870 - samples/sec: 90.88 - lr: 0.007812 +2021-03-26 07:38:58,921 epoch 66 - iter 25/50 - loss 2.04558960 - samples/sec: 73.39 - lr: 0.007812 +2021-03-26 07:39:00,804 epoch 66 - iter 30/50 - loss 2.07894075 - samples/sec: 85.07 - lr: 0.007812 +2021-03-26 07:39:02,528 epoch 66 - iter 35/50 - loss 2.08804049 - samples/sec: 92.88 - lr: 0.007812 +2021-03-26 07:39:04,480 epoch 66 - iter 40/50 - loss 2.09458185 - samples/sec: 82.03 - lr: 0.007812 +2021-03-26 07:39:06,413 epoch 66 - iter 45/50 - loss 2.11317086 - samples/sec: 82.83 - lr: 0.007812 +2021-03-26 07:39:08,271 epoch 66 - iter 50/50 - loss 2.07886559 - samples/sec: 86.20 - lr: 0.007812 +2021-03-26 07:39:08,272 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:39:08,272 EPOCH 66 done: loss 2.0789 - lr 0.0078125 +2021-03-26 07:39:09,033 DEV : loss 6.994531154632568 - score 0.9034 +2021-03-26 07:39:09,056 BAD EPOCHS (no improvement): 3 +2021-03-26 07:39:09,057 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:39:10,930 epoch 67 - iter 5/50 - loss 2.03548367 - samples/sec: 85.49 - lr: 0.007812 +2021-03-26 07:39:12,657 epoch 67 - iter 10/50 - loss 2.00109907 - samples/sec: 92.76 - lr: 0.007812 +2021-03-26 07:39:14,540 epoch 67 - iter 15/50 - loss 2.15618571 - samples/sec: 85.05 - lr: 0.007812 +2021-03-26 07:39:16,452 epoch 67 - iter 20/50 - loss 2.18579333 - samples/sec: 83.76 - lr: 0.007812 +2021-03-26 07:39:18,346 epoch 67 - iter 25/50 - loss 2.19617332 - samples/sec: 84.57 - lr: 0.007812 +2021-03-26 07:39:20,270 epoch 67 - iter 30/50 - loss 2.17452607 - samples/sec: 83.22 - lr: 0.007812 +2021-03-26 07:39:22,070 epoch 67 - iter 35/50 - loss 2.18120226 - samples/sec: 88.98 - lr: 0.007812 +2021-03-26 07:39:24,166 epoch 67 - iter 40/50 - loss 2.15019251 - samples/sec: 76.39 - lr: 0.007812 +2021-03-26 07:39:26,149 epoch 67 - iter 45/50 - loss 2.11730300 - samples/sec: 80.78 - lr: 0.007812 +2021-03-26 07:39:27,831 epoch 67 - iter 50/50 - loss 2.13063770 - samples/sec: 95.24 - lr: 0.007812 +2021-03-26 07:39:27,831 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:39:27,832 EPOCH 67 done: loss 2.1306 - lr 0.0078125 +2021-03-26 07:39:28,606 DEV : loss 6.979950904846191 - score 0.9026 +2021-03-26 07:39:28,630 BAD EPOCHS (no improvement): 4 +2021-03-26 07:39:28,631 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:39:30,486 epoch 68 - iter 5/50 - loss 2.37680974 - samples/sec: 86.37 - lr: 0.003906 +2021-03-26 07:39:32,411 epoch 68 - iter 10/50 - loss 2.26592950 - samples/sec: 83.17 - lr: 0.003906 +2021-03-26 07:39:34,282 epoch 68 - iter 15/50 - loss 2.11205655 - samples/sec: 85.57 - lr: 0.003906 +2021-03-26 07:39:36,162 epoch 68 - iter 20/50 - loss 2.03751066 - samples/sec: 85.19 - lr: 0.003906 +2021-03-26 07:39:37,970 epoch 68 - iter 25/50 - loss 2.08158858 - samples/sec: 88.58 - lr: 0.003906 +2021-03-26 07:39:39,700 epoch 68 - iter 30/50 - loss 2.10940475 - samples/sec: 92.57 - lr: 0.003906 +2021-03-26 07:39:41,620 epoch 68 - iter 35/50 - loss 2.15269453 - samples/sec: 83.43 - lr: 0.003906 +2021-03-26 07:39:43,378 epoch 68 - iter 40/50 - loss 2.13817808 - samples/sec: 91.07 - lr: 0.003906 +2021-03-26 07:39:45,278 epoch 68 - iter 45/50 - loss 2.13679235 - samples/sec: 84.29 - lr: 0.003906 +2021-03-26 07:39:47,019 epoch 68 - iter 50/50 - loss 2.10934273 - samples/sec: 92.02 - lr: 0.003906 +2021-03-26 07:39:47,019 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:39:47,020 EPOCH 68 done: loss 2.1093 - lr 0.0039062 +2021-03-26 07:39:47,820 DEV : loss 6.979579925537109 - score 0.903 +2021-03-26 07:39:47,843 BAD EPOCHS (no improvement): 1 +2021-03-26 07:39:47,844 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:39:49,797 epoch 69 - iter 5/50 - loss 2.19237757 - samples/sec: 82.01 - lr: 0.003906 +2021-03-26 07:39:51,672 epoch 69 - iter 10/50 - loss 2.21753824 - samples/sec: 85.41 - lr: 0.003906 +2021-03-26 07:39:53,603 epoch 69 - iter 15/50 - loss 2.31954001 - samples/sec: 82.93 - lr: 0.003906 +2021-03-26 07:39:55,499 epoch 69 - iter 20/50 - loss 2.24179201 - samples/sec: 84.45 - lr: 0.003906 +2021-03-26 07:39:57,354 epoch 69 - iter 25/50 - loss 2.20438880 - samples/sec: 86.31 - lr: 0.003906 +2021-03-26 07:39:59,096 epoch 69 - iter 30/50 - loss 2.17145384 - samples/sec: 91.95 - lr: 0.003906 +2021-03-26 07:40:00,874 epoch 69 - iter 35/50 - loss 2.11442671 - samples/sec: 90.09 - lr: 0.003906 +2021-03-26 07:40:02,829 epoch 69 - iter 40/50 - loss 2.15811507 - samples/sec: 81.90 - lr: 0.003906 +2021-03-26 07:40:04,641 epoch 69 - iter 45/50 - loss 2.16106107 - samples/sec: 88.38 - lr: 0.003906 +2021-03-26 07:40:06,419 epoch 69 - iter 50/50 - loss 2.23029785 - samples/sec: 90.08 - lr: 0.003906 +2021-03-26 07:40:06,420 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:40:06,420 EPOCH 69 done: loss 2.2303 - lr 0.0039062 +2021-03-26 07:40:07,189 DEV : loss 6.972920894622803 - score 0.9038 +2021-03-26 07:40:07,212 BAD EPOCHS (no improvement): 2 +2021-03-26 07:40:07,212 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:40:08,933 epoch 70 - iter 5/50 - loss 2.01184175 - samples/sec: 93.09 - lr: 0.003906 +2021-03-26 07:40:10,930 epoch 70 - iter 10/50 - loss 2.08696496 - samples/sec: 80.22 - lr: 0.003906 +2021-03-26 07:40:12,654 epoch 70 - iter 15/50 - loss 1.95998446 - samples/sec: 92.88 - lr: 0.003906 +2021-03-26 07:40:14,568 epoch 70 - iter 20/50 - loss 2.11174902 - samples/sec: 83.69 - lr: 0.003906 +2021-03-26 07:40:16,406 epoch 70 - iter 25/50 - loss 2.08903484 - samples/sec: 87.14 - lr: 0.003906 +2021-03-26 07:40:18,401 epoch 70 - iter 30/50 - loss 2.13611888 - samples/sec: 80.25 - lr: 0.003906 +2021-03-26 07:40:20,342 epoch 70 - iter 35/50 - loss 2.13178201 - samples/sec: 82.53 - lr: 0.003906 +2021-03-26 07:40:22,363 epoch 70 - iter 40/50 - loss 2.13080277 - samples/sec: 79.25 - lr: 0.003906 +2021-03-26 07:40:24,326 epoch 70 - iter 45/50 - loss 2.15558514 - samples/sec: 81.55 - lr: 0.003906 +2021-03-26 07:40:26,018 epoch 70 - iter 50/50 - loss 2.11314784 - samples/sec: 94.69 - lr: 0.003906 +2021-03-26 07:40:26,019 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:40:26,019 EPOCH 70 done: loss 2.1131 - lr 0.0039062 +2021-03-26 07:40:26,761 DEV : loss 6.973997116088867 - score 0.903 +2021-03-26 07:40:26,781 BAD EPOCHS (no improvement): 3 +2021-03-26 07:40:26,782 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:40:28,575 epoch 71 - iter 5/50 - loss 1.67595360 - samples/sec: 89.32 - lr: 0.003906 +2021-03-26 07:40:30,399 epoch 71 - iter 10/50 - loss 1.95769655 - samples/sec: 87.80 - lr: 0.003906 +2021-03-26 07:40:32,351 epoch 71 - iter 15/50 - loss 2.07777414 - samples/sec: 82.02 - lr: 0.003906 +2021-03-26 07:40:34,190 epoch 71 - iter 20/50 - loss 2.15321826 - samples/sec: 87.07 - lr: 0.003906 +2021-03-26 07:40:36,014 epoch 71 - iter 25/50 - loss 2.10259217 - samples/sec: 87.82 - lr: 0.003906 +2021-03-26 07:40:37,842 epoch 71 - iter 30/50 - loss 2.16118869 - samples/sec: 87.62 - lr: 0.003906 +2021-03-26 07:40:39,714 epoch 71 - iter 35/50 - loss 2.17599435 - samples/sec: 85.53 - lr: 0.003906 +2021-03-26 07:40:41,431 epoch 71 - iter 40/50 - loss 2.16178396 - samples/sec: 93.31 - lr: 0.003906 +2021-03-26 07:40:43,301 epoch 71 - iter 45/50 - loss 2.17204736 - samples/sec: 85.63 - lr: 0.003906 +2021-03-26 07:40:45,200 epoch 71 - iter 50/50 - loss 2.12972616 - samples/sec: 84.33 - lr: 0.003906 +2021-03-26 07:40:45,201 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:40:45,201 EPOCH 71 done: loss 2.1297 - lr 0.0039062 +2021-03-26 07:40:45,940 DEV : loss 6.972382545471191 - score 0.9034 +2021-03-26 07:40:45,963 BAD EPOCHS (no improvement): 4 +2021-03-26 07:40:45,963 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:40:47,754 epoch 72 - iter 5/50 - loss 1.90588937 - samples/sec: 89.42 - lr: 0.001953 +2021-03-26 07:40:49,579 epoch 72 - iter 10/50 - loss 2.24016843 - samples/sec: 87.75 - lr: 0.001953 +2021-03-26 07:40:51,509 epoch 72 - iter 15/50 - loss 2.16828419 - samples/sec: 82.98 - lr: 0.001953 +2021-03-26 07:40:53,601 epoch 72 - iter 20/50 - loss 2.19448624 - samples/sec: 76.53 - lr: 0.001953 +2021-03-26 07:40:55,455 epoch 72 - iter 25/50 - loss 2.19809166 - samples/sec: 86.44 - lr: 0.001953 +2021-03-26 07:40:57,508 epoch 72 - iter 30/50 - loss 2.20144189 - samples/sec: 77.98 - lr: 0.001953 +2021-03-26 07:40:59,316 epoch 72 - iter 35/50 - loss 2.22671560 - samples/sec: 88.59 - lr: 0.001953 +2021-03-26 07:41:01,173 epoch 72 - iter 40/50 - loss 2.17772023 - samples/sec: 86.24 - lr: 0.001953 +2021-03-26 07:41:03,015 epoch 72 - iter 45/50 - loss 2.18650738 - samples/sec: 86.95 - lr: 0.001953 +2021-03-26 07:41:04,680 epoch 72 - iter 50/50 - loss 2.16092991 - samples/sec: 96.16 - lr: 0.001953 +2021-03-26 07:41:04,681 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:41:04,681 EPOCH 72 done: loss 2.1609 - lr 0.0019531 +2021-03-26 07:41:05,422 DEV : loss 6.972001552581787 - score 0.9026 +2021-03-26 07:41:05,445 BAD EPOCHS (no improvement): 1 +2021-03-26 07:41:05,446 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:41:07,154 epoch 73 - iter 5/50 - loss 2.01508794 - samples/sec: 93.75 - lr: 0.001953 +2021-03-26 07:41:09,018 epoch 73 - iter 10/50 - loss 2.01582296 - samples/sec: 85.96 - lr: 0.001953 +2021-03-26 07:41:10,979 epoch 73 - iter 15/50 - loss 2.08683247 - samples/sec: 81.65 - lr: 0.001953 +2021-03-26 07:41:12,987 epoch 73 - iter 20/50 - loss 2.14391717 - samples/sec: 79.75 - lr: 0.001953 +2021-03-26 07:41:14,784 epoch 73 - iter 25/50 - loss 2.16491332 - samples/sec: 89.08 - lr: 0.001953 +2021-03-26 07:41:16,612 epoch 73 - iter 30/50 - loss 2.10842774 - samples/sec: 87.61 - lr: 0.001953 +2021-03-26 07:41:18,415 epoch 73 - iter 35/50 - loss 2.14298798 - samples/sec: 88.81 - lr: 0.001953 +2021-03-26 07:41:20,346 epoch 73 - iter 40/50 - loss 2.13127421 - samples/sec: 82.96 - lr: 0.001953 +2021-03-26 07:41:22,330 epoch 73 - iter 45/50 - loss 2.14531492 - samples/sec: 80.72 - lr: 0.001953 +2021-03-26 07:41:24,260 epoch 73 - iter 50/50 - loss 2.21428915 - samples/sec: 82.97 - lr: 0.001953 +2021-03-26 07:41:24,261 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:41:24,261 EPOCH 73 done: loss 2.2143 - lr 0.0019531 +2021-03-26 07:41:25,044 DEV : loss 6.968757152557373 - score 0.9021 +2021-03-26 07:41:25,068 BAD EPOCHS (no improvement): 2 +2021-03-26 07:41:25,069 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:41:26,904 epoch 74 - iter 5/50 - loss 1.73311095 - samples/sec: 87.31 - lr: 0.001953 +2021-03-26 07:41:28,809 epoch 74 - iter 10/50 - loss 1.89269283 - samples/sec: 84.05 - lr: 0.001953 +2021-03-26 07:41:30,945 epoch 74 - iter 15/50 - loss 2.01824257 - samples/sec: 74.96 - lr: 0.001953 +2021-03-26 07:41:32,798 epoch 74 - iter 20/50 - loss 2.00726241 - samples/sec: 86.47 - lr: 0.001953 +2021-03-26 07:41:34,757 epoch 74 - iter 25/50 - loss 2.07169030 - samples/sec: 81.73 - lr: 0.001953 +2021-03-26 07:41:36,715 epoch 74 - iter 30/50 - loss 2.04404934 - samples/sec: 81.80 - lr: 0.001953 +2021-03-26 07:41:38,708 epoch 74 - iter 35/50 - loss 2.02334532 - samples/sec: 80.32 - lr: 0.001953 +2021-03-26 07:41:40,620 epoch 74 - iter 40/50 - loss 2.01096104 - samples/sec: 83.78 - lr: 0.001953 +2021-03-26 07:41:42,494 epoch 74 - iter 45/50 - loss 2.04188972 - samples/sec: 85.45 - lr: 0.001953 +2021-03-26 07:41:44,434 epoch 74 - iter 50/50 - loss 2.05837031 - samples/sec: 82.53 - lr: 0.001953 +2021-03-26 07:41:44,435 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:41:44,436 EPOCH 74 done: loss 2.0584 - lr 0.0019531 +2021-03-26 07:41:45,180 DEV : loss 6.965629577636719 - score 0.9026 +2021-03-26 07:41:45,196 BAD EPOCHS (no improvement): 3 +2021-03-26 07:41:45,197 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:41:46,992 epoch 75 - iter 5/50 - loss 1.92352896 - samples/sec: 89.25 - lr: 0.001953 +2021-03-26 07:41:48,810 epoch 75 - iter 10/50 - loss 1.83091727 - samples/sec: 88.09 - lr: 0.001953 +2021-03-26 07:41:50,914 epoch 75 - iter 15/50 - loss 2.02670843 - samples/sec: 76.11 - lr: 0.001953 +2021-03-26 07:41:52,900 epoch 75 - iter 20/50 - loss 2.15639005 - samples/sec: 80.64 - lr: 0.001953 +2021-03-26 07:41:54,641 epoch 75 - iter 25/50 - loss 2.10837126 - samples/sec: 91.94 - lr: 0.001953 +2021-03-26 07:41:56,614 epoch 75 - iter 30/50 - loss 2.08132513 - samples/sec: 81.15 - lr: 0.001953 +2021-03-26 07:41:58,507 epoch 75 - iter 35/50 - loss 2.08502996 - samples/sec: 84.64 - lr: 0.001953 +2021-03-26 07:42:00,352 epoch 75 - iter 40/50 - loss 2.09589115 - samples/sec: 86.78 - lr: 0.001953 +2021-03-26 07:42:02,167 epoch 75 - iter 45/50 - loss 2.08379084 - samples/sec: 88.25 - lr: 0.001953 +2021-03-26 07:42:04,252 epoch 75 - iter 50/50 - loss 2.13769540 - samples/sec: 76.80 - lr: 0.001953 +2021-03-26 07:42:04,253 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:42:04,253 EPOCH 75 done: loss 2.1377 - lr 0.0019531 +2021-03-26 07:42:04,999 DEV : loss 6.962711334228516 - score 0.903 +2021-03-26 07:42:05,023 BAD EPOCHS (no improvement): 4 +2021-03-26 07:42:05,024 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:42:06,880 epoch 76 - iter 5/50 - loss 2.25840490 - samples/sec: 86.29 - lr: 0.000977 +2021-03-26 07:42:08,742 epoch 76 - iter 10/50 - loss 2.21133794 - samples/sec: 85.99 - lr: 0.000977 +2021-03-26 07:42:10,596 epoch 76 - iter 15/50 - loss 2.18264640 - samples/sec: 86.39 - lr: 0.000977 +2021-03-26 07:42:12,651 epoch 76 - iter 20/50 - loss 2.15330370 - samples/sec: 77.93 - lr: 0.000977 +2021-03-26 07:42:14,499 epoch 76 - iter 25/50 - loss 2.13097959 - samples/sec: 86.64 - lr: 0.000977 +2021-03-26 07:42:16,428 epoch 76 - iter 30/50 - loss 2.11351199 - samples/sec: 83.03 - lr: 0.000977 +2021-03-26 07:42:18,222 epoch 76 - iter 35/50 - loss 2.08954168 - samples/sec: 89.29 - lr: 0.000977 +2021-03-26 07:42:20,143 epoch 76 - iter 40/50 - loss 2.09097285 - samples/sec: 83.35 - lr: 0.000977 +2021-03-26 07:42:21,935 epoch 76 - iter 45/50 - loss 2.11737519 - samples/sec: 89.38 - lr: 0.000977 +2021-03-26 07:42:23,632 epoch 76 - iter 50/50 - loss 2.09018610 - samples/sec: 94.35 - lr: 0.000977 +2021-03-26 07:42:23,633 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:42:23,633 EPOCH 76 done: loss 2.0902 - lr 0.0009766 +2021-03-26 07:42:24,378 DEV : loss 6.960842132568359 - score 0.9026 +2021-03-26 07:42:24,393 BAD EPOCHS (no improvement): 1 +2021-03-26 07:42:24,394 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:42:26,331 epoch 77 - iter 5/50 - loss 2.89146829 - samples/sec: 82.66 - lr: 0.000977 +2021-03-26 07:42:28,101 epoch 77 - iter 10/50 - loss 2.54311110 - samples/sec: 90.51 - lr: 0.000977 +2021-03-26 07:42:30,008 epoch 77 - iter 15/50 - loss 2.33039279 - samples/sec: 83.98 - lr: 0.000977 +2021-03-26 07:42:31,779 epoch 77 - iter 20/50 - loss 2.27280054 - samples/sec: 90.40 - lr: 0.000977 +2021-03-26 07:42:33,568 epoch 77 - iter 25/50 - loss 2.19065623 - samples/sec: 89.55 - lr: 0.000977 +2021-03-26 07:42:35,402 epoch 77 - iter 30/50 - loss 2.19404272 - samples/sec: 87.34 - lr: 0.000977 +2021-03-26 07:42:37,251 epoch 77 - iter 35/50 - loss 2.22139244 - samples/sec: 86.59 - lr: 0.000977 +2021-03-26 07:42:39,080 epoch 77 - iter 40/50 - loss 2.23841817 - samples/sec: 87.55 - lr: 0.000977 +2021-03-26 07:42:40,981 epoch 77 - iter 45/50 - loss 2.23272769 - samples/sec: 84.24 - lr: 0.000977 +2021-03-26 07:42:42,748 epoch 77 - iter 50/50 - loss 2.25151328 - samples/sec: 90.64 - lr: 0.000977 +2021-03-26 07:42:42,749 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:42:42,749 EPOCH 77 done: loss 2.2515 - lr 0.0009766 +2021-03-26 07:42:43,578 DEV : loss 6.961365699768066 - score 0.9021 +2021-03-26 07:42:43,595 BAD EPOCHS (no improvement): 2 +2021-03-26 07:42:43,595 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:42:45,489 epoch 78 - iter 5/50 - loss 2.24054193 - samples/sec: 84.58 - lr: 0.000977 +2021-03-26 07:42:47,366 epoch 78 - iter 10/50 - loss 2.30875487 - samples/sec: 85.33 - lr: 0.000977 +2021-03-26 07:42:49,315 epoch 78 - iter 15/50 - loss 2.24832230 - samples/sec: 82.19 - lr: 0.000977 +2021-03-26 07:42:51,239 epoch 78 - iter 20/50 - loss 2.18487847 - samples/sec: 83.26 - lr: 0.000977 +2021-03-26 07:42:53,163 epoch 78 - iter 25/50 - loss 2.11373229 - samples/sec: 83.21 - lr: 0.000977 +2021-03-26 07:42:55,036 epoch 78 - iter 30/50 - loss 2.11837125 - samples/sec: 85.50 - lr: 0.000977 +2021-03-26 07:42:56,865 epoch 78 - iter 35/50 - loss 2.13645056 - samples/sec: 87.57 - lr: 0.000977 +2021-03-26 07:42:58,632 epoch 78 - iter 40/50 - loss 2.15983583 - samples/sec: 90.65 - lr: 0.000977 +2021-03-26 07:43:00,568 epoch 78 - iter 45/50 - loss 2.12658242 - samples/sec: 82.72 - lr: 0.000977 +2021-03-26 07:43:02,286 epoch 78 - iter 50/50 - loss 2.15449122 - samples/sec: 93.23 - lr: 0.000977 +2021-03-26 07:43:02,287 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:43:02,287 EPOCH 78 done: loss 2.1545 - lr 0.0009766 +2021-03-26 07:43:03,044 DEV : loss 6.962085247039795 - score 0.903 +2021-03-26 07:43:03,068 BAD EPOCHS (no improvement): 3 +2021-03-26 07:43:03,069 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:43:04,873 epoch 79 - iter 5/50 - loss 2.21793807 - samples/sec: 88.78 - lr: 0.000977 +2021-03-26 07:43:06,839 epoch 79 - iter 10/50 - loss 2.29863290 - samples/sec: 81.46 - lr: 0.000977 +2021-03-26 07:43:08,782 epoch 79 - iter 15/50 - loss 2.32131858 - samples/sec: 82.44 - lr: 0.000977 +2021-03-26 07:43:10,800 epoch 79 - iter 20/50 - loss 2.24925628 - samples/sec: 79.34 - lr: 0.000977 +2021-03-26 07:43:12,761 epoch 79 - iter 25/50 - loss 2.27966526 - samples/sec: 81.66 - lr: 0.000977 +2021-03-26 07:43:14,581 epoch 79 - iter 30/50 - loss 2.22692318 - samples/sec: 87.99 - lr: 0.000977 +2021-03-26 07:43:16,503 epoch 79 - iter 35/50 - loss 2.24432735 - samples/sec: 83.30 - lr: 0.000977 +2021-03-26 07:43:18,380 epoch 79 - iter 40/50 - loss 2.16589115 - samples/sec: 85.33 - lr: 0.000977 +2021-03-26 07:43:20,339 epoch 79 - iter 45/50 - loss 2.21727477 - samples/sec: 81.75 - lr: 0.000977 +2021-03-26 07:43:22,056 epoch 79 - iter 50/50 - loss 2.18660015 - samples/sec: 93.30 - lr: 0.000977 +2021-03-26 07:43:22,056 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:43:22,057 EPOCH 79 done: loss 2.1866 - lr 0.0009766 +2021-03-26 07:43:22,825 DEV : loss 6.9618353843688965 - score 0.9034 +2021-03-26 07:43:22,849 BAD EPOCHS (no improvement): 4 +2021-03-26 07:43:22,850 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:43:24,903 epoch 80 - iter 5/50 - loss 2.36558053 - samples/sec: 78.01 - lr: 0.000488 +2021-03-26 07:43:26,783 epoch 80 - iter 10/50 - loss 2.11772819 - samples/sec: 85.21 - lr: 0.000488 +2021-03-26 07:43:28,660 epoch 80 - iter 15/50 - loss 2.01982580 - samples/sec: 85.36 - lr: 0.000488 +2021-03-26 07:43:30,435 epoch 80 - iter 20/50 - loss 1.97181435 - samples/sec: 90.22 - lr: 0.000488 +2021-03-26 07:43:32,457 epoch 80 - iter 25/50 - loss 1.94486945 - samples/sec: 79.18 - lr: 0.000488 +2021-03-26 07:43:34,381 epoch 80 - iter 30/50 - loss 2.03408866 - samples/sec: 83.27 - lr: 0.000488 +2021-03-26 07:43:36,194 epoch 80 - iter 35/50 - loss 2.01605985 - samples/sec: 88.31 - lr: 0.000488 +2021-03-26 07:43:38,146 epoch 80 - iter 40/50 - loss 1.98962872 - samples/sec: 82.07 - lr: 0.000488 +2021-03-26 07:43:40,176 epoch 80 - iter 45/50 - loss 2.01850783 - samples/sec: 78.90 - lr: 0.000488 +2021-03-26 07:43:41,884 epoch 80 - iter 50/50 - loss 2.00040341 - samples/sec: 93.77 - lr: 0.000488 +2021-03-26 07:43:41,884 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:43:41,884 EPOCH 80 done: loss 2.0004 - lr 0.0004883 +2021-03-26 07:43:42,657 DEV : loss 6.960967063903809 - score 0.9034 +2021-03-26 07:43:42,680 BAD EPOCHS (no improvement): 1 +2021-03-26 07:43:42,681 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:43:44,534 epoch 81 - iter 5/50 - loss 1.82899351 - samples/sec: 86.40 - lr: 0.000488 +2021-03-26 07:43:46,503 epoch 81 - iter 10/50 - loss 2.06492004 - samples/sec: 81.36 - lr: 0.000488 +2021-03-26 07:43:48,577 epoch 81 - iter 15/50 - loss 2.03007421 - samples/sec: 77.20 - lr: 0.000488 +2021-03-26 07:43:50,461 epoch 81 - iter 20/50 - loss 2.05423158 - samples/sec: 84.97 - lr: 0.000488 +2021-03-26 07:43:52,368 epoch 81 - iter 25/50 - loss 2.15495116 - samples/sec: 84.01 - lr: 0.000488 +2021-03-26 07:43:54,207 epoch 81 - iter 30/50 - loss 2.11696693 - samples/sec: 87.07 - lr: 0.000488 +2021-03-26 07:43:56,144 epoch 81 - iter 35/50 - loss 2.10432662 - samples/sec: 82.70 - lr: 0.000488 +2021-03-26 07:43:58,144 epoch 81 - iter 40/50 - loss 2.13423683 - samples/sec: 80.08 - lr: 0.000488 +2021-03-26 07:43:59,931 epoch 81 - iter 45/50 - loss 2.13024547 - samples/sec: 89.61 - lr: 0.000488 +2021-03-26 07:44:01,891 epoch 81 - iter 50/50 - loss 2.16672405 - samples/sec: 81.71 - lr: 0.000488 +2021-03-26 07:44:01,891 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:44:01,892 EPOCH 81 done: loss 2.1667 - lr 0.0004883 +2021-03-26 07:44:02,680 DEV : loss 6.962381362915039 - score 0.903 +2021-03-26 07:44:02,696 BAD EPOCHS (no improvement): 2 +2021-03-26 07:44:02,697 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:44:04,854 epoch 82 - iter 5/50 - loss 2.00777359 - samples/sec: 74.25 - lr: 0.000488 +2021-03-26 07:44:06,801 epoch 82 - iter 10/50 - loss 2.22989006 - samples/sec: 82.27 - lr: 0.000488 +2021-03-26 07:44:08,627 epoch 82 - iter 15/50 - loss 2.09961801 - samples/sec: 87.68 - lr: 0.000488 +2021-03-26 07:44:10,477 epoch 82 - iter 20/50 - loss 2.15289982 - samples/sec: 86.59 - lr: 0.000488 +2021-03-26 07:44:12,330 epoch 82 - iter 25/50 - loss 2.18809028 - samples/sec: 86.45 - lr: 0.000488 +2021-03-26 07:44:14,319 epoch 82 - iter 30/50 - loss 2.18357460 - samples/sec: 80.52 - lr: 0.000488 +2021-03-26 07:44:16,347 epoch 82 - iter 35/50 - loss 2.13511537 - samples/sec: 78.97 - lr: 0.000488 +2021-03-26 07:44:18,339 epoch 82 - iter 40/50 - loss 2.12040598 - samples/sec: 80.40 - lr: 0.000488 +2021-03-26 07:44:20,167 epoch 82 - iter 45/50 - loss 2.11619196 - samples/sec: 87.63 - lr: 0.000488 +2021-03-26 07:44:22,028 epoch 82 - iter 50/50 - loss 2.11520464 - samples/sec: 86.02 - lr: 0.000488 +2021-03-26 07:44:22,029 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:44:22,029 EPOCH 82 done: loss 2.1152 - lr 0.0004883 +2021-03-26 07:44:22,828 DEV : loss 6.961461544036865 - score 0.9034 +2021-03-26 07:44:22,851 BAD EPOCHS (no improvement): 3 +2021-03-26 07:44:22,852 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:44:24,831 epoch 83 - iter 5/50 - loss 2.41702788 - samples/sec: 80.95 - lr: 0.000488 +2021-03-26 07:44:26,699 epoch 83 - iter 10/50 - loss 2.16960576 - samples/sec: 85.74 - lr: 0.000488 +2021-03-26 07:44:28,456 epoch 83 - iter 15/50 - loss 2.26841859 - samples/sec: 91.15 - lr: 0.000488 +2021-03-26 07:44:30,467 epoch 83 - iter 20/50 - loss 2.25694011 - samples/sec: 79.64 - lr: 0.000488 +2021-03-26 07:44:32,382 epoch 83 - iter 25/50 - loss 2.28131059 - samples/sec: 83.64 - lr: 0.000488 +2021-03-26 07:44:34,348 epoch 83 - iter 30/50 - loss 2.28328307 - samples/sec: 81.42 - lr: 0.000488 +2021-03-26 07:44:36,313 epoch 83 - iter 35/50 - loss 2.26969606 - samples/sec: 81.49 - lr: 0.000488 +2021-03-26 07:44:38,324 epoch 83 - iter 40/50 - loss 2.24495017 - samples/sec: 79.65 - lr: 0.000488 +2021-03-26 07:44:40,213 epoch 83 - iter 45/50 - loss 2.25571009 - samples/sec: 84.78 - lr: 0.000488 +2021-03-26 07:44:41,904 epoch 83 - iter 50/50 - loss 2.26575992 - samples/sec: 94.67 - lr: 0.000488 +2021-03-26 07:44:41,905 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:44:41,905 EPOCH 83 done: loss 2.2658 - lr 0.0004883 +2021-03-26 07:44:42,684 DEV : loss 6.96260404586792 - score 0.9034 +2021-03-26 07:44:42,703 BAD EPOCHS (no improvement): 4 +2021-03-26 07:44:42,704 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:44:44,763 epoch 84 - iter 5/50 - loss 2.29062424 - samples/sec: 77.77 - lr: 0.000244 +2021-03-26 07:44:46,685 epoch 84 - iter 10/50 - loss 2.16970339 - samples/sec: 83.34 - lr: 0.000244 +2021-03-26 07:44:48,469 epoch 84 - iter 15/50 - loss 2.22968817 - samples/sec: 89.76 - lr: 0.000244 +2021-03-26 07:44:50,497 epoch 84 - iter 20/50 - loss 2.26354269 - samples/sec: 78.95 - lr: 0.000244 +2021-03-26 07:44:52,408 epoch 84 - iter 25/50 - loss 2.23960634 - samples/sec: 83.81 - lr: 0.000244 +2021-03-26 07:44:54,257 epoch 84 - iter 30/50 - loss 2.20255948 - samples/sec: 86.63 - lr: 0.000244 +2021-03-26 07:44:56,223 epoch 84 - iter 35/50 - loss 2.21711638 - samples/sec: 81.44 - lr: 0.000244 +2021-03-26 07:44:58,059 epoch 84 - iter 40/50 - loss 2.22642836 - samples/sec: 87.22 - lr: 0.000244 +2021-03-26 07:44:59,900 epoch 84 - iter 45/50 - loss 2.24058124 - samples/sec: 87.03 - lr: 0.000244 +2021-03-26 07:45:01,711 epoch 84 - iter 50/50 - loss 2.21161734 - samples/sec: 88.39 - lr: 0.000244 +2021-03-26 07:45:01,712 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:45:01,712 EPOCH 84 done: loss 2.2116 - lr 0.0002441 +2021-03-26 07:45:02,459 DEV : loss 6.962458610534668 - score 0.903 +2021-03-26 07:45:02,481 BAD EPOCHS (no improvement): 1 +2021-03-26 07:45:02,482 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:45:04,349 epoch 85 - iter 5/50 - loss 1.92033653 - samples/sec: 85.77 - lr: 0.000244 +2021-03-26 07:45:06,066 epoch 85 - iter 10/50 - loss 1.91954648 - samples/sec: 93.28 - lr: 0.000244 +2021-03-26 07:45:08,037 epoch 85 - iter 15/50 - loss 2.06335115 - samples/sec: 81.21 - lr: 0.000244 +2021-03-26 07:45:09,907 epoch 85 - iter 20/50 - loss 2.08999578 - samples/sec: 85.66 - lr: 0.000244 +2021-03-26 07:45:11,884 epoch 85 - iter 25/50 - loss 2.09058705 - samples/sec: 80.99 - lr: 0.000244 +2021-03-26 07:45:13,731 epoch 85 - iter 30/50 - loss 2.07775679 - samples/sec: 86.72 - lr: 0.000244 +2021-03-26 07:45:15,624 epoch 85 - iter 35/50 - loss 2.07566880 - samples/sec: 84.56 - lr: 0.000244 +2021-03-26 07:45:17,543 epoch 85 - iter 40/50 - loss 2.07374095 - samples/sec: 83.46 - lr: 0.000244 +2021-03-26 07:45:19,480 epoch 85 - iter 45/50 - loss 2.06040113 - samples/sec: 82.71 - lr: 0.000244 +2021-03-26 07:45:21,332 epoch 85 - iter 50/50 - loss 2.11410238 - samples/sec: 86.45 - lr: 0.000244 +2021-03-26 07:45:21,333 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:45:21,333 EPOCH 85 done: loss 2.1141 - lr 0.0002441 +2021-03-26 07:45:22,084 DEV : loss 6.961507320404053 - score 0.903 +2021-03-26 07:45:22,103 BAD EPOCHS (no improvement): 2 +2021-03-26 07:45:22,104 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:45:24,176 epoch 86 - iter 5/50 - loss 2.08228850 - samples/sec: 77.30 - lr: 0.000244 +2021-03-26 07:45:26,138 epoch 86 - iter 10/50 - loss 2.17302984 - samples/sec: 81.63 - lr: 0.000244 +2021-03-26 07:45:28,015 epoch 86 - iter 15/50 - loss 2.14811590 - samples/sec: 85.32 - lr: 0.000244 +2021-03-26 07:45:29,892 epoch 86 - iter 20/50 - loss 2.11968042 - samples/sec: 85.31 - lr: 0.000244 +2021-03-26 07:45:31,674 epoch 86 - iter 25/50 - loss 2.03037696 - samples/sec: 89.86 - lr: 0.000244 +2021-03-26 07:45:33,639 epoch 86 - iter 30/50 - loss 2.10106587 - samples/sec: 81.51 - lr: 0.000244 +2021-03-26 07:45:35,595 epoch 86 - iter 35/50 - loss 2.07236118 - samples/sec: 81.88 - lr: 0.000244 +2021-03-26 07:45:37,637 epoch 86 - iter 40/50 - loss 2.11832686 - samples/sec: 78.44 - lr: 0.000244 +2021-03-26 07:45:39,532 epoch 86 - iter 45/50 - loss 2.13117549 - samples/sec: 84.53 - lr: 0.000244 +2021-03-26 07:45:41,400 epoch 86 - iter 50/50 - loss 2.14113355 - samples/sec: 85.74 - lr: 0.000244 +2021-03-26 07:45:41,400 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:45:41,401 EPOCH 86 done: loss 2.1411 - lr 0.0002441 +2021-03-26 07:45:42,164 DEV : loss 6.961318016052246 - score 0.903 +2021-03-26 07:45:42,184 BAD EPOCHS (no improvement): 3 +2021-03-26 07:45:42,184 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:45:44,109 epoch 87 - iter 5/50 - loss 2.50831332 - samples/sec: 83.19 - lr: 0.000244 +2021-03-26 07:45:45,999 epoch 87 - iter 10/50 - loss 2.19329880 - samples/sec: 84.75 - lr: 0.000244 +2021-03-26 07:45:47,854 epoch 87 - iter 15/50 - loss 2.13173851 - samples/sec: 86.31 - lr: 0.000244 +2021-03-26 07:45:49,763 epoch 87 - iter 20/50 - loss 2.13360924 - samples/sec: 83.88 - lr: 0.000244 +2021-03-26 07:45:51,596 epoch 87 - iter 25/50 - loss 2.10489856 - samples/sec: 87.39 - lr: 0.000244 +2021-03-26 07:45:53,621 epoch 87 - iter 30/50 - loss 2.07814337 - samples/sec: 79.06 - lr: 0.000244 +2021-03-26 07:45:55,520 epoch 87 - iter 35/50 - loss 2.11120173 - samples/sec: 84.37 - lr: 0.000244 +2021-03-26 07:45:57,411 epoch 87 - iter 40/50 - loss 2.06504587 - samples/sec: 84.68 - lr: 0.000244 +2021-03-26 07:45:59,961 epoch 87 - iter 45/50 - loss 2.06413985 - samples/sec: 62.79 - lr: 0.000244 +2021-03-26 07:46:01,714 epoch 87 - iter 50/50 - loss 2.10936777 - samples/sec: 91.35 - lr: 0.000244 +2021-03-26 07:46:01,715 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:46:01,716 EPOCH 87 done: loss 2.1094 - lr 0.0002441 +2021-03-26 07:46:02,459 DEV : loss 6.961304664611816 - score 0.903 +2021-03-26 07:46:02,482 BAD EPOCHS (no improvement): 4 +2021-03-26 07:46:02,482 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:46:04,294 epoch 88 - iter 5/50 - loss 1.52949798 - samples/sec: 88.37 - lr: 0.000122 +2021-03-26 07:46:06,213 epoch 88 - iter 10/50 - loss 1.76798514 - samples/sec: 83.47 - lr: 0.000122 +2021-03-26 07:46:08,009 epoch 88 - iter 15/50 - loss 1.96657056 - samples/sec: 89.19 - lr: 0.000122 +2021-03-26 07:46:09,863 epoch 88 - iter 20/50 - loss 2.16982436 - samples/sec: 86.35 - lr: 0.000122 +2021-03-26 07:46:11,814 epoch 88 - iter 25/50 - loss 2.20400687 - samples/sec: 82.12 - lr: 0.000122 +2021-03-26 07:46:13,717 epoch 88 - iter 30/50 - loss 2.09522241 - samples/sec: 84.12 - lr: 0.000122 +2021-03-26 07:46:15,620 epoch 88 - iter 35/50 - loss 2.09541041 - samples/sec: 84.14 - lr: 0.000122 +2021-03-26 07:46:17,503 epoch 88 - iter 40/50 - loss 2.10176090 - samples/sec: 85.10 - lr: 0.000122 +2021-03-26 07:46:19,396 epoch 88 - iter 45/50 - loss 2.09184035 - samples/sec: 84.58 - lr: 0.000122 +2021-03-26 07:46:21,335 epoch 88 - iter 50/50 - loss 2.08811824 - samples/sec: 82.60 - lr: 0.000122 +2021-03-26 07:46:21,336 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:46:21,336 EPOCH 88 done: loss 2.0881 - lr 0.0001221 +2021-03-26 07:46:22,099 DEV : loss 6.961170196533203 - score 0.903 +2021-03-26 07:46:22,123 BAD EPOCHS (no improvement): 1 +2021-03-26 07:46:22,124 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:46:24,152 epoch 89 - iter 5/50 - loss 1.76743813 - samples/sec: 78.98 - lr: 0.000122 +2021-03-26 07:46:26,045 epoch 89 - iter 10/50 - loss 1.99386131 - samples/sec: 84.64 - lr: 0.000122 +2021-03-26 07:46:27,924 epoch 89 - iter 15/50 - loss 1.92271976 - samples/sec: 85.26 - lr: 0.000122 +2021-03-26 07:46:29,738 epoch 89 - iter 20/50 - loss 2.00147249 - samples/sec: 88.33 - lr: 0.000122 +2021-03-26 07:46:31,611 epoch 89 - iter 25/50 - loss 2.12277925 - samples/sec: 85.50 - lr: 0.000122 +2021-03-26 07:46:33,492 epoch 89 - iter 30/50 - loss 2.13457530 - samples/sec: 85.14 - lr: 0.000122 +2021-03-26 07:46:35,399 epoch 89 - iter 35/50 - loss 2.16680305 - samples/sec: 83.96 - lr: 0.000122 +2021-03-26 07:46:37,298 epoch 89 - iter 40/50 - loss 2.15806007 - samples/sec: 84.36 - lr: 0.000122 +2021-03-26 07:46:39,166 epoch 89 - iter 45/50 - loss 2.18511029 - samples/sec: 85.70 - lr: 0.000122 +2021-03-26 07:46:40,945 epoch 89 - iter 50/50 - loss 2.26934085 - samples/sec: 90.05 - lr: 0.000122 +2021-03-26 07:46:40,946 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:46:40,946 EPOCH 89 done: loss 2.2693 - lr 0.0001221 +2021-03-26 07:46:41,712 DEV : loss 6.960937023162842 - score 0.903 +2021-03-26 07:46:41,739 BAD EPOCHS (no improvement): 2 +2021-03-26 07:46:41,740 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:46:43,582 epoch 90 - iter 5/50 - loss 2.45033231 - samples/sec: 86.95 - lr: 0.000122 +2021-03-26 07:46:45,564 epoch 90 - iter 10/50 - loss 2.30596037 - samples/sec: 80.79 - lr: 0.000122 +2021-03-26 07:46:47,486 epoch 90 - iter 15/50 - loss 2.30664878 - samples/sec: 83.36 - lr: 0.000122 +2021-03-26 07:46:49,472 epoch 90 - iter 20/50 - loss 2.22472465 - samples/sec: 80.61 - lr: 0.000122 +2021-03-26 07:46:51,344 epoch 90 - iter 25/50 - loss 2.25325889 - samples/sec: 85.55 - lr: 0.000122 +2021-03-26 07:46:53,309 epoch 90 - iter 30/50 - loss 2.24244933 - samples/sec: 81.52 - lr: 0.000122 +2021-03-26 07:46:55,212 epoch 90 - iter 35/50 - loss 2.22077815 - samples/sec: 84.16 - lr: 0.000122 +2021-03-26 07:46:56,960 epoch 90 - iter 40/50 - loss 2.15720444 - samples/sec: 91.59 - lr: 0.000122 +2021-03-26 07:46:58,769 epoch 90 - iter 45/50 - loss 2.16676752 - samples/sec: 88.53 - lr: 0.000122 +2021-03-26 07:47:00,485 epoch 90 - iter 50/50 - loss 2.14653236 - samples/sec: 93.33 - lr: 0.000122 +2021-03-26 07:47:00,486 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:47:00,486 EPOCH 90 done: loss 2.1465 - lr 0.0001221 +2021-03-26 07:47:01,218 DEV : loss 6.960792064666748 - score 0.903 +2021-03-26 07:47:01,241 BAD EPOCHS (no improvement): 3 +2021-03-26 07:47:01,242 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:47:03,074 epoch 91 - iter 5/50 - loss 2.31844633 - samples/sec: 87.42 - lr: 0.000122 +2021-03-26 07:47:04,983 epoch 91 - iter 10/50 - loss 2.14653064 - samples/sec: 83.88 - lr: 0.000122 +2021-03-26 07:47:06,843 epoch 91 - iter 15/50 - loss 2.03093856 - samples/sec: 86.09 - lr: 0.000122 +2021-03-26 07:47:08,666 epoch 91 - iter 20/50 - loss 2.11692855 - samples/sec: 87.88 - lr: 0.000122 +2021-03-26 07:47:10,518 epoch 91 - iter 25/50 - loss 2.11030499 - samples/sec: 86.44 - lr: 0.000122 +2021-03-26 07:47:12,443 epoch 91 - iter 30/50 - loss 2.12978576 - samples/sec: 83.20 - lr: 0.000122 +2021-03-26 07:47:14,267 epoch 91 - iter 35/50 - loss 2.13929472 - samples/sec: 87.82 - lr: 0.000122 +2021-03-26 07:47:16,110 epoch 91 - iter 40/50 - loss 2.13259790 - samples/sec: 86.85 - lr: 0.000122 +2021-03-26 07:47:18,008 epoch 91 - iter 45/50 - loss 2.16532231 - samples/sec: 84.43 - lr: 0.000122 +2021-03-26 07:47:19,685 epoch 91 - iter 50/50 - loss 2.11413988 - samples/sec: 95.48 - lr: 0.000122 +2021-03-26 07:47:19,685 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:47:19,686 EPOCH 91 done: loss 2.1141 - lr 0.0001221 +2021-03-26 07:47:20,415 DEV : loss 6.9608001708984375 - score 0.903 +2021-03-26 07:47:20,438 BAD EPOCHS (no improvement): 4 +2021-03-26 07:47:20,439 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:47:20,439 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:47:20,439 learning rate too small - quitting training! +2021-03-26 07:47:20,440 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:47:29,508 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:47:29,509 Testing using best model ... +2021-03-26 07:47:29,510 loading file /home/tmp/megahedm/models/multipos/multipos_UDMADAR_4Diale-LEV_EGY_GLF_MGR__fasttext_flairbwfw__32__0.5_202103260713/best-model.pt +2021-03-26 07:47:36,788 0.9055 +2021-03-26 07:47:36,788 +Results: +- F-score (micro): 0.9012 +- F-score (macro): 0.4811 +- Accuracy (incl. no class): 0.9055 + +By class: + precision recall f1-score support + + CCONJ 0.9865 0.9865 0.9865 74 + PRON 0.9725 0.9833 0.9779 180 + PART 0.9617 0.9757 0.9687 206 + VERB 0.9549 0.8819 0.9170 144 + ADJ 0.8947 0.7589 0.8213 112 + NOUN 0.8897 0.9322 0.9104 398 + ADP 0.9263 0.9565 0.9412 92 + DET 0.9821 0.9649 0.9735 57 + SCONJ 1.0000 0.9697 0.9846 33 + ADV 0.9074 0.8829 0.8950 111 + AUX 0.8387 0.9286 0.8814 28 + PROPN 1.0000 1.0000 1.0000 10 + NUM 0.9333 0.8750 0.9032 16 + INTJ 1.0000 1.0000 1.0000 17 + PUNCT 1.0000 1.0000 1.0000 20 + V 0.9651 0.8300 0.8925 100 + NOUN+PRON 0.7975 0.8873 0.8400 71 + PREP+DET+ADJ 1.0000 0.0000 0.0000 1 + PREP+DET+NOUN 0.5385 0.8750 0.6667 8 + PROG_PART 1.0000 0.0000 0.0000 1 + PREP+PRON 0.8056 0.9667 0.8788 30 + PRON+DET+NOUN 0.5000 1.0000 0.6667 1 + HASH 1.0000 0.9524 0.9756 21 + EOS 1.0000 1.0000 1.0000 70 + V+PRON+PRON 0.6000 0.3000 0.4000 10 + NOUN+NSUFF 0.8261 0.8837 0.8539 43 + PREP+NOUN+PRON 0.6667 0.4000 0.5000 5 + PUNC 1.0000 1.0000 1.0000 127 + V+PRON 0.7164 0.8727 0.7869 55 + V+PRON+PREP+PRON 0.0000 0.0000 0.0000 4 + V+PREP+PRON 0.2000 0.2500 0.2222 4 + DET+NOUN+NSUFF 0.9310 0.8710 0.9000 31 + NOUN+NSUFF+PRON 0.9375 0.7143 0.8108 21 + CONJ 0.9667 1.0000 0.9831 29 + FOREIGN 1.0000 0.0000 0.0000 3 + MENTION 0.9412 1.0000 0.9697 16 + DET+NOUN 0.9615 0.9804 0.9709 51 + PROG_PART+V 0.9000 0.9730 0.9351 37 + PREP+NOUN+PRON+PRON 1.0000 0.0000 0.0000 1 + PREP+NOUN+NSUFF+PRON 0.0000 1.0000 0.0000 0 + PART+PRON 1.0000 0.8636 0.9268 22 + PREP+V+PRON 1.0000 0.0000 0.0000 1 + CONJ+V+PRON 0.5556 0.8333 0.6667 6 + PREP 0.9464 0.9815 0.9636 54 + ADJ+NSUFF 0.7632 0.9667 0.8529 30 + PREP+V 0.5000 0.5000 0.5000 2 + PREP+NOUN 0.8947 0.8500 0.8718 20 + PREP+NOUN+NSUFF 1.0000 0.7500 0.8571 4 + PROG_PART+V+PRON 0.7143 1.0000 0.8333 10 + DET+ADJ+NSUFF 0.5000 0.5000 0.5000 4 + CONJ+DET+NOUN 1.0000 1.0000 1.0000 2 + FUT_PART+V+PRON 1.0000 0.3333 0.5000 6 + CONJ+V 0.8889 0.8000 0.8421 10 + CONJ+PROG_PART+V 0.5000 1.0000 0.6667 2 + NOUN+NSUFF+NSUFF 1.0000 0.0000 0.0000 1 + CONJ+NOUN+PRON 0.0000 0.0000 0.0000 2 + CONJ+DET+NOUN+NSUFF 0.0000 1.0000 0.0000 0 + CONJ+PREP+DET+NOUN 0.0000 1.0000 0.0000 0 + ADJ+PRON 0.2500 0.2500 0.2500 4 + CONJ+PRON 1.0000 1.0000 1.0000 9 + URL 1.0000 1.0000 1.0000 3 + ADJ+PREP+PRON 1.0000 0.0000 0.0000 5 + PART+NOUN 0.7500 1.0000 0.8571 3 + CONJ+NOUN 0.8333 0.7143 0.7692 7 + PREP+ADJ+NSUFF 0.0000 1.0000 0.0000 0 + EMOT 1.0000 1.0000 1.0000 15 + PART+NOUN+PRON 0.0000 1.0000 0.0000 0 + CONJ+PREP 1.0000 1.0000 1.0000 1 + PREP+PRON+DET+NOUN 1.0000 0.0000 0.0000 1 +PREP+DET+NOUN+NSUFF+PREP+PRON 0.0000 1.0000 0.0000 0 + DET+ADJ 0.5000 1.0000 0.6667 2 + PREP+DET+NOUN+NSUFF 0.7500 0.7500 0.7500 4 + PREP+DET+ADV 1.0000 0.0000 0.0000 1 + CONJ+NOUN+NSUFF 0.7500 1.0000 0.8571 3 + CONJ+PART+V+NOUN 1.0000 0.0000 0.0000 1 + CONJ+ADJ 1.0000 0.0000 0.0000 2 + V+PRON+PRON+NEG_PART 1.0000 0.0000 0.0000 1 + PART+V+PRON+NEG_PART 0.5556 0.7143 0.6250 7 + FUT_PART+V 0.9000 0.9000 0.9000 10 + CONJ+PART 1.0000 1.0000 1.0000 5 + PART+V+NEG_PART 1.0000 1.0000 1.0000 3 + FUT_PART 1.0000 1.0000 1.0000 1 + PART+V 0.0000 0.0000 0.0000 1 + CONJ+V+PREP+PRON 0.0000 1.0000 0.0000 0 + NOUN+PREP+PRON 1.0000 0.0000 0.0000 1 + PREP+PART 1.0000 1.0000 1.0000 3 + CONJ+PART+V+PRON 1.0000 0.0000 0.0000 2 + PRON+DET+NOUN+NSUFF 1.0000 0.0000 0.0000 1 + PREP+PART+PRON 1.0000 1.0000 1.0000 1 + V+NOUN 1.0000 0.0000 0.0000 1 + CONJ+PART+PROG_PART+V 1.0000 0.0000 0.0000 1 + NOUN+CASE 0.6000 0.7500 0.6667 4 + CONJ+PART+V+NEG_PART 1.0000 0.0000 0.0000 2 + CONJ+FUT_PART+V 0.0000 0.0000 0.0000 1 + PART+PROG_PART+V+NEG_PART 0.0000 0.0000 0.0000 1 + PART+PREP+NEG_PART 1.0000 1.0000 1.0000 1 + PREP+DET 1.0000 0.0000 0.0000 1 + PROG_PART+V+PREP+PRON 1.0000 0.0000 0.0000 1 + FUT_PART+V+PREP+PRON 0.0000 0.0000 0.0000 1 + PART+NOUN+NEG_PART 1.0000 1.0000 1.0000 1 + PART+V+PRON 1.0000 0.0000 0.0000 1 + PROG_PART+V+PRON+PRON 1.0000 0.0000 0.0000 1 + PART+PREP+PRON+NEG_PART 0.0000 1.0000 0.0000 0 + DET+NUM 1.0000 0.0000 0.0000 1 + PART+NSUFF 1.0000 0.0000 0.0000 1 + ADJ+CASE 0.0000 1.0000 0.0000 0 + FUT_PART+V+PRON+PRON 0.0000 1.0000 0.0000 0 + V+NEG_PART 0.0000 0.0000 0.0000 2 + PART+NOUN+PRON+NEG_PART 1.0000 0.0000 0.0000 1 + NOUN+PRON+NEG_PART 0.0000 1.0000 0.0000 0 + NOUN+CASE+PRON 1.0000 0.0000 0.0000 1 + CONJ+PROG_PART+V+PRON+PRON 1.0000 0.0000 0.0000 1 + CONJ+PROG_PART+V+PRON 0.0000 1.0000 0.0000 0 + ADV+NSUFF 1.0000 1.0000 1.0000 1 + PART+V+PRON+PRON+NEG_PART 1.0000 0.0000 0.0000 1 + PART+V+PREP+PRON+NEG_PART 0.0000 0.0000 0.0000 1 + PART+PREP+PRON 1.0000 0.0000 0.0000 1 + CONJ+ADV+NSUFF 1.0000 0.0000 0.0000 1 + CONJ+FUT_PART+V+PREP+PRON 0.0000 1.0000 0.0000 0 + NUM+CASE 1.0000 0.0000 0.0000 2 + + micro avg 0.9016 0.9009 0.9012 2542 + macro avg 0.7388 0.6026 0.4811 2542 + weighted avg 0.9136 0.9009 0.8966 2542 + +2021-03-26 07:47:36,789 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:47:36,789 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:47:42,946 Reading data from ../../Datasets_adhoc/CSCS_corpus-GUC +2021-03-26 07:47:42,947 Train: ../../Datasets_adhoc/CSCS_corpus-GUC/all_participants.conllu +2021-03-26 07:47:42,947 Dev: None +2021-03-26 07:47:42,947 Test: None +2021-03-26 07:47:43,237 Reading data from ../../Datasets_adhoc/UD_MADAR +2021-03-26 07:47:43,237 Train: ../../Datasets_adhoc/UD_MADAR/ajp_madar-ud-test-edit.conllu +2021-03-26 07:47:43,238 Dev: None +2021-03-26 07:47:43,238 Test: None +2021-03-26 07:47:43,282 Reading data from ../../Datasets_adhoc/dialectal_arabic_resources-master +2021-03-26 07:47:43,282 Train: ../../Datasets_adhoc/dialectal_arabic_resources-master/seg_plus_pos_lev.txt +2021-03-26 07:47:43,282 Dev: None +2021-03-26 07:47:43,282 Test: None +2021-03-26 07:47:43,434 Reading data from ../../Datasets_adhoc/dialectal_arabic_resources-master +2021-03-26 07:47:43,435 Train: ../../Datasets_adhoc/dialectal_arabic_resources-master/seg_plus_pos_egy.txt +2021-03-26 07:47:43,435 Dev: None +2021-03-26 07:47:43,436 Test: None +2021-03-26 07:47:43,590 Reading data from ../../Datasets_adhoc/dialectal_arabic_resources-master +2021-03-26 07:47:43,591 Train: ../../Datasets_adhoc/dialectal_arabic_resources-master/seg_plus_pos_glf.txt +2021-03-26 07:47:43,591 Dev: None +2021-03-26 07:47:43,591 Test: None +2021-03-26 07:47:43,758 Reading data from ../../Datasets_adhoc/dialectal_arabic_resources-master +2021-03-26 07:47:43,759 Train: ../../Datasets_adhoc/dialectal_arabic_resources-master/seg_plus_pos_mgr.txt +2021-03-26 07:47:43,759 Dev: None +2021-03-26 07:47:43,760 Test: None +2021-03-26 07:47:43,923 Filtering long sentences +2021-03-26 07:47:43,962 MultiCorpus: 1573 train + 176 dev + 195 test sentences + - ColumnCorpus Corpus: 934 train + 104 dev + 115 test sentences + - ColumnCorpus Corpus: 81 train + 9 dev + 10 test sentences + - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences + - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences +2021-03-26 07:47:44,365 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:47:44,366 Model: "SequenceTagger( + (embeddings): StackedEmbeddings( + (list_embedding_0): WordEmbeddings('ar') + (list_embedding_1): FlairEmbeddings( + (lm): LanguageModel( + (drop): Dropout(p=0.1, inplace=False) + (encoder): Embedding(7125, 100) + (rnn): LSTM(100, 2048) + (decoder): Linear(in_features=2048, out_features=7125, bias=True) + ) + ) + (list_embedding_2): FlairEmbeddings( + (lm): LanguageModel( + (drop): Dropout(p=0.1, inplace=False) + (encoder): Embedding(7125, 100) + (rnn): LSTM(100, 2048) + (decoder): Linear(in_features=2048, out_features=7125, bias=True) + ) + ) + ) + (word_dropout): WordDropout(p=0.05) + (locked_dropout): LockedDropout(p=0.5) + (embedding2nn): Linear(in_features=4396, out_features=4396, bias=True) + (rnn): LSTM(4396, 256, batch_first=True, bidirectional=True) + (linear): Linear(in_features=512, out_features=206, bias=True) + (beta): 1.0 + (weights): None + (weight_tensor) None +)" +2021-03-26 07:47:44,367 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:47:44,367 Corpus: "MultiCorpus: 1573 train + 176 dev + 195 test sentences + - ColumnCorpus Corpus: 934 train + 104 dev + 115 test sentences + - ColumnCorpus Corpus: 81 train + 9 dev + 10 test sentences + - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences + - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences" +2021-03-26 07:47:44,368 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:47:44,368 Parameters: +2021-03-26 07:47:44,368 - learning_rate: "0.5" +2021-03-26 07:47:44,369 - mini_batch_size: "64" +2021-03-26 07:47:44,369 - patience: "3" +2021-03-26 07:47:44,370 - anneal_factor: "0.5" +2021-03-26 07:47:44,370 - max_epochs: "150" +2021-03-26 07:47:44,370 - shuffle: "True" +2021-03-26 07:47:44,371 - train_with_dev: "False" +2021-03-26 07:47:44,371 - batch_growth_annealing: "False" +2021-03-26 07:47:44,371 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:47:44,372 Model training base path: "/home/tmp/megahedm/models/multipos/multipos_UDMADAR_4Diale-LEV_EGY_GLF_MGR__fasttext_flairbwfw__64__0.5_202103260747" +2021-03-26 07:47:44,372 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:47:44,373 Device: cuda:0 +2021-03-26 07:47:44,373 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:47:44,373 Embeddings storage mode: cpu +2021-03-26 07:47:44,375 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:47:46,063 epoch 1 - iter 2/25 - loss 82.29800034 - samples/sec: 75.91 - lr: 0.500000 +2021-03-26 07:47:47,596 epoch 1 - iter 4/25 - loss 86.63934135 - samples/sec: 83.61 - lr: 0.500000 +2021-03-26 07:47:48,959 epoch 1 - iter 6/25 - loss 79.94940567 - samples/sec: 93.96 - lr: 0.500000 +2021-03-26 07:47:50,361 epoch 1 - iter 8/25 - loss 77.86869812 - samples/sec: 91.37 - lr: 0.500000 +2021-03-26 07:47:51,702 epoch 1 - iter 10/25 - loss 74.69780655 - samples/sec: 95.57 - lr: 0.500000 +2021-03-26 07:47:52,924 epoch 1 - iter 12/25 - loss 72.09690285 - samples/sec: 104.85 - lr: 0.500000 +2021-03-26 07:47:54,260 epoch 1 - iter 14/25 - loss 70.73670796 - samples/sec: 95.90 - lr: 0.500000 +2021-03-26 07:47:55,589 epoch 1 - iter 16/25 - loss 68.22431612 - samples/sec: 96.47 - lr: 0.500000 +2021-03-26 07:47:57,001 epoch 1 - iter 18/25 - loss 66.37756750 - samples/sec: 90.72 - lr: 0.500000 +2021-03-26 07:47:58,261 epoch 1 - iter 20/25 - loss 64.80691071 - samples/sec: 101.65 - lr: 0.500000 +2021-03-26 07:47:59,516 epoch 1 - iter 22/25 - loss 63.18399325 - samples/sec: 102.11 - lr: 0.500000 +2021-03-26 07:48:00,785 epoch 1 - iter 24/25 - loss 62.12906313 - samples/sec: 101.03 - lr: 0.500000 +2021-03-26 07:48:01,328 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:48:01,328 EPOCH 1 done: loss 62.0460 - lr 0.5000000 +2021-03-26 07:48:02,503 DEV : loss 41.141868591308594 - score 0.3298 +2021-03-26 07:48:02,525 BAD EPOCHS (no improvement): 0 +2021-03-26 07:48:11,783 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:48:12,737 epoch 2 - iter 2/25 - loss 40.23099422 - samples/sec: 134.34 - lr: 0.500000 +2021-03-26 07:48:13,743 epoch 2 - iter 4/25 - loss 43.27464151 - samples/sec: 127.49 - lr: 0.500000 +2021-03-26 07:48:14,783 epoch 2 - iter 6/25 - loss 42.34753831 - samples/sec: 123.20 - lr: 0.500000 +2021-03-26 07:48:15,725 epoch 2 - iter 8/25 - loss 41.78717351 - samples/sec: 136.09 - lr: 0.500000 +2021-03-26 07:48:16,657 epoch 2 - iter 10/25 - loss 41.55979214 - samples/sec: 137.55 - lr: 0.500000 +2021-03-26 07:48:17,587 epoch 2 - iter 12/25 - loss 40.78030396 - samples/sec: 137.84 - lr: 0.500000 +2021-03-26 07:48:18,534 epoch 2 - iter 14/25 - loss 39.94518444 - samples/sec: 135.34 - lr: 0.500000 +2021-03-26 07:48:19,497 epoch 2 - iter 16/25 - loss 39.79153347 - samples/sec: 133.14 - lr: 0.500000 +2021-03-26 07:48:20,461 epoch 2 - iter 18/25 - loss 39.26791233 - samples/sec: 133.07 - lr: 0.500000 +2021-03-26 07:48:21,479 epoch 2 - iter 20/25 - loss 38.48154440 - samples/sec: 125.87 - lr: 0.500000 +2021-03-26 07:48:22,407 epoch 2 - iter 22/25 - loss 37.72989420 - samples/sec: 138.28 - lr: 0.500000 +2021-03-26 07:48:23,413 epoch 2 - iter 24/25 - loss 37.17000763 - samples/sec: 127.35 - lr: 0.500000 +2021-03-26 07:48:23,756 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:48:23,757 EPOCH 2 done: loss 36.6704 - lr 0.5000000 +2021-03-26 07:48:24,470 DEV : loss 27.52168846130371 - score 0.5767 +2021-03-26 07:48:24,488 BAD EPOCHS (no improvement): 0 +2021-03-26 07:48:33,906 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:48:34,930 epoch 3 - iter 2/25 - loss 31.42631340 - samples/sec: 125.23 - lr: 0.500000 +2021-03-26 07:48:35,829 epoch 3 - iter 4/25 - loss 30.74935913 - samples/sec: 142.67 - lr: 0.500000 +2021-03-26 07:48:36,887 epoch 3 - iter 6/25 - loss 30.67000516 - samples/sec: 121.16 - lr: 0.500000 +2021-03-26 07:48:37,905 epoch 3 - iter 8/25 - loss 30.57007957 - samples/sec: 125.87 - lr: 0.500000 +2021-03-26 07:48:38,826 epoch 3 - iter 10/25 - loss 29.24179325 - samples/sec: 139.08 - lr: 0.500000 +2021-03-26 07:48:39,726 epoch 3 - iter 12/25 - loss 28.71167231 - samples/sec: 142.44 - lr: 0.500000 +2021-03-26 07:48:40,635 epoch 3 - iter 14/25 - loss 28.50176103 - samples/sec: 141.10 - lr: 0.500000 +2021-03-26 07:48:41,535 epoch 3 - iter 16/25 - loss 27.73931015 - samples/sec: 142.40 - lr: 0.500000 +2021-03-26 07:48:42,535 epoch 3 - iter 18/25 - loss 27.32102193 - samples/sec: 128.23 - lr: 0.500000 +2021-03-26 07:48:43,648 epoch 3 - iter 20/25 - loss 26.90655651 - samples/sec: 115.15 - lr: 0.500000 +2021-03-26 07:48:44,618 epoch 3 - iter 22/25 - loss 26.83808604 - samples/sec: 132.17 - lr: 0.500000 +2021-03-26 07:48:45,525 epoch 3 - iter 24/25 - loss 26.56665818 - samples/sec: 141.39 - lr: 0.500000 +2021-03-26 07:48:46,009 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:48:46,009 EPOCH 3 done: loss 26.6577 - lr 0.5000000 +2021-03-26 07:48:46,736 DEV : loss 19.457054138183594 - score 0.6646 +2021-03-26 07:48:46,759 BAD EPOCHS (no improvement): 0 +2021-03-26 07:48:56,214 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:48:57,224 epoch 4 - iter 2/25 - loss 22.99335194 - samples/sec: 126.92 - lr: 0.500000 +2021-03-26 07:48:58,204 epoch 4 - iter 4/25 - loss 21.46911478 - samples/sec: 130.90 - lr: 0.500000 +2021-03-26 07:48:59,167 epoch 4 - iter 6/25 - loss 21.33844217 - samples/sec: 133.15 - lr: 0.500000 +2021-03-26 07:49:00,171 epoch 4 - iter 8/25 - loss 21.12010169 - samples/sec: 127.80 - lr: 0.500000 +2021-03-26 07:49:01,216 epoch 4 - iter 10/25 - loss 20.65613098 - samples/sec: 122.57 - lr: 0.500000 +2021-03-26 07:49:02,258 epoch 4 - iter 12/25 - loss 20.85910098 - samples/sec: 123.21 - lr: 0.500000 +2021-03-26 07:49:03,221 epoch 4 - iter 14/25 - loss 20.61655971 - samples/sec: 133.16 - lr: 0.500000 +2021-03-26 07:49:04,185 epoch 4 - iter 16/25 - loss 20.45642447 - samples/sec: 133.02 - lr: 0.500000 +2021-03-26 07:49:05,120 epoch 4 - iter 18/25 - loss 20.19972791 - samples/sec: 137.10 - lr: 0.500000 +2021-03-26 07:49:06,146 epoch 4 - iter 20/25 - loss 20.46301136 - samples/sec: 124.92 - lr: 0.500000 +2021-03-26 07:49:07,218 epoch 4 - iter 22/25 - loss 20.59231134 - samples/sec: 119.61 - lr: 0.500000 +2021-03-26 07:49:08,211 epoch 4 - iter 24/25 - loss 20.43321848 - samples/sec: 129.24 - lr: 0.500000 +2021-03-26 07:49:08,686 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:49:08,686 EPOCH 4 done: loss 20.3970 - lr 0.5000000 +2021-03-26 07:49:09,409 DEV : loss 14.515106201171875 - score 0.7436 +2021-03-26 07:49:09,432 BAD EPOCHS (no improvement): 0 +2021-03-26 07:49:18,910 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:49:19,938 epoch 5 - iter 2/25 - loss 16.30500841 - samples/sec: 124.87 - lr: 0.500000 +2021-03-26 07:49:21,073 epoch 5 - iter 4/25 - loss 16.66907477 - samples/sec: 113.01 - lr: 0.500000 +2021-03-26 07:49:22,070 epoch 5 - iter 6/25 - loss 16.89636453 - samples/sec: 128.51 - lr: 0.500000 +2021-03-26 07:49:23,008 epoch 5 - iter 8/25 - loss 17.29816079 - samples/sec: 136.71 - lr: 0.500000 +2021-03-26 07:49:24,077 epoch 5 - iter 10/25 - loss 17.58455524 - samples/sec: 120.01 - lr: 0.500000 +2021-03-26 07:49:25,115 epoch 5 - iter 12/25 - loss 17.42553425 - samples/sec: 123.42 - lr: 0.500000 +2021-03-26 07:49:26,059 epoch 5 - iter 14/25 - loss 17.41548838 - samples/sec: 135.86 - lr: 0.500000 +2021-03-26 07:49:27,146 epoch 5 - iter 16/25 - loss 17.00069827 - samples/sec: 117.90 - lr: 0.500000 +2021-03-26 07:49:28,079 epoch 5 - iter 18/25 - loss 17.08402385 - samples/sec: 137.47 - lr: 0.500000 +2021-03-26 07:49:29,011 epoch 5 - iter 20/25 - loss 17.05419016 - samples/sec: 137.58 - lr: 0.500000 +2021-03-26 07:49:30,011 epoch 5 - iter 22/25 - loss 16.79865685 - samples/sec: 128.23 - lr: 0.500000 +2021-03-26 07:49:30,955 epoch 5 - iter 24/25 - loss 16.75121248 - samples/sec: 135.78 - lr: 0.500000 +2021-03-26 07:49:31,389 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:49:31,389 EPOCH 5 done: loss 16.7338 - lr 0.5000000 +2021-03-26 07:49:32,101 DEV : loss 11.223102569580078 - score 0.7999 +2021-03-26 07:49:32,125 BAD EPOCHS (no improvement): 0 +2021-03-26 07:49:41,554 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:49:42,543 epoch 6 - iter 2/25 - loss 13.59014797 - samples/sec: 129.75 - lr: 0.500000 +2021-03-26 07:49:43,617 epoch 6 - iter 4/25 - loss 15.00557327 - samples/sec: 119.32 - lr: 0.500000 +2021-03-26 07:49:44,655 epoch 6 - iter 6/25 - loss 15.50979487 - samples/sec: 123.50 - lr: 0.500000 +2021-03-26 07:49:45,650 epoch 6 - iter 8/25 - loss 15.80515766 - samples/sec: 128.84 - lr: 0.500000 +2021-03-26 07:49:46,584 epoch 6 - iter 10/25 - loss 15.28509512 - samples/sec: 137.26 - lr: 0.500000 +2021-03-26 07:49:47,567 epoch 6 - iter 12/25 - loss 14.93954698 - samples/sec: 130.42 - lr: 0.500000 +2021-03-26 07:49:48,566 epoch 6 - iter 14/25 - loss 15.16099828 - samples/sec: 128.34 - lr: 0.500000 +2021-03-26 07:49:49,564 epoch 6 - iter 16/25 - loss 14.71998441 - samples/sec: 128.47 - lr: 0.500000 +2021-03-26 07:49:50,587 epoch 6 - iter 18/25 - loss 14.49083164 - samples/sec: 125.45 - lr: 0.500000 +2021-03-26 07:49:51,542 epoch 6 - iter 20/25 - loss 14.44817972 - samples/sec: 134.06 - lr: 0.500000 +2021-03-26 07:49:52,538 epoch 6 - iter 22/25 - loss 14.51069000 - samples/sec: 128.74 - lr: 0.500000 +2021-03-26 07:49:53,434 epoch 6 - iter 24/25 - loss 14.39883967 - samples/sec: 143.11 - lr: 0.500000 +2021-03-26 07:49:53,875 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:49:53,875 EPOCH 6 done: loss 14.4636 - lr 0.5000000 +2021-03-26 07:49:54,605 DEV : loss 9.920324325561523 - score 0.8191 +2021-03-26 07:49:54,628 BAD EPOCHS (no improvement): 0 +2021-03-26 07:50:04,174 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:50:05,175 epoch 7 - iter 2/25 - loss 13.02674007 - samples/sec: 128.18 - lr: 0.500000 +2021-03-26 07:50:06,133 epoch 7 - iter 4/25 - loss 12.42072916 - samples/sec: 133.74 - lr: 0.500000 +2021-03-26 07:50:07,111 epoch 7 - iter 6/25 - loss 13.04151154 - samples/sec: 131.06 - lr: 0.500000 +2021-03-26 07:50:08,098 epoch 7 - iter 8/25 - loss 13.00413275 - samples/sec: 130.06 - lr: 0.500000 +2021-03-26 07:50:09,096 epoch 7 - iter 10/25 - loss 13.29990978 - samples/sec: 128.32 - lr: 0.500000 +2021-03-26 07:50:10,077 epoch 7 - iter 12/25 - loss 13.34977380 - samples/sec: 130.78 - lr: 0.500000 +2021-03-26 07:50:10,983 epoch 7 - iter 14/25 - loss 13.37637084 - samples/sec: 141.41 - lr: 0.500000 +2021-03-26 07:50:11,973 epoch 7 - iter 16/25 - loss 13.16614377 - samples/sec: 129.41 - lr: 0.500000 +2021-03-26 07:50:12,880 epoch 7 - iter 18/25 - loss 13.04947027 - samples/sec: 141.42 - lr: 0.500000 +2021-03-26 07:50:13,845 epoch 7 - iter 20/25 - loss 13.18624725 - samples/sec: 132.82 - lr: 0.500000 +2021-03-26 07:50:14,782 epoch 7 - iter 22/25 - loss 13.21761140 - samples/sec: 136.79 - lr: 0.500000 +2021-03-26 07:50:15,775 epoch 7 - iter 24/25 - loss 13.07190700 - samples/sec: 129.11 - lr: 0.500000 +2021-03-26 07:50:16,101 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:50:16,102 EPOCH 7 done: loss 12.9792 - lr 0.5000000 +2021-03-26 07:50:16,808 DEV : loss 8.733400344848633 - score 0.8492 +2021-03-26 07:50:16,831 BAD EPOCHS (no improvement): 0 +2021-03-26 07:50:26,329 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:50:27,316 epoch 8 - iter 2/25 - loss 11.35763407 - samples/sec: 130.12 - lr: 0.500000 +2021-03-26 07:50:28,217 epoch 8 - iter 4/25 - loss 12.17101026 - samples/sec: 142.29 - lr: 0.500000 +2021-03-26 07:50:29,203 epoch 8 - iter 6/25 - loss 12.12305895 - samples/sec: 129.94 - lr: 0.500000 +2021-03-26 07:50:30,141 epoch 8 - iter 8/25 - loss 11.69760966 - samples/sec: 136.60 - lr: 0.500000 +2021-03-26 07:50:31,170 epoch 8 - iter 10/25 - loss 11.54711952 - samples/sec: 124.59 - lr: 0.500000 +2021-03-26 07:50:32,163 epoch 8 - iter 12/25 - loss 11.50161934 - samples/sec: 129.13 - lr: 0.500000 +2021-03-26 07:50:33,209 epoch 8 - iter 14/25 - loss 11.45421546 - samples/sec: 122.52 - lr: 0.500000 +2021-03-26 07:50:34,148 epoch 8 - iter 16/25 - loss 11.58866638 - samples/sec: 136.51 - lr: 0.500000 +2021-03-26 07:50:35,143 epoch 8 - iter 18/25 - loss 11.90951268 - samples/sec: 128.91 - lr: 0.500000 +2021-03-26 07:50:36,152 epoch 8 - iter 20/25 - loss 11.85078907 - samples/sec: 126.97 - lr: 0.500000 +2021-03-26 07:50:37,289 epoch 8 - iter 22/25 - loss 11.94904106 - samples/sec: 112.76 - lr: 0.500000 +2021-03-26 07:50:38,344 epoch 8 - iter 24/25 - loss 11.85964342 - samples/sec: 121.51 - lr: 0.500000 +2021-03-26 07:50:38,732 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:50:38,734 EPOCH 8 done: loss 11.8554 - lr 0.5000000 +2021-03-26 07:50:39,477 DEV : loss 8.442928314208984 - score 0.8514 +2021-03-26 07:50:39,492 BAD EPOCHS (no improvement): 0 +2021-03-26 07:50:49,122 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:50:50,141 epoch 9 - iter 2/25 - loss 9.26947498 - samples/sec: 125.97 - lr: 0.500000 +2021-03-26 07:50:51,119 epoch 9 - iter 4/25 - loss 9.99433255 - samples/sec: 131.15 - lr: 0.500000 +2021-03-26 07:50:52,141 epoch 9 - iter 6/25 - loss 10.41451152 - samples/sec: 125.36 - lr: 0.500000 +2021-03-26 07:50:53,079 epoch 9 - iter 8/25 - loss 10.95319188 - samples/sec: 136.71 - lr: 0.500000 +2021-03-26 07:50:54,140 epoch 9 - iter 10/25 - loss 11.07087307 - samples/sec: 120.88 - lr: 0.500000 +2021-03-26 07:50:55,277 epoch 9 - iter 12/25 - loss 11.12073040 - samples/sec: 112.75 - lr: 0.500000 +2021-03-26 07:50:56,251 epoch 9 - iter 14/25 - loss 10.87324211 - samples/sec: 131.64 - lr: 0.500000 +2021-03-26 07:50:57,192 epoch 9 - iter 16/25 - loss 10.93167228 - samples/sec: 136.26 - lr: 0.500000 +2021-03-26 07:50:58,211 epoch 9 - iter 18/25 - loss 10.78684590 - samples/sec: 125.85 - lr: 0.500000 +2021-03-26 07:50:59,305 epoch 9 - iter 20/25 - loss 10.82854271 - samples/sec: 117.21 - lr: 0.500000 +2021-03-26 07:51:00,267 epoch 9 - iter 22/25 - loss 10.73710918 - samples/sec: 133.28 - lr: 0.500000 +2021-03-26 07:51:01,216 epoch 9 - iter 24/25 - loss 10.62689821 - samples/sec: 135.07 - lr: 0.500000 +2021-03-26 07:51:01,562 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:51:01,562 EPOCH 9 done: loss 10.6405 - lr 0.5000000 +2021-03-26 07:51:02,273 DEV : loss 7.894908905029297 - score 0.8567 +2021-03-26 07:51:02,289 BAD EPOCHS (no improvement): 0 +2021-03-26 07:51:11,862 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:51:12,885 epoch 10 - iter 2/25 - loss 9.14244080 - samples/sec: 125.42 - lr: 0.500000 +2021-03-26 07:51:13,987 epoch 10 - iter 4/25 - loss 9.32414103 - samples/sec: 116.23 - lr: 0.500000 +2021-03-26 07:51:14,926 epoch 10 - iter 6/25 - loss 9.80224291 - samples/sec: 136.64 - lr: 0.500000 +2021-03-26 07:51:15,828 epoch 10 - iter 8/25 - loss 9.85451603 - samples/sec: 142.12 - lr: 0.500000 +2021-03-26 07:51:16,856 epoch 10 - iter 10/25 - loss 10.12377825 - samples/sec: 124.72 - lr: 0.500000 +2021-03-26 07:51:17,805 epoch 10 - iter 12/25 - loss 10.22787682 - samples/sec: 135.21 - lr: 0.500000 +2021-03-26 07:51:18,782 epoch 10 - iter 14/25 - loss 10.23994528 - samples/sec: 131.21 - lr: 0.500000 +2021-03-26 07:51:19,750 epoch 10 - iter 16/25 - loss 10.26584750 - samples/sec: 132.35 - lr: 0.500000 +2021-03-26 07:51:20,800 epoch 10 - iter 18/25 - loss 10.03869134 - samples/sec: 122.03 - lr: 0.500000 +2021-03-26 07:51:21,765 epoch 10 - iter 20/25 - loss 10.07074564 - samples/sec: 132.86 - lr: 0.500000 +2021-03-26 07:51:22,791 epoch 10 - iter 22/25 - loss 10.11766666 - samples/sec: 124.93 - lr: 0.500000 +2021-03-26 07:51:23,761 epoch 10 - iter 24/25 - loss 10.09356664 - samples/sec: 132.20 - lr: 0.500000 +2021-03-26 07:51:24,210 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:51:24,211 EPOCH 10 done: loss 10.0166 - lr 0.5000000 +2021-03-26 07:51:24,909 DEV : loss 7.626852989196777 - score 0.8632 +2021-03-26 07:51:24,931 BAD EPOCHS (no improvement): 0 +2021-03-26 07:51:34,414 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:51:35,444 epoch 11 - iter 2/25 - loss 8.91252089 - samples/sec: 124.50 - lr: 0.500000 +2021-03-26 07:51:36,482 epoch 11 - iter 4/25 - loss 10.49854732 - samples/sec: 123.46 - lr: 0.500000 +2021-03-26 07:51:37,509 epoch 11 - iter 6/25 - loss 10.22471849 - samples/sec: 124.78 - lr: 0.500000 +2021-03-26 07:51:38,507 epoch 11 - iter 8/25 - loss 10.09014302 - samples/sec: 128.50 - lr: 0.500000 +2021-03-26 07:51:39,488 epoch 11 - iter 10/25 - loss 9.73106799 - samples/sec: 130.67 - lr: 0.500000 +2021-03-26 07:51:40,420 epoch 11 - iter 12/25 - loss 9.70978502 - samples/sec: 137.52 - lr: 0.500000 +2021-03-26 07:51:41,410 epoch 11 - iter 14/25 - loss 9.73665486 - samples/sec: 129.50 - lr: 0.500000 +2021-03-26 07:51:42,513 epoch 11 - iter 16/25 - loss 9.72871694 - samples/sec: 116.19 - lr: 0.500000 +2021-03-26 07:51:43,538 epoch 11 - iter 18/25 - loss 9.64404469 - samples/sec: 124.99 - lr: 0.500000 +2021-03-26 07:51:44,531 epoch 11 - iter 20/25 - loss 9.57440164 - samples/sec: 129.19 - lr: 0.500000 +2021-03-26 07:51:45,455 epoch 11 - iter 22/25 - loss 9.50417950 - samples/sec: 138.90 - lr: 0.500000 +2021-03-26 07:51:46,481 epoch 11 - iter 24/25 - loss 9.46735579 - samples/sec: 125.01 - lr: 0.500000 +2021-03-26 07:51:46,855 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:51:46,855 EPOCH 11 done: loss 9.4820 - lr 0.5000000 +2021-03-26 07:51:47,566 DEV : loss 7.620308876037598 - score 0.8716 +2021-03-26 07:51:47,585 BAD EPOCHS (no improvement): 0 +2021-03-26 07:51:57,050 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:51:58,083 epoch 12 - iter 2/25 - loss 7.67201424 - samples/sec: 124.09 - lr: 0.500000 +2021-03-26 07:51:59,256 epoch 12 - iter 4/25 - loss 8.82815576 - samples/sec: 109.28 - lr: 0.500000 +2021-03-26 07:52:00,158 epoch 12 - iter 6/25 - loss 8.54262582 - samples/sec: 142.13 - lr: 0.500000 +2021-03-26 07:52:01,363 epoch 12 - iter 8/25 - loss 8.54254794 - samples/sec: 106.30 - lr: 0.500000 +2021-03-26 07:52:02,511 epoch 12 - iter 10/25 - loss 8.47853055 - samples/sec: 111.64 - lr: 0.500000 +2021-03-26 07:52:03,550 epoch 12 - iter 12/25 - loss 8.54671808 - samples/sec: 123.45 - lr: 0.500000 +2021-03-26 07:52:04,530 epoch 12 - iter 14/25 - loss 8.79490154 - samples/sec: 130.86 - lr: 0.500000 +2021-03-26 07:52:05,419 epoch 12 - iter 16/25 - loss 8.82608131 - samples/sec: 144.29 - lr: 0.500000 +2021-03-26 07:52:06,437 epoch 12 - iter 18/25 - loss 8.86837135 - samples/sec: 125.84 - lr: 0.500000 +2021-03-26 07:52:07,425 epoch 12 - iter 20/25 - loss 8.90788066 - samples/sec: 129.71 - lr: 0.500000 +2021-03-26 07:52:08,398 epoch 12 - iter 22/25 - loss 8.91862988 - samples/sec: 131.79 - lr: 0.500000 +2021-03-26 07:52:09,330 epoch 12 - iter 24/25 - loss 8.84400580 - samples/sec: 137.58 - lr: 0.500000 +2021-03-26 07:52:09,699 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:52:09,700 EPOCH 12 done: loss 8.9432 - lr 0.5000000 +2021-03-26 07:52:10,423 DEV : loss 6.980446815490723 - score 0.8768 +2021-03-26 07:52:10,446 BAD EPOCHS (no improvement): 0 +2021-03-26 07:52:20,022 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:52:20,956 epoch 13 - iter 2/25 - loss 7.05646849 - samples/sec: 137.35 - lr: 0.500000 +2021-03-26 07:52:21,982 epoch 13 - iter 4/25 - loss 7.92554772 - samples/sec: 124.97 - lr: 0.500000 +2021-03-26 07:52:22,942 epoch 13 - iter 6/25 - loss 7.98379747 - samples/sec: 133.41 - lr: 0.500000 +2021-03-26 07:52:23,958 epoch 13 - iter 8/25 - loss 8.20600587 - samples/sec: 126.20 - lr: 0.500000 +2021-03-26 07:52:24,950 epoch 13 - iter 10/25 - loss 8.51461835 - samples/sec: 129.37 - lr: 0.500000 +2021-03-26 07:52:25,909 epoch 13 - iter 12/25 - loss 8.30212975 - samples/sec: 133.64 - lr: 0.500000 +2021-03-26 07:52:26,958 epoch 13 - iter 14/25 - loss 8.24377768 - samples/sec: 122.13 - lr: 0.500000 +2021-03-26 07:52:27,908 epoch 13 - iter 16/25 - loss 8.16531828 - samples/sec: 135.01 - lr: 0.500000 +2021-03-26 07:52:29,409 epoch 13 - iter 18/25 - loss 8.26274286 - samples/sec: 85.57 - lr: 0.500000 +2021-03-26 07:52:30,333 epoch 13 - iter 20/25 - loss 8.25441029 - samples/sec: 138.93 - lr: 0.500000 +2021-03-26 07:52:31,310 epoch 13 - iter 22/25 - loss 8.27757309 - samples/sec: 131.11 - lr: 0.500000 +2021-03-26 07:52:32,235 epoch 13 - iter 24/25 - loss 8.19572711 - samples/sec: 138.71 - lr: 0.500000 +2021-03-26 07:52:32,605 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:52:32,606 EPOCH 13 done: loss 8.1866 - lr 0.5000000 +2021-03-26 07:52:33,404 DEV : loss 7.257625579833984 - score 0.8733 +2021-03-26 07:52:33,435 BAD EPOCHS (no improvement): 1 +2021-03-26 07:52:33,436 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:52:34,424 epoch 14 - iter 2/25 - loss 6.78682852 - samples/sec: 129.80 - lr: 0.500000 +2021-03-26 07:52:35,467 epoch 14 - iter 4/25 - loss 7.39494026 - samples/sec: 122.86 - lr: 0.500000 +2021-03-26 07:52:36,561 epoch 14 - iter 6/25 - loss 7.72300220 - samples/sec: 117.14 - lr: 0.500000 +2021-03-26 07:52:37,529 epoch 14 - iter 8/25 - loss 7.89962667 - samples/sec: 132.48 - lr: 0.500000 +2021-03-26 07:52:38,382 epoch 14 - iter 10/25 - loss 7.97575660 - samples/sec: 150.40 - lr: 0.500000 +2021-03-26 07:52:39,356 epoch 14 - iter 12/25 - loss 8.17424468 - samples/sec: 131.59 - lr: 0.500000 +2021-03-26 07:52:40,322 epoch 14 - iter 14/25 - loss 8.15351108 - samples/sec: 132.69 - lr: 0.500000 +2021-03-26 07:52:41,273 epoch 14 - iter 16/25 - loss 8.02801412 - samples/sec: 134.77 - lr: 0.500000 +2021-03-26 07:52:42,181 epoch 14 - iter 18/25 - loss 8.01207256 - samples/sec: 141.41 - lr: 0.500000 +2021-03-26 07:52:43,190 epoch 14 - iter 20/25 - loss 8.08496556 - samples/sec: 126.91 - lr: 0.500000 +2021-03-26 07:52:44,144 epoch 14 - iter 22/25 - loss 8.06667874 - samples/sec: 134.46 - lr: 0.500000 +2021-03-26 07:52:45,052 epoch 14 - iter 24/25 - loss 7.95059959 - samples/sec: 141.03 - lr: 0.500000 +2021-03-26 07:52:45,397 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:52:45,398 EPOCH 14 done: loss 7.9863 - lr 0.5000000 +2021-03-26 07:52:46,098 DEV : loss 6.4345903396606445 - score 0.8932 +2021-03-26 07:52:46,120 BAD EPOCHS (no improvement): 0 +2021-03-26 07:52:55,331 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:52:56,273 epoch 15 - iter 2/25 - loss 6.71810126 - samples/sec: 136.21 - lr: 0.500000 +2021-03-26 07:52:57,302 epoch 15 - iter 4/25 - loss 7.19238472 - samples/sec: 124.59 - lr: 0.500000 +2021-03-26 07:52:58,313 epoch 15 - iter 6/25 - loss 7.44798557 - samples/sec: 126.80 - lr: 0.500000 +2021-03-26 07:52:59,309 epoch 15 - iter 8/25 - loss 7.45711040 - samples/sec: 128.56 - lr: 0.500000 +2021-03-26 07:53:00,285 epoch 15 - iter 10/25 - loss 7.55697489 - samples/sec: 131.53 - lr: 0.500000 +2021-03-26 07:53:01,241 epoch 15 - iter 12/25 - loss 7.40633933 - samples/sec: 134.05 - lr: 0.500000 +2021-03-26 07:53:02,215 epoch 15 - iter 14/25 - loss 7.48868448 - samples/sec: 131.48 - lr: 0.500000 +2021-03-26 07:53:03,189 epoch 15 - iter 16/25 - loss 7.32153139 - samples/sec: 131.63 - lr: 0.500000 +2021-03-26 07:53:04,157 epoch 15 - iter 18/25 - loss 7.27229791 - samples/sec: 132.38 - lr: 0.500000 +2021-03-26 07:53:05,160 epoch 15 - iter 20/25 - loss 7.24484162 - samples/sec: 127.90 - lr: 0.500000 +2021-03-26 07:53:06,214 epoch 15 - iter 22/25 - loss 7.36682300 - samples/sec: 121.52 - lr: 0.500000 +2021-03-26 07:53:07,226 epoch 15 - iter 24/25 - loss 7.40492298 - samples/sec: 126.72 - lr: 0.500000 +2021-03-26 07:53:07,634 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:53:07,634 EPOCH 15 done: loss 7.3385 - lr 0.5000000 +2021-03-26 07:53:08,344 DEV : loss 6.836724281311035 - score 0.8777 +2021-03-26 07:53:08,367 BAD EPOCHS (no improvement): 1 +2021-03-26 07:53:08,367 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:53:09,413 epoch 16 - iter 2/25 - loss 6.85027146 - samples/sec: 122.62 - lr: 0.500000 +2021-03-26 07:53:10,360 epoch 16 - iter 4/25 - loss 7.33330190 - samples/sec: 135.55 - lr: 0.500000 +2021-03-26 07:53:11,350 epoch 16 - iter 6/25 - loss 7.08793155 - samples/sec: 129.51 - lr: 0.500000 +2021-03-26 07:53:12,379 epoch 16 - iter 8/25 - loss 6.93696636 - samples/sec: 124.58 - lr: 0.500000 +2021-03-26 07:53:13,315 epoch 16 - iter 10/25 - loss 7.01335969 - samples/sec: 136.89 - lr: 0.500000 +2021-03-26 07:53:14,336 epoch 16 - iter 12/25 - loss 7.08117270 - samples/sec: 125.55 - lr: 0.500000 +2021-03-26 07:53:15,324 epoch 16 - iter 14/25 - loss 7.18411749 - samples/sec: 129.81 - lr: 0.500000 +2021-03-26 07:53:16,324 epoch 16 - iter 16/25 - loss 7.09306252 - samples/sec: 128.08 - lr: 0.500000 +2021-03-26 07:53:17,201 epoch 16 - iter 18/25 - loss 7.03924876 - samples/sec: 146.25 - lr: 0.500000 +2021-03-26 07:53:18,261 epoch 16 - iter 20/25 - loss 7.06018779 - samples/sec: 120.94 - lr: 0.500000 +2021-03-26 07:53:19,187 epoch 16 - iter 22/25 - loss 6.94654627 - samples/sec: 138.52 - lr: 0.500000 +2021-03-26 07:53:20,187 epoch 16 - iter 24/25 - loss 6.94220535 - samples/sec: 128.15 - lr: 0.500000 +2021-03-26 07:53:20,553 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:53:20,554 EPOCH 16 done: loss 6.9130 - lr 0.5000000 +2021-03-26 07:53:21,271 DEV : loss 6.996776580810547 - score 0.8729 +2021-03-26 07:53:21,288 BAD EPOCHS (no improvement): 2 +2021-03-26 07:53:21,288 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:53:22,350 epoch 17 - iter 2/25 - loss 7.18355846 - samples/sec: 120.72 - lr: 0.500000 +2021-03-26 07:53:23,351 epoch 17 - iter 4/25 - loss 6.90322018 - samples/sec: 128.19 - lr: 0.500000 +2021-03-26 07:53:24,380 epoch 17 - iter 6/25 - loss 6.44195573 - samples/sec: 124.54 - lr: 0.500000 +2021-03-26 07:53:25,429 epoch 17 - iter 8/25 - loss 6.49139535 - samples/sec: 122.26 - lr: 0.500000 +2021-03-26 07:53:26,473 epoch 17 - iter 10/25 - loss 6.46172991 - samples/sec: 122.68 - lr: 0.500000 +2021-03-26 07:53:27,439 epoch 17 - iter 12/25 - loss 6.50268042 - samples/sec: 132.81 - lr: 0.500000 +2021-03-26 07:53:28,396 epoch 17 - iter 14/25 - loss 6.74807654 - samples/sec: 133.84 - lr: 0.500000 +2021-03-26 07:53:29,381 epoch 17 - iter 16/25 - loss 6.66306704 - samples/sec: 130.21 - lr: 0.500000 +2021-03-26 07:53:30,329 epoch 17 - iter 18/25 - loss 6.71620478 - samples/sec: 135.14 - lr: 0.500000 +2021-03-26 07:53:31,283 epoch 17 - iter 20/25 - loss 6.61219385 - samples/sec: 134.35 - lr: 0.500000 +2021-03-26 07:53:32,313 epoch 17 - iter 22/25 - loss 6.64762094 - samples/sec: 124.53 - lr: 0.500000 +2021-03-26 07:53:33,351 epoch 17 - iter 24/25 - loss 6.66213008 - samples/sec: 123.49 - lr: 0.500000 +2021-03-26 07:53:33,764 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:53:33,765 EPOCH 17 done: loss 6.7078 - lr 0.5000000 +2021-03-26 07:53:34,470 DEV : loss 7.163369655609131 - score 0.8774 +2021-03-26 07:53:34,487 BAD EPOCHS (no improvement): 3 +2021-03-26 07:53:34,487 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:53:35,424 epoch 18 - iter 2/25 - loss 5.66614342 - samples/sec: 136.85 - lr: 0.500000 +2021-03-26 07:53:36,422 epoch 18 - iter 4/25 - loss 6.22169757 - samples/sec: 128.51 - lr: 0.500000 +2021-03-26 07:53:37,479 epoch 18 - iter 6/25 - loss 6.44018420 - samples/sec: 121.25 - lr: 0.500000 +2021-03-26 07:53:38,479 epoch 18 - iter 8/25 - loss 6.35334659 - samples/sec: 128.20 - lr: 0.500000 +2021-03-26 07:53:39,441 epoch 18 - iter 10/25 - loss 6.60762820 - samples/sec: 133.28 - lr: 0.500000 +2021-03-26 07:53:40,318 epoch 18 - iter 12/25 - loss 6.52668659 - samples/sec: 146.07 - lr: 0.500000 +2021-03-26 07:53:41,289 epoch 18 - iter 14/25 - loss 6.49653762 - samples/sec: 132.09 - lr: 0.500000 +2021-03-26 07:53:42,288 epoch 18 - iter 16/25 - loss 6.45660698 - samples/sec: 128.25 - lr: 0.500000 +2021-03-26 07:53:43,313 epoch 18 - iter 18/25 - loss 6.49343742 - samples/sec: 125.05 - lr: 0.500000 +2021-03-26 07:53:44,321 epoch 18 - iter 20/25 - loss 6.41346722 - samples/sec: 127.13 - lr: 0.500000 +2021-03-26 07:53:45,243 epoch 18 - iter 22/25 - loss 6.40939960 - samples/sec: 139.04 - lr: 0.500000 +2021-03-26 07:53:46,247 epoch 18 - iter 24/25 - loss 6.44130540 - samples/sec: 127.68 - lr: 0.500000 +2021-03-26 07:53:46,713 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:53:46,713 EPOCH 18 done: loss 6.4110 - lr 0.5000000 +2021-03-26 07:53:47,407 DEV : loss 6.419670104980469 - score 0.8917 +2021-03-26 07:53:47,430 BAD EPOCHS (no improvement): 4 +2021-03-26 07:53:47,431 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:53:48,485 epoch 19 - iter 2/25 - loss 6.66521072 - samples/sec: 121.70 - lr: 0.250000 +2021-03-26 07:53:49,509 epoch 19 - iter 4/25 - loss 6.19065154 - samples/sec: 125.08 - lr: 0.250000 +2021-03-26 07:53:50,400 epoch 19 - iter 6/25 - loss 5.78149088 - samples/sec: 143.87 - lr: 0.250000 +2021-03-26 07:53:51,319 epoch 19 - iter 8/25 - loss 5.54224241 - samples/sec: 139.50 - lr: 0.250000 +2021-03-26 07:53:52,359 epoch 19 - iter 10/25 - loss 5.49967384 - samples/sec: 123.35 - lr: 0.250000 +2021-03-26 07:53:53,332 epoch 19 - iter 12/25 - loss 5.58384212 - samples/sec: 131.92 - lr: 0.250000 +2021-03-26 07:53:54,260 epoch 19 - iter 14/25 - loss 5.44728538 - samples/sec: 138.16 - lr: 0.250000 +2021-03-26 07:53:55,227 epoch 19 - iter 16/25 - loss 5.31798606 - samples/sec: 132.47 - lr: 0.250000 +2021-03-26 07:53:56,159 epoch 19 - iter 18/25 - loss 5.34107133 - samples/sec: 137.57 - lr: 0.250000 +2021-03-26 07:53:57,098 epoch 19 - iter 20/25 - loss 5.36848167 - samples/sec: 136.56 - lr: 0.250000 +2021-03-26 07:53:58,164 epoch 19 - iter 22/25 - loss 5.34087520 - samples/sec: 120.18 - lr: 0.250000 +2021-03-26 07:54:00,147 epoch 19 - iter 24/25 - loss 5.38866775 - samples/sec: 64.60 - lr: 0.250000 +2021-03-26 07:54:00,529 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:54:00,530 EPOCH 19 done: loss 5.3676 - lr 0.2500000 +2021-03-26 07:54:01,219 DEV : loss 6.131153106689453 - score 0.9014 +2021-03-26 07:54:01,242 BAD EPOCHS (no improvement): 0 +2021-03-26 07:54:10,431 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:54:11,321 epoch 20 - iter 2/25 - loss 5.13135290 - samples/sec: 144.08 - lr: 0.250000 +2021-03-26 07:54:12,240 epoch 20 - iter 4/25 - loss 4.89559913 - samples/sec: 139.45 - lr: 0.250000 +2021-03-26 07:54:13,232 epoch 20 - iter 6/25 - loss 4.89744592 - samples/sec: 129.28 - lr: 0.250000 +2021-03-26 07:54:14,291 epoch 20 - iter 8/25 - loss 5.01821065 - samples/sec: 121.03 - lr: 0.250000 +2021-03-26 07:54:15,261 epoch 20 - iter 10/25 - loss 5.12131805 - samples/sec: 132.08 - lr: 0.250000 +2021-03-26 07:54:16,311 epoch 20 - iter 12/25 - loss 5.06408310 - samples/sec: 122.06 - lr: 0.250000 +2021-03-26 07:54:17,336 epoch 20 - iter 14/25 - loss 5.08166361 - samples/sec: 125.08 - lr: 0.250000 +2021-03-26 07:54:18,357 epoch 20 - iter 16/25 - loss 5.11939466 - samples/sec: 125.51 - lr: 0.250000 +2021-03-26 07:54:19,365 epoch 20 - iter 18/25 - loss 5.16788875 - samples/sec: 127.25 - lr: 0.250000 +2021-03-26 07:54:20,331 epoch 20 - iter 20/25 - loss 5.13450365 - samples/sec: 132.66 - lr: 0.250000 +2021-03-26 07:54:21,269 epoch 20 - iter 22/25 - loss 5.11057932 - samples/sec: 136.79 - lr: 0.250000 +2021-03-26 07:54:22,210 epoch 20 - iter 24/25 - loss 5.13755359 - samples/sec: 136.13 - lr: 0.250000 +2021-03-26 07:54:22,603 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:54:22,604 EPOCH 20 done: loss 5.1303 - lr 0.2500000 +2021-03-26 07:54:23,308 DEV : loss 6.083251476287842 - score 0.9051 +2021-03-26 07:54:23,332 BAD EPOCHS (no improvement): 0 +2021-03-26 07:54:32,902 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:54:33,912 epoch 21 - iter 2/25 - loss 5.07513094 - samples/sec: 126.92 - lr: 0.250000 +2021-03-26 07:54:34,865 epoch 21 - iter 4/25 - loss 4.59668767 - samples/sec: 134.46 - lr: 0.250000 +2021-03-26 07:54:35,851 epoch 21 - iter 6/25 - loss 5.09568977 - samples/sec: 129.98 - lr: 0.250000 +2021-03-26 07:54:36,825 epoch 21 - iter 8/25 - loss 5.05758089 - samples/sec: 131.64 - lr: 0.250000 +2021-03-26 07:54:37,841 epoch 21 - iter 10/25 - loss 5.14766736 - samples/sec: 126.22 - lr: 0.250000 +2021-03-26 07:54:38,836 epoch 21 - iter 12/25 - loss 5.02646943 - samples/sec: 128.90 - lr: 0.250000 +2021-03-26 07:54:39,832 epoch 21 - iter 14/25 - loss 5.08124334 - samples/sec: 128.62 - lr: 0.250000 +2021-03-26 07:54:40,856 epoch 21 - iter 16/25 - loss 5.03595272 - samples/sec: 125.25 - lr: 0.250000 +2021-03-26 07:54:41,819 epoch 21 - iter 18/25 - loss 4.96463317 - samples/sec: 133.00 - lr: 0.250000 +2021-03-26 07:54:42,750 epoch 21 - iter 20/25 - loss 5.01546376 - samples/sec: 137.84 - lr: 0.250000 +2021-03-26 07:54:43,756 epoch 21 - iter 22/25 - loss 5.00643774 - samples/sec: 127.34 - lr: 0.250000 +2021-03-26 07:54:44,773 epoch 21 - iter 24/25 - loss 4.98594165 - samples/sec: 126.09 - lr: 0.250000 +2021-03-26 07:54:45,267 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:54:45,268 EPOCH 21 done: loss 4.9665 - lr 0.2500000 +2021-03-26 07:54:45,997 DEV : loss 5.991503715515137 - score 0.9039 +2021-03-26 07:54:46,023 BAD EPOCHS (no improvement): 1 +2021-03-26 07:54:46,023 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:54:47,082 epoch 22 - iter 2/25 - loss 4.61021471 - samples/sec: 121.14 - lr: 0.250000 +2021-03-26 07:54:48,179 epoch 22 - iter 4/25 - loss 4.90505421 - samples/sec: 116.73 - lr: 0.250000 +2021-03-26 07:54:49,127 epoch 22 - iter 6/25 - loss 4.72144151 - samples/sec: 135.31 - lr: 0.250000 +2021-03-26 07:54:50,087 epoch 22 - iter 8/25 - loss 4.85369271 - samples/sec: 133.47 - lr: 0.250000 +2021-03-26 07:54:51,112 epoch 22 - iter 10/25 - loss 4.81175332 - samples/sec: 124.94 - lr: 0.250000 +2021-03-26 07:54:52,276 epoch 22 - iter 12/25 - loss 4.84444952 - samples/sec: 110.16 - lr: 0.250000 +2021-03-26 07:54:53,381 epoch 22 - iter 14/25 - loss 4.80926984 - samples/sec: 115.97 - lr: 0.250000 +2021-03-26 07:54:54,307 epoch 22 - iter 16/25 - loss 4.71817878 - samples/sec: 138.44 - lr: 0.250000 +2021-03-26 07:54:55,316 epoch 22 - iter 18/25 - loss 4.80634297 - samples/sec: 127.00 - lr: 0.250000 +2021-03-26 07:54:56,435 epoch 22 - iter 20/25 - loss 4.80697734 - samples/sec: 114.55 - lr: 0.250000 +2021-03-26 07:54:57,396 epoch 22 - iter 22/25 - loss 4.84137470 - samples/sec: 133.46 - lr: 0.250000 +2021-03-26 07:54:58,321 epoch 22 - iter 24/25 - loss 4.78170185 - samples/sec: 138.50 - lr: 0.250000 +2021-03-26 07:54:58,681 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:54:58,682 EPOCH 22 done: loss 4.7751 - lr 0.2500000 +2021-03-26 07:54:59,410 DEV : loss 6.314764976501465 - score 0.9001 +2021-03-26 07:54:59,437 BAD EPOCHS (no improvement): 2 +2021-03-26 07:54:59,438 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:55:00,397 epoch 23 - iter 2/25 - loss 4.72261405 - samples/sec: 133.65 - lr: 0.250000 +2021-03-26 07:55:01,388 epoch 23 - iter 4/25 - loss 4.50521022 - samples/sec: 129.32 - lr: 0.250000 +2021-03-26 07:55:02,391 epoch 23 - iter 6/25 - loss 4.55807165 - samples/sec: 127.91 - lr: 0.250000 +2021-03-26 07:55:03,469 epoch 23 - iter 8/25 - loss 4.49088663 - samples/sec: 118.83 - lr: 0.250000 +2021-03-26 07:55:04,517 epoch 23 - iter 10/25 - loss 4.44302006 - samples/sec: 122.35 - lr: 0.250000 +2021-03-26 07:55:05,511 epoch 23 - iter 12/25 - loss 4.43786772 - samples/sec: 128.97 - lr: 0.250000 +2021-03-26 07:55:06,439 epoch 23 - iter 14/25 - loss 4.38659229 - samples/sec: 138.04 - lr: 0.250000 +2021-03-26 07:55:07,430 epoch 23 - iter 16/25 - loss 4.30455354 - samples/sec: 129.39 - lr: 0.250000 +2021-03-26 07:55:08,486 epoch 23 - iter 18/25 - loss 4.44754314 - samples/sec: 121.42 - lr: 0.250000 +2021-03-26 07:55:09,624 epoch 23 - iter 20/25 - loss 4.51354938 - samples/sec: 112.55 - lr: 0.250000 +2021-03-26 07:55:10,623 epoch 23 - iter 22/25 - loss 4.57599319 - samples/sec: 128.33 - lr: 0.250000 +2021-03-26 07:55:11,578 epoch 23 - iter 24/25 - loss 4.56353696 - samples/sec: 134.26 - lr: 0.250000 +2021-03-26 07:55:11,972 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:55:11,973 EPOCH 23 done: loss 4.6286 - lr 0.2500000 +2021-03-26 07:55:12,670 DEV : loss 6.0209550857543945 - score 0.9043 +2021-03-26 07:55:12,687 BAD EPOCHS (no improvement): 3 +2021-03-26 07:55:12,688 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:55:13,617 epoch 24 - iter 2/25 - loss 4.41330647 - samples/sec: 137.88 - lr: 0.250000 +2021-03-26 07:55:14,613 epoch 24 - iter 4/25 - loss 4.45420444 - samples/sec: 128.80 - lr: 0.250000 +2021-03-26 07:55:15,576 epoch 24 - iter 6/25 - loss 4.73466698 - samples/sec: 133.14 - lr: 0.250000 +2021-03-26 07:55:16,675 epoch 24 - iter 8/25 - loss 4.60940790 - samples/sec: 116.64 - lr: 0.250000 +2021-03-26 07:55:17,679 epoch 24 - iter 10/25 - loss 4.70087466 - samples/sec: 127.74 - lr: 0.250000 +2021-03-26 07:55:18,752 epoch 24 - iter 12/25 - loss 4.71539728 - samples/sec: 119.48 - lr: 0.250000 +2021-03-26 07:55:19,818 epoch 24 - iter 14/25 - loss 4.94809369 - samples/sec: 120.33 - lr: 0.250000 +2021-03-26 07:55:20,707 epoch 24 - iter 16/25 - loss 4.83176947 - samples/sec: 144.17 - lr: 0.250000 +2021-03-26 07:55:21,710 epoch 24 - iter 18/25 - loss 4.76679060 - samples/sec: 127.78 - lr: 0.250000 +2021-03-26 07:55:22,612 epoch 24 - iter 20/25 - loss 4.67498168 - samples/sec: 142.17 - lr: 0.250000 +2021-03-26 07:55:23,582 epoch 24 - iter 22/25 - loss 4.66213375 - samples/sec: 132.25 - lr: 0.250000 +2021-03-26 07:55:24,509 epoch 24 - iter 24/25 - loss 4.64017315 - samples/sec: 138.16 - lr: 0.250000 +2021-03-26 07:55:25,050 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:55:25,051 EPOCH 24 done: loss 4.6423 - lr 0.2500000 +2021-03-26 07:55:25,770 DEV : loss 6.313882827758789 - score 0.8984 +2021-03-26 07:55:25,787 BAD EPOCHS (no improvement): 4 +2021-03-26 07:55:25,788 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:55:26,720 epoch 25 - iter 2/25 - loss 4.32253599 - samples/sec: 137.42 - lr: 0.125000 +2021-03-26 07:55:27,739 epoch 25 - iter 4/25 - loss 4.18250024 - samples/sec: 125.83 - lr: 0.125000 +2021-03-26 07:55:28,672 epoch 25 - iter 6/25 - loss 4.49590453 - samples/sec: 137.51 - lr: 0.125000 +2021-03-26 07:55:29,685 epoch 25 - iter 8/25 - loss 4.32094258 - samples/sec: 126.65 - lr: 0.125000 +2021-03-26 07:55:30,652 epoch 25 - iter 10/25 - loss 4.35360093 - samples/sec: 132.42 - lr: 0.125000 +2021-03-26 07:55:31,693 epoch 25 - iter 12/25 - loss 4.36569381 - samples/sec: 123.19 - lr: 0.125000 +2021-03-26 07:55:32,614 epoch 25 - iter 14/25 - loss 4.32159005 - samples/sec: 139.21 - lr: 0.125000 +2021-03-26 07:55:33,653 epoch 25 - iter 16/25 - loss 4.24558356 - samples/sec: 123.38 - lr: 0.125000 +2021-03-26 07:55:34,617 epoch 25 - iter 18/25 - loss 4.22551627 - samples/sec: 133.01 - lr: 0.125000 +2021-03-26 07:55:35,577 epoch 25 - iter 20/25 - loss 4.23848104 - samples/sec: 133.47 - lr: 0.125000 +2021-03-26 07:55:36,581 epoch 25 - iter 22/25 - loss 4.18101357 - samples/sec: 127.68 - lr: 0.125000 +2021-03-26 07:55:37,655 epoch 25 - iter 24/25 - loss 4.18645438 - samples/sec: 119.40 - lr: 0.125000 +2021-03-26 07:55:38,209 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:55:38,209 EPOCH 25 done: loss 4.1730 - lr 0.1250000 +2021-03-26 07:55:38,921 DEV : loss 5.988753318786621 - score 0.9076 +2021-03-26 07:55:38,944 BAD EPOCHS (no improvement): 0 +2021-03-26 07:55:48,363 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:55:49,285 epoch 26 - iter 2/25 - loss 4.35821223 - samples/sec: 139.18 - lr: 0.125000 +2021-03-26 07:55:50,217 epoch 26 - iter 4/25 - loss 4.52768040 - samples/sec: 137.51 - lr: 0.125000 +2021-03-26 07:55:51,212 epoch 26 - iter 6/25 - loss 4.56613183 - samples/sec: 128.75 - lr: 0.125000 +2021-03-26 07:55:52,148 epoch 26 - iter 8/25 - loss 4.34084165 - samples/sec: 137.01 - lr: 0.125000 +2021-03-26 07:55:53,102 epoch 26 - iter 10/25 - loss 4.26465180 - samples/sec: 134.35 - lr: 0.125000 +2021-03-26 07:55:54,004 epoch 26 - iter 12/25 - loss 4.19501319 - samples/sec: 142.17 - lr: 0.125000 +2021-03-26 07:55:55,014 epoch 26 - iter 14/25 - loss 4.12131996 - samples/sec: 126.85 - lr: 0.125000 +2021-03-26 07:55:56,021 epoch 26 - iter 16/25 - loss 4.20322640 - samples/sec: 127.34 - lr: 0.125000 +2021-03-26 07:55:57,307 epoch 26 - iter 18/25 - loss 4.15836038 - samples/sec: 99.62 - lr: 0.125000 +2021-03-26 07:55:58,309 epoch 26 - iter 20/25 - loss 4.14088476 - samples/sec: 127.91 - lr: 0.125000 +2021-03-26 07:55:59,211 epoch 26 - iter 22/25 - loss 4.06868502 - samples/sec: 142.16 - lr: 0.125000 +2021-03-26 07:56:00,131 epoch 26 - iter 24/25 - loss 4.07539245 - samples/sec: 139.38 - lr: 0.125000 +2021-03-26 07:56:00,505 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:56:00,506 EPOCH 26 done: loss 4.0593 - lr 0.1250000 +2021-03-26 07:56:01,210 DEV : loss 5.971909999847412 - score 0.9089 +2021-03-26 07:56:01,232 BAD EPOCHS (no improvement): 0 +2021-03-26 07:56:10,845 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:56:11,860 epoch 27 - iter 2/25 - loss 4.68327546 - samples/sec: 126.33 - lr: 0.125000 +2021-03-26 07:56:12,875 epoch 27 - iter 4/25 - loss 4.54550594 - samples/sec: 126.33 - lr: 0.125000 +2021-03-26 07:56:13,771 epoch 27 - iter 6/25 - loss 4.35505827 - samples/sec: 143.85 - lr: 0.125000 +2021-03-26 07:56:14,750 epoch 27 - iter 8/25 - loss 4.39272493 - samples/sec: 130.93 - lr: 0.125000 +2021-03-26 07:56:16,153 epoch 27 - iter 10/25 - loss 4.28880415 - samples/sec: 91.35 - lr: 0.125000 +2021-03-26 07:56:17,447 epoch 27 - iter 12/25 - loss 4.25273832 - samples/sec: 99.00 - lr: 0.125000 +2021-03-26 07:56:18,522 epoch 27 - iter 14/25 - loss 4.23168690 - samples/sec: 119.20 - lr: 0.125000 +2021-03-26 07:56:19,857 epoch 27 - iter 16/25 - loss 4.15360086 - samples/sec: 96.01 - lr: 0.125000 +2021-03-26 07:56:21,082 epoch 27 - iter 18/25 - loss 4.17100709 - samples/sec: 104.65 - lr: 0.125000 +2021-03-26 07:56:22,382 epoch 27 - iter 20/25 - loss 4.19213790 - samples/sec: 98.51 - lr: 0.125000 +2021-03-26 07:56:23,650 epoch 27 - iter 22/25 - loss 4.14735520 - samples/sec: 101.07 - lr: 0.125000 +2021-03-26 07:56:24,807 epoch 27 - iter 24/25 - loss 4.09487817 - samples/sec: 110.81 - lr: 0.125000 +2021-03-26 07:56:25,255 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:56:25,256 EPOCH 27 done: loss 4.0652 - lr 0.1250000 +2021-03-26 07:56:25,971 DEV : loss 5.983981132507324 - score 0.9106 +2021-03-26 07:56:25,987 BAD EPOCHS (no improvement): 0 +2021-03-26 07:56:35,553 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:56:36,823 epoch 28 - iter 2/25 - loss 3.55781388 - samples/sec: 101.03 - lr: 0.125000 +2021-03-26 07:56:38,258 epoch 28 - iter 4/25 - loss 4.03270161 - samples/sec: 89.30 - lr: 0.125000 +2021-03-26 07:56:39,398 epoch 28 - iter 6/25 - loss 4.05528371 - samples/sec: 112.44 - lr: 0.125000 +2021-03-26 07:56:40,275 epoch 28 - iter 8/25 - loss 3.89411348 - samples/sec: 146.25 - lr: 0.125000 +2021-03-26 07:56:41,225 epoch 28 - iter 10/25 - loss 3.93902600 - samples/sec: 134.86 - lr: 0.125000 +2021-03-26 07:56:42,204 epoch 28 - iter 12/25 - loss 3.89163711 - samples/sec: 130.96 - lr: 0.125000 +2021-03-26 07:56:43,268 epoch 28 - iter 14/25 - loss 3.83746786 - samples/sec: 120.52 - lr: 0.125000 +2021-03-26 07:56:44,240 epoch 28 - iter 16/25 - loss 3.78483669 - samples/sec: 131.87 - lr: 0.125000 +2021-03-26 07:56:45,195 epoch 28 - iter 18/25 - loss 3.86538806 - samples/sec: 134.37 - lr: 0.125000 +2021-03-26 07:56:46,164 epoch 28 - iter 20/25 - loss 3.83518734 - samples/sec: 132.42 - lr: 0.125000 +2021-03-26 07:56:47,233 epoch 28 - iter 22/25 - loss 3.83221832 - samples/sec: 119.91 - lr: 0.125000 +2021-03-26 07:56:48,466 epoch 28 - iter 24/25 - loss 3.86373717 - samples/sec: 103.89 - lr: 0.125000 +2021-03-26 07:56:48,977 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:56:48,977 EPOCH 28 done: loss 3.8487 - lr 0.1250000 +2021-03-26 07:56:49,695 DEV : loss 6.044693946838379 - score 0.9064 +2021-03-26 07:56:49,717 BAD EPOCHS (no improvement): 1 +2021-03-26 07:56:49,717 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:56:50,936 epoch 29 - iter 2/25 - loss 2.99994659 - samples/sec: 105.17 - lr: 0.125000 +2021-03-26 07:56:52,113 epoch 29 - iter 4/25 - loss 3.65278989 - samples/sec: 108.94 - lr: 0.125000 +2021-03-26 07:56:53,079 epoch 29 - iter 6/25 - loss 3.79836094 - samples/sec: 132.63 - lr: 0.125000 +2021-03-26 07:56:54,131 epoch 29 - iter 8/25 - loss 3.85753009 - samples/sec: 121.81 - lr: 0.125000 +2021-03-26 07:56:55,074 epoch 29 - iter 10/25 - loss 3.81739969 - samples/sec: 135.93 - lr: 0.125000 +2021-03-26 07:56:56,008 epoch 29 - iter 12/25 - loss 3.90435483 - samples/sec: 137.43 - lr: 0.125000 +2021-03-26 07:56:57,121 epoch 29 - iter 14/25 - loss 3.80697027 - samples/sec: 115.14 - lr: 0.125000 +2021-03-26 07:56:58,281 epoch 29 - iter 16/25 - loss 3.77527751 - samples/sec: 110.49 - lr: 0.125000 +2021-03-26 07:56:59,406 epoch 29 - iter 18/25 - loss 3.78040032 - samples/sec: 113.91 - lr: 0.125000 +2021-03-26 07:57:00,543 epoch 29 - iter 20/25 - loss 3.73319964 - samples/sec: 112.71 - lr: 0.125000 +2021-03-26 07:57:01,681 epoch 29 - iter 22/25 - loss 3.72866162 - samples/sec: 112.74 - lr: 0.125000 +2021-03-26 07:57:02,612 epoch 29 - iter 24/25 - loss 3.70865181 - samples/sec: 137.60 - lr: 0.125000 +2021-03-26 07:57:03,003 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:57:03,004 EPOCH 29 done: loss 3.7015 - lr 0.1250000 +2021-03-26 07:57:03,779 DEV : loss 6.004291534423828 - score 0.9089 +2021-03-26 07:57:03,799 BAD EPOCHS (no improvement): 2 +2021-03-26 07:57:03,800 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:57:04,894 epoch 30 - iter 2/25 - loss 3.83538103 - samples/sec: 117.24 - lr: 0.125000 +2021-03-26 07:57:05,923 epoch 30 - iter 4/25 - loss 3.71162516 - samples/sec: 124.56 - lr: 0.125000 +2021-03-26 07:57:06,884 epoch 30 - iter 6/25 - loss 3.87396975 - samples/sec: 133.32 - lr: 0.125000 +2021-03-26 07:57:07,869 epoch 30 - iter 8/25 - loss 3.91214129 - samples/sec: 130.05 - lr: 0.125000 +2021-03-26 07:57:08,862 epoch 30 - iter 10/25 - loss 3.95587745 - samples/sec: 129.22 - lr: 0.125000 +2021-03-26 07:57:09,862 epoch 30 - iter 12/25 - loss 3.95001904 - samples/sec: 128.22 - lr: 0.125000 +2021-03-26 07:57:10,820 epoch 30 - iter 14/25 - loss 3.84484034 - samples/sec: 133.84 - lr: 0.125000 +2021-03-26 07:57:11,801 epoch 30 - iter 16/25 - loss 3.82006696 - samples/sec: 130.73 - lr: 0.125000 +2021-03-26 07:57:12,815 epoch 30 - iter 18/25 - loss 3.82373320 - samples/sec: 126.50 - lr: 0.125000 +2021-03-26 07:57:13,873 epoch 30 - iter 20/25 - loss 3.71675273 - samples/sec: 121.20 - lr: 0.125000 +2021-03-26 07:57:14,950 epoch 30 - iter 22/25 - loss 3.74051334 - samples/sec: 119.05 - lr: 0.125000 +2021-03-26 07:57:15,987 epoch 30 - iter 24/25 - loss 3.75257254 - samples/sec: 123.69 - lr: 0.125000 +2021-03-26 07:57:16,422 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:57:16,423 EPOCH 30 done: loss 3.7078 - lr 0.1250000 +2021-03-26 07:57:17,145 DEV : loss 5.97253942489624 - score 0.9102 +2021-03-26 07:57:17,168 BAD EPOCHS (no improvement): 3 +2021-03-26 07:57:17,168 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:57:18,306 epoch 31 - iter 2/25 - loss 3.89688802 - samples/sec: 112.68 - lr: 0.125000 +2021-03-26 07:57:19,187 epoch 31 - iter 4/25 - loss 3.70891809 - samples/sec: 145.60 - lr: 0.125000 +2021-03-26 07:57:20,129 epoch 31 - iter 6/25 - loss 3.78291845 - samples/sec: 136.03 - lr: 0.125000 +2021-03-26 07:57:21,123 epoch 31 - iter 8/25 - loss 3.78016075 - samples/sec: 129.10 - lr: 0.125000 +2021-03-26 07:57:22,101 epoch 31 - iter 10/25 - loss 3.74238210 - samples/sec: 131.21 - lr: 0.125000 +2021-03-26 07:57:23,084 epoch 31 - iter 12/25 - loss 3.75834699 - samples/sec: 130.42 - lr: 0.125000 +2021-03-26 07:57:24,184 epoch 31 - iter 14/25 - loss 3.67023890 - samples/sec: 116.44 - lr: 0.125000 +2021-03-26 07:57:25,137 epoch 31 - iter 16/25 - loss 3.59937003 - samples/sec: 134.60 - lr: 0.125000 +2021-03-26 07:57:26,051 epoch 31 - iter 18/25 - loss 3.61951150 - samples/sec: 140.16 - lr: 0.125000 +2021-03-26 07:57:27,041 epoch 31 - iter 20/25 - loss 3.66483914 - samples/sec: 129.52 - lr: 0.125000 +2021-03-26 07:57:28,030 epoch 31 - iter 22/25 - loss 3.61880018 - samples/sec: 129.63 - lr: 0.125000 +2021-03-26 07:57:28,950 epoch 31 - iter 24/25 - loss 3.60269628 - samples/sec: 139.32 - lr: 0.125000 +2021-03-26 07:57:29,419 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:57:29,419 EPOCH 31 done: loss 3.6264 - lr 0.1250000 +2021-03-26 07:57:30,161 DEV : loss 5.973457336425781 - score 0.9139 +2021-03-26 07:57:30,182 BAD EPOCHS (no improvement): 0 +2021-03-26 07:57:39,545 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:57:40,610 epoch 32 - iter 2/25 - loss 3.11975098 - samples/sec: 120.45 - lr: 0.125000 +2021-03-26 07:57:41,581 epoch 32 - iter 4/25 - loss 3.29605865 - samples/sec: 132.13 - lr: 0.125000 +2021-03-26 07:57:42,541 epoch 32 - iter 6/25 - loss 3.50240274 - samples/sec: 133.40 - lr: 0.125000 +2021-03-26 07:57:43,498 epoch 32 - iter 8/25 - loss 3.41950604 - samples/sec: 134.00 - lr: 0.125000 +2021-03-26 07:57:44,451 epoch 32 - iter 10/25 - loss 3.45227201 - samples/sec: 134.48 - lr: 0.125000 +2021-03-26 07:57:45,596 epoch 32 - iter 12/25 - loss 3.46193991 - samples/sec: 111.94 - lr: 0.125000 +2021-03-26 07:57:46,620 epoch 32 - iter 14/25 - loss 3.43003614 - samples/sec: 125.13 - lr: 0.125000 +2021-03-26 07:57:47,548 epoch 32 - iter 16/25 - loss 3.40545425 - samples/sec: 138.31 - lr: 0.125000 +2021-03-26 07:57:48,546 epoch 32 - iter 18/25 - loss 3.41395897 - samples/sec: 128.40 - lr: 0.125000 +2021-03-26 07:57:49,505 epoch 32 - iter 20/25 - loss 3.36117598 - samples/sec: 133.62 - lr: 0.125000 +2021-03-26 07:57:50,544 epoch 32 - iter 22/25 - loss 3.38892034 - samples/sec: 123.47 - lr: 0.125000 +2021-03-26 07:57:51,502 epoch 32 - iter 24/25 - loss 3.40688891 - samples/sec: 133.87 - lr: 0.125000 +2021-03-26 07:57:51,961 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:57:51,962 EPOCH 32 done: loss 3.4289 - lr 0.1250000 +2021-03-26 07:57:52,668 DEV : loss 6.120488166809082 - score 0.9131 +2021-03-26 07:57:52,686 BAD EPOCHS (no improvement): 1 +2021-03-26 07:57:52,686 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:57:53,583 epoch 33 - iter 2/25 - loss 3.59765363 - samples/sec: 142.92 - lr: 0.125000 +2021-03-26 07:57:54,539 epoch 33 - iter 4/25 - loss 3.69549191 - samples/sec: 133.96 - lr: 0.125000 +2021-03-26 07:57:55,547 epoch 33 - iter 6/25 - loss 3.97329044 - samples/sec: 127.28 - lr: 0.125000 +2021-03-26 07:57:56,547 epoch 33 - iter 8/25 - loss 3.83983880 - samples/sec: 128.23 - lr: 0.125000 +2021-03-26 07:57:57,617 epoch 33 - iter 10/25 - loss 3.84227064 - samples/sec: 119.77 - lr: 0.125000 +2021-03-26 07:57:58,659 epoch 33 - iter 12/25 - loss 3.78336207 - samples/sec: 123.00 - lr: 0.125000 +2021-03-26 07:57:59,608 epoch 33 - iter 14/25 - loss 3.68971201 - samples/sec: 135.00 - lr: 0.125000 +2021-03-26 07:58:00,559 epoch 33 - iter 16/25 - loss 3.55988289 - samples/sec: 134.91 - lr: 0.125000 +2021-03-26 07:58:01,609 epoch 33 - iter 18/25 - loss 3.52473515 - samples/sec: 122.04 - lr: 0.125000 +2021-03-26 07:58:02,540 epoch 33 - iter 20/25 - loss 3.49631225 - samples/sec: 137.71 - lr: 0.125000 +2021-03-26 07:58:03,496 epoch 33 - iter 22/25 - loss 3.54496320 - samples/sec: 134.09 - lr: 0.125000 +2021-03-26 07:58:04,537 epoch 33 - iter 24/25 - loss 3.56121511 - samples/sec: 123.13 - lr: 0.125000 +2021-03-26 07:58:04,910 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:58:04,911 EPOCH 33 done: loss 3.5053 - lr 0.1250000 +2021-03-26 07:58:05,626 DEV : loss 6.25508451461792 - score 0.9093 +2021-03-26 07:58:05,649 BAD EPOCHS (no improvement): 2 +2021-03-26 07:58:05,650 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:58:06,672 epoch 34 - iter 2/25 - loss 3.48426843 - samples/sec: 125.49 - lr: 0.125000 +2021-03-26 07:58:07,726 epoch 34 - iter 4/25 - loss 3.28659660 - samples/sec: 121.57 - lr: 0.125000 +2021-03-26 07:58:08,750 epoch 34 - iter 6/25 - loss 3.18423168 - samples/sec: 125.25 - lr: 0.125000 +2021-03-26 07:58:09,805 epoch 34 - iter 8/25 - loss 3.21228367 - samples/sec: 121.47 - lr: 0.125000 +2021-03-26 07:58:10,858 epoch 34 - iter 10/25 - loss 3.12810292 - samples/sec: 121.67 - lr: 0.125000 +2021-03-26 07:58:11,879 epoch 34 - iter 12/25 - loss 3.27248840 - samples/sec: 125.73 - lr: 0.125000 +2021-03-26 07:58:13,291 epoch 34 - iter 14/25 - loss 3.33075367 - samples/sec: 90.74 - lr: 0.125000 +2021-03-26 07:58:14,675 epoch 34 - iter 16/25 - loss 3.39736582 - samples/sec: 92.58 - lr: 0.125000 +2021-03-26 07:58:16,055 epoch 34 - iter 18/25 - loss 3.37778645 - samples/sec: 92.87 - lr: 0.125000 +2021-03-26 07:58:17,327 epoch 34 - iter 20/25 - loss 3.35320867 - samples/sec: 100.73 - lr: 0.125000 +2021-03-26 07:58:18,337 epoch 34 - iter 22/25 - loss 3.38617272 - samples/sec: 126.90 - lr: 0.125000 +2021-03-26 07:58:19,281 epoch 34 - iter 24/25 - loss 3.36547829 - samples/sec: 135.78 - lr: 0.125000 +2021-03-26 07:58:19,735 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:58:19,735 EPOCH 34 done: loss 3.3455 - lr 0.1250000 +2021-03-26 07:58:20,451 DEV : loss 6.16842794418335 - score 0.9148 +2021-03-26 07:58:20,474 BAD EPOCHS (no improvement): 0 +2021-03-26 07:58:29,897 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:58:30,954 epoch 35 - iter 2/25 - loss 2.87888587 - samples/sec: 121.35 - lr: 0.125000 +2021-03-26 07:58:31,884 epoch 35 - iter 4/25 - loss 3.05180985 - samples/sec: 137.88 - lr: 0.125000 +2021-03-26 07:58:32,911 epoch 35 - iter 6/25 - loss 2.96273021 - samples/sec: 124.74 - lr: 0.125000 +2021-03-26 07:58:33,942 epoch 35 - iter 8/25 - loss 3.08268613 - samples/sec: 124.41 - lr: 0.125000 +2021-03-26 07:58:34,926 epoch 35 - iter 10/25 - loss 3.14363208 - samples/sec: 130.30 - lr: 0.125000 +2021-03-26 07:58:35,927 epoch 35 - iter 12/25 - loss 3.29011472 - samples/sec: 127.96 - lr: 0.125000 +2021-03-26 07:58:36,831 epoch 35 - iter 14/25 - loss 3.34185306 - samples/sec: 141.86 - lr: 0.125000 +2021-03-26 07:58:37,949 epoch 35 - iter 16/25 - loss 3.37906143 - samples/sec: 114.59 - lr: 0.125000 +2021-03-26 07:58:38,958 epoch 35 - iter 18/25 - loss 3.36027298 - samples/sec: 127.15 - lr: 0.125000 +2021-03-26 07:58:39,892 epoch 35 - iter 20/25 - loss 3.36290877 - samples/sec: 137.43 - lr: 0.125000 +2021-03-26 07:58:40,836 epoch 35 - iter 22/25 - loss 3.39653666 - samples/sec: 135.74 - lr: 0.125000 +2021-03-26 07:58:41,828 epoch 35 - iter 24/25 - loss 3.39750968 - samples/sec: 129.22 - lr: 0.125000 +2021-03-26 07:58:42,263 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:58:42,264 EPOCH 35 done: loss 3.3719 - lr 0.1250000 +2021-03-26 07:58:42,981 DEV : loss 6.130885124206543 - score 0.9135 +2021-03-26 07:58:42,997 BAD EPOCHS (no improvement): 1 +2021-03-26 07:58:42,997 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:58:44,006 epoch 36 - iter 2/25 - loss 3.68344629 - samples/sec: 127.05 - lr: 0.125000 +2021-03-26 07:58:45,076 epoch 36 - iter 4/25 - loss 3.82729638 - samples/sec: 119.80 - lr: 0.125000 +2021-03-26 07:58:45,950 epoch 36 - iter 6/25 - loss 3.52623641 - samples/sec: 146.70 - lr: 0.125000 +2021-03-26 07:58:46,895 epoch 36 - iter 8/25 - loss 3.51207724 - samples/sec: 135.66 - lr: 0.125000 +2021-03-26 07:58:47,779 epoch 36 - iter 10/25 - loss 3.54672844 - samples/sec: 145.14 - lr: 0.125000 +2021-03-26 07:58:48,749 epoch 36 - iter 12/25 - loss 3.48546157 - samples/sec: 132.10 - lr: 0.125000 +2021-03-26 07:58:49,619 epoch 36 - iter 14/25 - loss 3.43780621 - samples/sec: 147.49 - lr: 0.125000 +2021-03-26 07:58:50,709 epoch 36 - iter 16/25 - loss 3.35306793 - samples/sec: 117.58 - lr: 0.125000 +2021-03-26 07:58:51,670 epoch 36 - iter 18/25 - loss 3.29722373 - samples/sec: 133.35 - lr: 0.125000 +2021-03-26 07:58:52,684 epoch 36 - iter 20/25 - loss 3.34835799 - samples/sec: 126.47 - lr: 0.125000 +2021-03-26 07:58:53,682 epoch 36 - iter 22/25 - loss 3.32949518 - samples/sec: 128.42 - lr: 0.125000 +2021-03-26 07:58:54,728 epoch 36 - iter 24/25 - loss 3.32508539 - samples/sec: 122.54 - lr: 0.125000 +2021-03-26 07:58:55,161 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:58:55,162 EPOCH 36 done: loss 3.3480 - lr 0.1250000 +2021-03-26 07:58:55,881 DEV : loss 6.107944488525391 - score 0.9118 +2021-03-26 07:58:55,897 BAD EPOCHS (no improvement): 2 +2021-03-26 07:58:55,898 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:58:56,914 epoch 37 - iter 2/25 - loss 3.42133987 - samples/sec: 126.21 - lr: 0.125000 +2021-03-26 07:58:57,950 epoch 37 - iter 4/25 - loss 3.14635503 - samples/sec: 123.69 - lr: 0.125000 +2021-03-26 07:58:58,940 epoch 37 - iter 6/25 - loss 3.18325603 - samples/sec: 129.51 - lr: 0.125000 +2021-03-26 07:58:59,906 epoch 37 - iter 8/25 - loss 3.35971868 - samples/sec: 132.78 - lr: 0.125000 +2021-03-26 07:59:00,893 epoch 37 - iter 10/25 - loss 3.33887649 - samples/sec: 129.78 - lr: 0.125000 +2021-03-26 07:59:01,841 epoch 37 - iter 12/25 - loss 3.28747088 - samples/sec: 135.28 - lr: 0.125000 +2021-03-26 07:59:02,774 epoch 37 - iter 14/25 - loss 3.31001389 - samples/sec: 137.48 - lr: 0.125000 +2021-03-26 07:59:03,825 epoch 37 - iter 16/25 - loss 3.27298586 - samples/sec: 122.00 - lr: 0.125000 +2021-03-26 07:59:04,997 epoch 37 - iter 18/25 - loss 3.28302748 - samples/sec: 109.26 - lr: 0.125000 +2021-03-26 07:59:05,943 epoch 37 - iter 20/25 - loss 3.32614937 - samples/sec: 135.59 - lr: 0.125000 +2021-03-26 07:59:06,910 epoch 37 - iter 22/25 - loss 3.35880077 - samples/sec: 132.50 - lr: 0.125000 +2021-03-26 07:59:07,803 epoch 37 - iter 24/25 - loss 3.34755198 - samples/sec: 143.58 - lr: 0.125000 +2021-03-26 07:59:08,232 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:59:08,232 EPOCH 37 done: loss 3.3532 - lr 0.1250000 +2021-03-26 07:59:08,941 DEV : loss 5.940009117126465 - score 0.916 +2021-03-26 07:59:08,964 BAD EPOCHS (no improvement): 0 +2021-03-26 07:59:18,132 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:59:18,997 epoch 38 - iter 2/25 - loss 3.08662271 - samples/sec: 148.43 - lr: 0.125000 +2021-03-26 07:59:19,947 epoch 38 - iter 4/25 - loss 3.46704584 - samples/sec: 135.02 - lr: 0.125000 +2021-03-26 07:59:20,870 epoch 38 - iter 6/25 - loss 3.45128655 - samples/sec: 138.92 - lr: 0.125000 +2021-03-26 07:59:21,902 epoch 38 - iter 8/25 - loss 3.49598333 - samples/sec: 124.21 - lr: 0.125000 +2021-03-26 07:59:22,925 epoch 38 - iter 10/25 - loss 3.31913579 - samples/sec: 125.35 - lr: 0.125000 +2021-03-26 07:59:24,020 epoch 38 - iter 12/25 - loss 3.36305465 - samples/sec: 117.11 - lr: 0.125000 +2021-03-26 07:59:25,021 epoch 38 - iter 14/25 - loss 3.27468027 - samples/sec: 128.02 - lr: 0.125000 +2021-03-26 07:59:25,956 epoch 38 - iter 16/25 - loss 3.22047535 - samples/sec: 137.09 - lr: 0.125000 +2021-03-26 07:59:27,007 epoch 38 - iter 18/25 - loss 3.27085086 - samples/sec: 122.08 - lr: 0.125000 +2021-03-26 07:59:28,007 epoch 38 - iter 20/25 - loss 3.20065688 - samples/sec: 128.18 - lr: 0.125000 +2021-03-26 07:59:29,013 epoch 38 - iter 22/25 - loss 3.20793955 - samples/sec: 127.45 - lr: 0.125000 +2021-03-26 07:59:30,112 epoch 38 - iter 24/25 - loss 3.22620339 - samples/sec: 116.61 - lr: 0.125000 +2021-03-26 07:59:30,574 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:59:30,574 EPOCH 38 done: loss 3.2033 - lr 0.1250000 +2021-03-26 07:59:31,315 DEV : loss 6.086957931518555 - score 0.9135 +2021-03-26 07:59:31,341 BAD EPOCHS (no improvement): 1 +2021-03-26 07:59:31,342 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:59:32,431 epoch 39 - iter 2/25 - loss 3.51137233 - samples/sec: 117.79 - lr: 0.125000 +2021-03-26 07:59:33,483 epoch 39 - iter 4/25 - loss 3.32509232 - samples/sec: 121.80 - lr: 0.125000 +2021-03-26 07:59:34,524 epoch 39 - iter 6/25 - loss 3.28555572 - samples/sec: 123.17 - lr: 0.125000 +2021-03-26 07:59:35,573 epoch 39 - iter 8/25 - loss 3.28893805 - samples/sec: 122.32 - lr: 0.125000 +2021-03-26 07:59:36,536 epoch 39 - iter 10/25 - loss 3.32282088 - samples/sec: 133.08 - lr: 0.125000 +2021-03-26 07:59:37,530 epoch 39 - iter 12/25 - loss 3.31090313 - samples/sec: 128.97 - lr: 0.125000 +2021-03-26 07:59:38,529 epoch 39 - iter 14/25 - loss 3.28282419 - samples/sec: 128.38 - lr: 0.125000 +2021-03-26 07:59:39,567 epoch 39 - iter 16/25 - loss 3.28705211 - samples/sec: 123.40 - lr: 0.125000 +2021-03-26 07:59:40,584 epoch 39 - iter 18/25 - loss 3.35787143 - samples/sec: 126.16 - lr: 0.125000 +2021-03-26 07:59:41,554 epoch 39 - iter 20/25 - loss 3.34916631 - samples/sec: 132.06 - lr: 0.125000 +2021-03-26 07:59:42,576 epoch 39 - iter 22/25 - loss 3.33695148 - samples/sec: 125.43 - lr: 0.125000 +2021-03-26 07:59:43,521 epoch 39 - iter 24/25 - loss 3.33044088 - samples/sec: 135.67 - lr: 0.125000 +2021-03-26 07:59:44,035 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:59:44,035 EPOCH 39 done: loss 3.3141 - lr 0.1250000 +2021-03-26 07:59:44,769 DEV : loss 6.12675666809082 - score 0.9118 +2021-03-26 07:59:44,788 BAD EPOCHS (no improvement): 2 +2021-03-26 07:59:44,789 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:59:45,848 epoch 40 - iter 2/25 - loss 3.39053595 - samples/sec: 121.01 - lr: 0.125000 +2021-03-26 07:59:46,885 epoch 40 - iter 4/25 - loss 3.01093823 - samples/sec: 123.67 - lr: 0.125000 +2021-03-26 07:59:47,829 epoch 40 - iter 6/25 - loss 3.07951188 - samples/sec: 135.76 - lr: 0.125000 +2021-03-26 07:59:48,870 epoch 40 - iter 8/25 - loss 2.95052671 - samples/sec: 123.12 - lr: 0.125000 +2021-03-26 07:59:49,803 epoch 40 - iter 10/25 - loss 2.97352629 - samples/sec: 137.51 - lr: 0.125000 +2021-03-26 07:59:50,717 epoch 40 - iter 12/25 - loss 3.00901592 - samples/sec: 140.26 - lr: 0.125000 +2021-03-26 07:59:51,599 epoch 40 - iter 14/25 - loss 3.02175573 - samples/sec: 145.35 - lr: 0.125000 +2021-03-26 07:59:52,584 epoch 40 - iter 16/25 - loss 3.04483742 - samples/sec: 130.12 - lr: 0.125000 +2021-03-26 07:59:53,524 epoch 40 - iter 18/25 - loss 3.16045962 - samples/sec: 136.38 - lr: 0.125000 +2021-03-26 07:59:54,635 epoch 40 - iter 20/25 - loss 3.21983838 - samples/sec: 115.39 - lr: 0.125000 +2021-03-26 07:59:55,686 epoch 40 - iter 22/25 - loss 3.30566324 - samples/sec: 122.03 - lr: 0.125000 +2021-03-26 07:59:56,596 epoch 40 - iter 24/25 - loss 3.26253485 - samples/sec: 140.78 - lr: 0.125000 +2021-03-26 07:59:56,945 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:59:56,945 EPOCH 40 done: loss 3.2575 - lr 0.1250000 +2021-03-26 07:59:57,658 DEV : loss 6.137569427490234 - score 0.9135 +2021-03-26 07:59:57,680 BAD EPOCHS (no improvement): 3 +2021-03-26 07:59:57,681 ---------------------------------------------------------------------------------------------------- +2021-03-26 07:59:58,718 epoch 41 - iter 2/25 - loss 3.11338151 - samples/sec: 123.60 - lr: 0.125000 +2021-03-26 07:59:59,662 epoch 41 - iter 4/25 - loss 2.86183566 - samples/sec: 135.82 - lr: 0.125000 +2021-03-26 08:00:00,674 epoch 41 - iter 6/25 - loss 3.09293191 - samples/sec: 126.66 - lr: 0.125000 +2021-03-26 08:00:01,587 epoch 41 - iter 8/25 - loss 3.08784479 - samples/sec: 140.40 - lr: 0.125000 +2021-03-26 08:00:02,515 epoch 41 - iter 10/25 - loss 3.09713953 - samples/sec: 138.11 - lr: 0.125000 +2021-03-26 08:00:03,456 epoch 41 - iter 12/25 - loss 3.03154357 - samples/sec: 136.29 - lr: 0.125000 +2021-03-26 08:00:04,461 epoch 41 - iter 14/25 - loss 3.16953162 - samples/sec: 127.60 - lr: 0.125000 +2021-03-26 08:00:05,458 epoch 41 - iter 16/25 - loss 3.24205971 - samples/sec: 128.47 - lr: 0.125000 +2021-03-26 08:00:06,382 epoch 41 - iter 18/25 - loss 3.22677610 - samples/sec: 138.77 - lr: 0.125000 +2021-03-26 08:00:07,273 epoch 41 - iter 20/25 - loss 3.19914433 - samples/sec: 143.88 - lr: 0.125000 +2021-03-26 08:00:08,260 epoch 41 - iter 22/25 - loss 3.25783063 - samples/sec: 129.85 - lr: 0.125000 +2021-03-26 08:00:09,198 epoch 41 - iter 24/25 - loss 3.25372606 - samples/sec: 136.70 - lr: 0.125000 +2021-03-26 08:00:09,540 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:00:09,540 EPOCH 41 done: loss 3.2544 - lr 0.1250000 +2021-03-26 08:00:10,220 DEV : loss 6.269647598266602 - score 0.9102 +2021-03-26 08:00:10,243 BAD EPOCHS (no improvement): 4 +2021-03-26 08:00:10,243 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:00:11,223 epoch 42 - iter 2/25 - loss 3.20560980 - samples/sec: 130.83 - lr: 0.062500 +2021-03-26 08:00:12,201 epoch 42 - iter 4/25 - loss 3.05283481 - samples/sec: 131.28 - lr: 0.062500 +2021-03-26 08:00:13,126 epoch 42 - iter 6/25 - loss 3.11910899 - samples/sec: 138.65 - lr: 0.062500 +2021-03-26 08:00:14,062 epoch 42 - iter 8/25 - loss 3.23691389 - samples/sec: 136.90 - lr: 0.062500 +2021-03-26 08:00:14,985 epoch 42 - iter 10/25 - loss 3.31064866 - samples/sec: 138.91 - lr: 0.062500 +2021-03-26 08:00:15,963 epoch 42 - iter 12/25 - loss 3.20677197 - samples/sec: 131.07 - lr: 0.062500 +2021-03-26 08:00:17,113 epoch 42 - iter 14/25 - loss 3.16082122 - samples/sec: 111.48 - lr: 0.062500 +2021-03-26 08:00:18,181 epoch 42 - iter 16/25 - loss 3.12908374 - samples/sec: 119.98 - lr: 0.062500 +2021-03-26 08:00:19,111 epoch 42 - iter 18/25 - loss 3.12907228 - samples/sec: 138.01 - lr: 0.062500 +2021-03-26 08:00:20,110 epoch 42 - iter 20/25 - loss 3.13966805 - samples/sec: 128.26 - lr: 0.062500 +2021-03-26 08:00:21,155 epoch 42 - iter 22/25 - loss 3.12461065 - samples/sec: 122.65 - lr: 0.062500 +2021-03-26 08:00:22,126 epoch 42 - iter 24/25 - loss 3.09273655 - samples/sec: 132.07 - lr: 0.062500 +2021-03-26 08:00:22,535 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:00:22,536 EPOCH 42 done: loss 3.1158 - lr 0.0625000 +2021-03-26 08:00:23,241 DEV : loss 6.14277982711792 - score 0.9118 +2021-03-26 08:00:23,264 BAD EPOCHS (no improvement): 1 +2021-03-26 08:00:23,265 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:00:24,222 epoch 43 - iter 2/25 - loss 3.33047795 - samples/sec: 134.00 - lr: 0.062500 +2021-03-26 08:00:25,191 epoch 43 - iter 4/25 - loss 3.21108902 - samples/sec: 132.35 - lr: 0.062500 +2021-03-26 08:00:26,208 epoch 43 - iter 6/25 - loss 3.19124313 - samples/sec: 126.03 - lr: 0.062500 +2021-03-26 08:00:27,152 epoch 43 - iter 8/25 - loss 3.31731814 - samples/sec: 135.76 - lr: 0.062500 +2021-03-26 08:00:28,168 epoch 43 - iter 10/25 - loss 3.35041995 - samples/sec: 126.20 - lr: 0.062500 +2021-03-26 08:00:29,203 epoch 43 - iter 12/25 - loss 3.29511603 - samples/sec: 123.81 - lr: 0.062500 +2021-03-26 08:00:30,155 epoch 43 - iter 14/25 - loss 3.16224260 - samples/sec: 134.72 - lr: 0.062500 +2021-03-26 08:00:31,046 epoch 43 - iter 16/25 - loss 3.12122820 - samples/sec: 143.92 - lr: 0.062500 +2021-03-26 08:00:32,078 epoch 43 - iter 18/25 - loss 3.08958398 - samples/sec: 124.21 - lr: 0.062500 +2021-03-26 08:00:33,229 epoch 43 - iter 20/25 - loss 3.06157826 - samples/sec: 111.36 - lr: 0.062500 +2021-03-26 08:00:34,129 epoch 43 - iter 22/25 - loss 3.06261855 - samples/sec: 142.44 - lr: 0.062500 +2021-03-26 08:00:35,160 epoch 43 - iter 24/25 - loss 3.03469709 - samples/sec: 124.45 - lr: 0.062500 +2021-03-26 08:00:35,526 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:00:35,526 EPOCH 43 done: loss 3.0090 - lr 0.0625000 +2021-03-26 08:00:36,251 DEV : loss 6.123751640319824 - score 0.9131 +2021-03-26 08:00:36,275 BAD EPOCHS (no improvement): 2 +2021-03-26 08:00:36,276 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:00:37,279 epoch 44 - iter 2/25 - loss 2.47428727 - samples/sec: 127.77 - lr: 0.062500 +2021-03-26 08:00:38,298 epoch 44 - iter 4/25 - loss 2.93294007 - samples/sec: 125.85 - lr: 0.062500 +2021-03-26 08:00:39,301 epoch 44 - iter 6/25 - loss 2.94476159 - samples/sec: 127.78 - lr: 0.062500 +2021-03-26 08:00:40,377 epoch 44 - iter 8/25 - loss 2.87063783 - samples/sec: 119.07 - lr: 0.062500 +2021-03-26 08:00:41,428 epoch 44 - iter 10/25 - loss 2.81950252 - samples/sec: 122.07 - lr: 0.062500 +2021-03-26 08:00:42,400 epoch 44 - iter 12/25 - loss 2.83043108 - samples/sec: 131.87 - lr: 0.062500 +2021-03-26 08:00:43,405 epoch 44 - iter 14/25 - loss 2.80339539 - samples/sec: 127.63 - lr: 0.062500 +2021-03-26 08:00:44,385 epoch 44 - iter 16/25 - loss 2.83290577 - samples/sec: 130.85 - lr: 0.062500 +2021-03-26 08:00:45,276 epoch 44 - iter 18/25 - loss 2.81380676 - samples/sec: 143.81 - lr: 0.062500 +2021-03-26 08:00:46,270 epoch 44 - iter 20/25 - loss 2.86733302 - samples/sec: 128.89 - lr: 0.062500 +2021-03-26 08:00:47,213 epoch 44 - iter 22/25 - loss 2.89061084 - samples/sec: 136.20 - lr: 0.062500 +2021-03-26 08:00:48,134 epoch 44 - iter 24/25 - loss 2.88428577 - samples/sec: 139.19 - lr: 0.062500 +2021-03-26 08:00:48,531 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:00:48,531 EPOCH 44 done: loss 2.9018 - lr 0.0625000 +2021-03-26 08:00:49,238 DEV : loss 6.177193641662598 - score 0.9131 +2021-03-26 08:00:49,261 BAD EPOCHS (no improvement): 3 +2021-03-26 08:00:49,262 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:00:50,201 epoch 45 - iter 2/25 - loss 3.11781955 - samples/sec: 136.46 - lr: 0.062500 +2021-03-26 08:00:51,263 epoch 45 - iter 4/25 - loss 3.27651954 - samples/sec: 120.78 - lr: 0.062500 +2021-03-26 08:00:52,184 epoch 45 - iter 6/25 - loss 3.09253220 - samples/sec: 139.15 - lr: 0.062500 +2021-03-26 08:00:53,203 epoch 45 - iter 8/25 - loss 3.09594426 - samples/sec: 125.75 - lr: 0.062500 +2021-03-26 08:00:54,148 epoch 45 - iter 10/25 - loss 3.06181715 - samples/sec: 135.70 - lr: 0.062500 +2021-03-26 08:00:55,128 epoch 45 - iter 12/25 - loss 3.08044499 - samples/sec: 130.72 - lr: 0.062500 +2021-03-26 08:00:56,165 epoch 45 - iter 14/25 - loss 3.07888399 - samples/sec: 123.64 - lr: 0.062500 +2021-03-26 08:00:57,333 epoch 45 - iter 16/25 - loss 3.10207799 - samples/sec: 109.79 - lr: 0.062500 +2021-03-26 08:00:58,353 epoch 45 - iter 18/25 - loss 3.08716683 - samples/sec: 125.76 - lr: 0.062500 +2021-03-26 08:00:59,401 epoch 45 - iter 20/25 - loss 3.07444774 - samples/sec: 122.40 - lr: 0.062500 +2021-03-26 08:01:00,443 epoch 45 - iter 22/25 - loss 3.08478190 - samples/sec: 122.96 - lr: 0.062500 +2021-03-26 08:01:01,516 epoch 45 - iter 24/25 - loss 3.18055204 - samples/sec: 119.48 - lr: 0.062500 +2021-03-26 08:01:01,929 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:01:01,930 EPOCH 45 done: loss 3.1790 - lr 0.0625000 +2021-03-26 08:01:02,641 DEV : loss 6.166326522827148 - score 0.9135 +2021-03-26 08:01:02,664 BAD EPOCHS (no improvement): 4 +2021-03-26 08:01:02,664 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:01:03,613 epoch 46 - iter 2/25 - loss 2.76693845 - samples/sec: 135.28 - lr: 0.031250 +2021-03-26 08:01:04,550 epoch 46 - iter 4/25 - loss 2.73128182 - samples/sec: 136.83 - lr: 0.031250 +2021-03-26 08:01:05,521 epoch 46 - iter 6/25 - loss 2.70524808 - samples/sec: 131.96 - lr: 0.031250 +2021-03-26 08:01:06,528 epoch 46 - iter 8/25 - loss 2.72330138 - samples/sec: 127.29 - lr: 0.031250 +2021-03-26 08:01:07,538 epoch 46 - iter 10/25 - loss 2.83866251 - samples/sec: 126.93 - lr: 0.031250 +2021-03-26 08:01:08,607 epoch 46 - iter 12/25 - loss 2.86050973 - samples/sec: 119.98 - lr: 0.031250 +2021-03-26 08:01:09,582 epoch 46 - iter 14/25 - loss 2.90982878 - samples/sec: 131.45 - lr: 0.031250 +2021-03-26 08:01:10,611 epoch 46 - iter 16/25 - loss 2.95056996 - samples/sec: 124.60 - lr: 0.031250 +2021-03-26 08:01:11,612 epoch 46 - iter 18/25 - loss 2.92731520 - samples/sec: 128.01 - lr: 0.031250 +2021-03-26 08:01:12,586 epoch 46 - iter 20/25 - loss 2.92244887 - samples/sec: 131.69 - lr: 0.031250 +2021-03-26 08:01:13,513 epoch 46 - iter 22/25 - loss 2.93590753 - samples/sec: 138.27 - lr: 0.031250 +2021-03-26 08:01:14,560 epoch 46 - iter 24/25 - loss 2.95312613 - samples/sec: 122.46 - lr: 0.031250 +2021-03-26 08:01:14,950 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:01:14,951 EPOCH 46 done: loss 2.9404 - lr 0.0312500 +2021-03-26 08:01:15,671 DEV : loss 6.119317054748535 - score 0.9156 +2021-03-26 08:01:15,687 BAD EPOCHS (no improvement): 1 +2021-03-26 08:01:15,687 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:01:16,674 epoch 47 - iter 2/25 - loss 3.48621786 - samples/sec: 129.82 - lr: 0.031250 +2021-03-26 08:01:17,701 epoch 47 - iter 4/25 - loss 3.34163475 - samples/sec: 124.98 - lr: 0.031250 +2021-03-26 08:01:18,782 epoch 47 - iter 6/25 - loss 3.01573094 - samples/sec: 118.47 - lr: 0.031250 +2021-03-26 08:01:19,807 epoch 47 - iter 8/25 - loss 2.88064113 - samples/sec: 125.08 - lr: 0.031250 +2021-03-26 08:01:20,787 epoch 47 - iter 10/25 - loss 2.93670418 - samples/sec: 131.25 - lr: 0.031250 +2021-03-26 08:01:21,677 epoch 47 - iter 12/25 - loss 2.78884999 - samples/sec: 144.15 - lr: 0.031250 +2021-03-26 08:01:22,824 epoch 47 - iter 14/25 - loss 2.83959506 - samples/sec: 111.78 - lr: 0.031250 +2021-03-26 08:01:23,840 epoch 47 - iter 16/25 - loss 2.85051008 - samples/sec: 126.15 - lr: 0.031250 +2021-03-26 08:01:24,859 epoch 47 - iter 18/25 - loss 2.89104986 - samples/sec: 125.78 - lr: 0.031250 +2021-03-26 08:01:25,912 epoch 47 - iter 20/25 - loss 2.84013094 - samples/sec: 121.77 - lr: 0.031250 +2021-03-26 08:01:26,946 epoch 47 - iter 22/25 - loss 2.81927793 - samples/sec: 123.89 - lr: 0.031250 +2021-03-26 08:01:27,855 epoch 47 - iter 24/25 - loss 2.78637945 - samples/sec: 141.13 - lr: 0.031250 +2021-03-26 08:01:28,295 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:01:28,295 EPOCH 47 done: loss 2.8195 - lr 0.0312500 +2021-03-26 08:01:29,019 DEV : loss 6.151057720184326 - score 0.9152 +2021-03-26 08:01:29,042 BAD EPOCHS (no improvement): 2 +2021-03-26 08:01:29,042 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:01:30,167 epoch 48 - iter 2/25 - loss 2.96697593 - samples/sec: 114.02 - lr: 0.031250 +2021-03-26 08:01:31,246 epoch 48 - iter 4/25 - loss 2.93493122 - samples/sec: 118.78 - lr: 0.031250 +2021-03-26 08:01:32,282 epoch 48 - iter 6/25 - loss 2.72235334 - samples/sec: 123.77 - lr: 0.031250 +2021-03-26 08:01:33,287 epoch 48 - iter 8/25 - loss 2.87266427 - samples/sec: 127.50 - lr: 0.031250 +2021-03-26 08:01:34,244 epoch 48 - iter 10/25 - loss 2.86259272 - samples/sec: 133.84 - lr: 0.031250 +2021-03-26 08:01:35,324 epoch 48 - iter 12/25 - loss 2.92353100 - samples/sec: 118.71 - lr: 0.031250 +2021-03-26 08:01:36,311 epoch 48 - iter 14/25 - loss 2.92250403 - samples/sec: 129.78 - lr: 0.031250 +2021-03-26 08:01:37,175 epoch 48 - iter 16/25 - loss 2.90623675 - samples/sec: 148.52 - lr: 0.031250 +2021-03-26 08:01:38,215 epoch 48 - iter 18/25 - loss 2.89371492 - samples/sec: 123.35 - lr: 0.031250 +2021-03-26 08:01:39,261 epoch 48 - iter 20/25 - loss 2.87912372 - samples/sec: 122.54 - lr: 0.031250 +2021-03-26 08:01:40,338 epoch 48 - iter 22/25 - loss 2.87841503 - samples/sec: 119.11 - lr: 0.031250 +2021-03-26 08:01:41,359 epoch 48 - iter 24/25 - loss 2.88370716 - samples/sec: 125.56 - lr: 0.031250 +2021-03-26 08:01:41,780 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:01:41,781 EPOCH 48 done: loss 2.9046 - lr 0.0312500 +2021-03-26 08:01:42,496 DEV : loss 6.098921775817871 - score 0.9169 +2021-03-26 08:01:42,519 BAD EPOCHS (no improvement): 0 +2021-03-26 08:01:51,972 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:01:52,991 epoch 49 - iter 2/25 - loss 2.45004141 - samples/sec: 125.84 - lr: 0.031250 +2021-03-26 08:01:53,911 epoch 49 - iter 4/25 - loss 2.69668263 - samples/sec: 139.29 - lr: 0.031250 +2021-03-26 08:01:54,987 epoch 49 - iter 6/25 - loss 2.57327108 - samples/sec: 119.17 - lr: 0.031250 +2021-03-26 08:01:55,986 epoch 49 - iter 8/25 - loss 2.64431378 - samples/sec: 128.25 - lr: 0.031250 +2021-03-26 08:01:57,033 epoch 49 - iter 10/25 - loss 2.68631110 - samples/sec: 122.35 - lr: 0.031250 +2021-03-26 08:01:57,998 epoch 49 - iter 12/25 - loss 2.64004058 - samples/sec: 132.92 - lr: 0.031250 +2021-03-26 08:01:58,999 epoch 49 - iter 14/25 - loss 2.69672627 - samples/sec: 127.97 - lr: 0.031250 +2021-03-26 08:01:59,938 epoch 49 - iter 16/25 - loss 2.72755112 - samples/sec: 136.53 - lr: 0.031250 +2021-03-26 08:02:00,883 epoch 49 - iter 18/25 - loss 2.75644163 - samples/sec: 135.63 - lr: 0.031250 +2021-03-26 08:02:01,870 epoch 49 - iter 20/25 - loss 2.72888288 - samples/sec: 129.82 - lr: 0.031250 +2021-03-26 08:02:02,864 epoch 49 - iter 22/25 - loss 2.73804910 - samples/sec: 128.98 - lr: 0.031250 +2021-03-26 08:02:03,872 epoch 49 - iter 24/25 - loss 2.72713549 - samples/sec: 127.16 - lr: 0.031250 +2021-03-26 08:02:04,248 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:02:04,249 EPOCH 49 done: loss 2.7030 - lr 0.0312500 +2021-03-26 08:02:04,966 DEV : loss 6.120596885681152 - score 0.9156 +2021-03-26 08:02:04,984 BAD EPOCHS (no improvement): 1 +2021-03-26 08:02:04,985 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:02:06,072 epoch 50 - iter 2/25 - loss 2.36916256 - samples/sec: 117.87 - lr: 0.031250 +2021-03-26 08:02:07,116 epoch 50 - iter 4/25 - loss 2.65234518 - samples/sec: 122.84 - lr: 0.031250 +2021-03-26 08:02:08,202 epoch 50 - iter 6/25 - loss 2.63142109 - samples/sec: 118.08 - lr: 0.031250 +2021-03-26 08:02:09,176 epoch 50 - iter 8/25 - loss 2.78051311 - samples/sec: 131.60 - lr: 0.031250 +2021-03-26 08:02:10,136 epoch 50 - iter 10/25 - loss 2.68362582 - samples/sec: 133.59 - lr: 0.031250 +2021-03-26 08:02:11,067 epoch 50 - iter 12/25 - loss 2.75772727 - samples/sec: 137.74 - lr: 0.031250 +2021-03-26 08:02:12,088 epoch 50 - iter 14/25 - loss 2.77727463 - samples/sec: 125.51 - lr: 0.031250 +2021-03-26 08:02:13,021 epoch 50 - iter 16/25 - loss 2.79809412 - samples/sec: 137.46 - lr: 0.031250 +2021-03-26 08:02:13,965 epoch 50 - iter 18/25 - loss 2.80761631 - samples/sec: 135.84 - lr: 0.031250 +2021-03-26 08:02:15,013 epoch 50 - iter 20/25 - loss 2.76133984 - samples/sec: 122.29 - lr: 0.031250 +2021-03-26 08:02:16,007 epoch 50 - iter 22/25 - loss 2.76965055 - samples/sec: 128.94 - lr: 0.031250 +2021-03-26 08:02:16,999 epoch 50 - iter 24/25 - loss 2.75284729 - samples/sec: 129.24 - lr: 0.031250 +2021-03-26 08:02:17,409 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:02:17,410 EPOCH 50 done: loss 2.7666 - lr 0.0312500 +2021-03-26 08:02:18,172 DEV : loss 6.149482727050781 - score 0.916 +2021-03-26 08:02:18,187 BAD EPOCHS (no improvement): 2 +2021-03-26 08:02:18,188 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:02:19,205 epoch 51 - iter 2/25 - loss 2.66018462 - samples/sec: 126.03 - lr: 0.031250 +2021-03-26 08:02:20,302 epoch 51 - iter 4/25 - loss 2.74374110 - samples/sec: 116.79 - lr: 0.031250 +2021-03-26 08:02:21,422 epoch 51 - iter 6/25 - loss 2.78392152 - samples/sec: 114.48 - lr: 0.031250 +2021-03-26 08:02:22,698 epoch 51 - iter 8/25 - loss 2.76540592 - samples/sec: 100.38 - lr: 0.031250 +2021-03-26 08:02:23,749 epoch 51 - iter 10/25 - loss 2.92016602 - samples/sec: 122.00 - lr: 0.031250 +2021-03-26 08:02:24,735 epoch 51 - iter 12/25 - loss 3.01396398 - samples/sec: 130.09 - lr: 0.031250 +2021-03-26 08:02:25,611 epoch 51 - iter 14/25 - loss 2.88940683 - samples/sec: 146.28 - lr: 0.031250 +2021-03-26 08:02:26,691 epoch 51 - iter 16/25 - loss 2.86734806 - samples/sec: 118.71 - lr: 0.031250 +2021-03-26 08:02:27,586 epoch 51 - iter 18/25 - loss 2.84448752 - samples/sec: 143.35 - lr: 0.031250 +2021-03-26 08:02:28,457 epoch 51 - iter 20/25 - loss 2.81145627 - samples/sec: 147.07 - lr: 0.031250 +2021-03-26 08:02:29,367 epoch 51 - iter 22/25 - loss 2.78330393 - samples/sec: 140.93 - lr: 0.031250 +2021-03-26 08:02:30,321 epoch 51 - iter 24/25 - loss 2.80717117 - samples/sec: 134.35 - lr: 0.031250 +2021-03-26 08:02:30,680 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:02:30,681 EPOCH 51 done: loss 2.8318 - lr 0.0312500 +2021-03-26 08:02:31,374 DEV : loss 6.155113220214844 - score 0.9152 +2021-03-26 08:02:31,388 BAD EPOCHS (no improvement): 3 +2021-03-26 08:02:31,389 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:02:32,238 epoch 52 - iter 2/25 - loss 2.47030675 - samples/sec: 151.04 - lr: 0.031250 +2021-03-26 08:02:34,188 epoch 52 - iter 4/25 - loss 2.87434727 - samples/sec: 65.69 - lr: 0.031250 +2021-03-26 08:02:35,209 epoch 52 - iter 6/25 - loss 2.79234389 - samples/sec: 125.59 - lr: 0.031250 +2021-03-26 08:02:36,172 epoch 52 - iter 8/25 - loss 2.85479537 - samples/sec: 133.17 - lr: 0.031250 +2021-03-26 08:02:37,181 epoch 52 - iter 10/25 - loss 2.77208908 - samples/sec: 127.03 - lr: 0.031250 +2021-03-26 08:02:38,167 epoch 52 - iter 12/25 - loss 2.76445206 - samples/sec: 130.04 - lr: 0.031250 +2021-03-26 08:02:39,119 epoch 52 - iter 14/25 - loss 2.73319456 - samples/sec: 134.67 - lr: 0.031250 +2021-03-26 08:02:40,021 epoch 52 - iter 16/25 - loss 2.71554077 - samples/sec: 142.15 - lr: 0.031250 +2021-03-26 08:02:41,002 epoch 52 - iter 18/25 - loss 2.74959408 - samples/sec: 130.58 - lr: 0.031250 +2021-03-26 08:02:42,042 epoch 52 - iter 20/25 - loss 2.73781605 - samples/sec: 123.32 - lr: 0.031250 +2021-03-26 08:02:43,116 epoch 52 - iter 22/25 - loss 2.72883092 - samples/sec: 119.23 - lr: 0.031250 +2021-03-26 08:02:44,163 epoch 52 - iter 24/25 - loss 2.74334789 - samples/sec: 122.47 - lr: 0.031250 +2021-03-26 08:02:44,598 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:02:44,598 EPOCH 52 done: loss 2.7554 - lr 0.0312500 +2021-03-26 08:02:45,314 DEV : loss 6.191333770751953 - score 0.9143 +2021-03-26 08:02:45,337 BAD EPOCHS (no improvement): 4 +2021-03-26 08:02:45,338 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:02:46,323 epoch 53 - iter 2/25 - loss 3.07710552 - samples/sec: 130.14 - lr: 0.015625 +2021-03-26 08:02:47,307 epoch 53 - iter 4/25 - loss 2.85961819 - samples/sec: 130.30 - lr: 0.015625 +2021-03-26 08:02:48,234 epoch 53 - iter 6/25 - loss 2.77078088 - samples/sec: 138.32 - lr: 0.015625 +2021-03-26 08:02:49,175 epoch 53 - iter 8/25 - loss 2.94273353 - samples/sec: 136.19 - lr: 0.015625 +2021-03-26 08:02:50,153 epoch 53 - iter 10/25 - loss 2.90624659 - samples/sec: 131.13 - lr: 0.015625 +2021-03-26 08:02:51,187 epoch 53 - iter 12/25 - loss 2.89880838 - samples/sec: 123.88 - lr: 0.015625 +2021-03-26 08:02:52,097 epoch 53 - iter 14/25 - loss 2.93234507 - samples/sec: 140.85 - lr: 0.015625 +2021-03-26 08:02:53,001 epoch 53 - iter 16/25 - loss 2.95822586 - samples/sec: 141.86 - lr: 0.015625 +2021-03-26 08:02:54,014 epoch 53 - iter 18/25 - loss 2.93325014 - samples/sec: 126.50 - lr: 0.015625 +2021-03-26 08:02:54,941 epoch 53 - iter 20/25 - loss 2.86297564 - samples/sec: 138.37 - lr: 0.015625 +2021-03-26 08:02:56,011 epoch 53 - iter 22/25 - loss 2.81225182 - samples/sec: 119.79 - lr: 0.015625 +2021-03-26 08:02:57,026 epoch 53 - iter 24/25 - loss 2.79774497 - samples/sec: 126.31 - lr: 0.015625 +2021-03-26 08:02:57,437 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:02:57,438 EPOCH 53 done: loss 2.8176 - lr 0.0156250 +2021-03-26 08:02:58,158 DEV : loss 6.180680751800537 - score 0.9139 +2021-03-26 08:02:58,182 BAD EPOCHS (no improvement): 1 +2021-03-26 08:02:58,183 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:02:59,191 epoch 54 - iter 2/25 - loss 2.82024026 - samples/sec: 127.11 - lr: 0.015625 +2021-03-26 08:03:00,137 epoch 54 - iter 4/25 - loss 2.84587026 - samples/sec: 135.55 - lr: 0.015625 +2021-03-26 08:03:01,214 epoch 54 - iter 6/25 - loss 3.08639145 - samples/sec: 119.01 - lr: 0.015625 +2021-03-26 08:03:02,171 epoch 54 - iter 8/25 - loss 3.04659450 - samples/sec: 133.97 - lr: 0.015625 +2021-03-26 08:03:03,099 epoch 54 - iter 10/25 - loss 2.94338441 - samples/sec: 138.21 - lr: 0.015625 +2021-03-26 08:03:04,121 epoch 54 - iter 12/25 - loss 2.86436790 - samples/sec: 125.34 - lr: 0.015625 +2021-03-26 08:03:05,144 epoch 54 - iter 14/25 - loss 2.85726164 - samples/sec: 125.38 - lr: 0.015625 +2021-03-26 08:03:06,139 epoch 54 - iter 16/25 - loss 2.84729083 - samples/sec: 128.86 - lr: 0.015625 +2021-03-26 08:03:07,054 epoch 54 - iter 18/25 - loss 2.87125883 - samples/sec: 140.14 - lr: 0.015625 +2021-03-26 08:03:07,957 epoch 54 - iter 20/25 - loss 2.86455238 - samples/sec: 142.02 - lr: 0.015625 +2021-03-26 08:03:08,839 epoch 54 - iter 22/25 - loss 2.85972881 - samples/sec: 145.24 - lr: 0.015625 +2021-03-26 08:03:09,997 epoch 54 - iter 24/25 - loss 2.88444576 - samples/sec: 110.71 - lr: 0.015625 +2021-03-26 08:03:10,551 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:03:10,552 EPOCH 54 done: loss 2.8656 - lr 0.0156250 +2021-03-26 08:03:11,251 DEV : loss 6.184042930603027 - score 0.9148 +2021-03-26 08:03:11,273 BAD EPOCHS (no improvement): 2 +2021-03-26 08:03:11,274 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:03:12,567 epoch 55 - iter 2/25 - loss 2.71724856 - samples/sec: 99.09 - lr: 0.015625 +2021-03-26 08:03:13,572 epoch 55 - iter 4/25 - loss 2.92992836 - samples/sec: 127.53 - lr: 0.015625 +2021-03-26 08:03:14,521 epoch 55 - iter 6/25 - loss 2.92324722 - samples/sec: 135.12 - lr: 0.015625 +2021-03-26 08:03:15,552 epoch 55 - iter 8/25 - loss 3.01632318 - samples/sec: 124.29 - lr: 0.015625 +2021-03-26 08:03:16,469 epoch 55 - iter 10/25 - loss 2.85993338 - samples/sec: 139.77 - lr: 0.015625 +2021-03-26 08:03:17,506 epoch 55 - iter 12/25 - loss 2.88023615 - samples/sec: 123.60 - lr: 0.015625 +2021-03-26 08:03:18,448 epoch 55 - iter 14/25 - loss 2.78017191 - samples/sec: 136.85 - lr: 0.015625 +2021-03-26 08:03:19,322 epoch 55 - iter 16/25 - loss 2.75286925 - samples/sec: 146.64 - lr: 0.015625 +2021-03-26 08:03:20,441 epoch 55 - iter 18/25 - loss 2.78123773 - samples/sec: 114.54 - lr: 0.015625 +2021-03-26 08:03:21,305 epoch 55 - iter 20/25 - loss 2.82595617 - samples/sec: 148.62 - lr: 0.015625 +2021-03-26 08:03:22,283 epoch 55 - iter 22/25 - loss 2.82012420 - samples/sec: 131.04 - lr: 0.015625 +2021-03-26 08:03:23,237 epoch 55 - iter 24/25 - loss 2.82870817 - samples/sec: 134.39 - lr: 0.015625 +2021-03-26 08:03:23,607 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:03:23,608 EPOCH 55 done: loss 2.8143 - lr 0.0156250 +2021-03-26 08:03:24,314 DEV : loss 6.195590972900391 - score 0.9152 +2021-03-26 08:03:24,337 BAD EPOCHS (no improvement): 3 +2021-03-26 08:03:24,338 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:03:25,307 epoch 56 - iter 2/25 - loss 2.88957417 - samples/sec: 132.41 - lr: 0.015625 +2021-03-26 08:03:26,310 epoch 56 - iter 4/25 - loss 2.73559302 - samples/sec: 127.86 - lr: 0.015625 +2021-03-26 08:03:27,334 epoch 56 - iter 6/25 - loss 2.70232844 - samples/sec: 125.13 - lr: 0.015625 +2021-03-26 08:03:28,243 epoch 56 - iter 8/25 - loss 2.69687900 - samples/sec: 141.08 - lr: 0.015625 +2021-03-26 08:03:29,265 epoch 56 - iter 10/25 - loss 2.80344183 - samples/sec: 125.43 - lr: 0.015625 +2021-03-26 08:03:30,362 epoch 56 - iter 12/25 - loss 2.83803695 - samples/sec: 116.77 - lr: 0.015625 +2021-03-26 08:03:31,314 epoch 56 - iter 14/25 - loss 2.82412973 - samples/sec: 134.73 - lr: 0.015625 +2021-03-26 08:03:32,293 epoch 56 - iter 16/25 - loss 2.85887678 - samples/sec: 130.91 - lr: 0.015625 +2021-03-26 08:03:33,367 epoch 56 - iter 18/25 - loss 2.81283089 - samples/sec: 119.29 - lr: 0.015625 +2021-03-26 08:03:34,344 epoch 56 - iter 20/25 - loss 2.76934352 - samples/sec: 131.22 - lr: 0.015625 +2021-03-26 08:03:35,274 epoch 56 - iter 22/25 - loss 2.78952987 - samples/sec: 137.91 - lr: 0.015625 +2021-03-26 08:03:36,304 epoch 56 - iter 24/25 - loss 2.85014056 - samples/sec: 124.53 - lr: 0.015625 +2021-03-26 08:03:36,703 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:03:36,704 EPOCH 56 done: loss 2.8169 - lr 0.0156250 +2021-03-26 08:03:37,402 DEV : loss 6.179908752441406 - score 0.916 +2021-03-26 08:03:37,425 BAD EPOCHS (no improvement): 4 +2021-03-26 08:03:37,426 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:03:38,402 epoch 57 - iter 2/25 - loss 2.89332473 - samples/sec: 131.40 - lr: 0.007812 +2021-03-26 08:03:39,368 epoch 57 - iter 4/25 - loss 2.77982998 - samples/sec: 132.66 - lr: 0.007812 +2021-03-26 08:03:40,401 epoch 57 - iter 6/25 - loss 2.71375469 - samples/sec: 124.09 - lr: 0.007812 +2021-03-26 08:03:41,394 epoch 57 - iter 8/25 - loss 2.68291500 - samples/sec: 129.03 - lr: 0.007812 +2021-03-26 08:03:42,516 epoch 57 - iter 10/25 - loss 2.70066304 - samples/sec: 114.27 - lr: 0.007812 +2021-03-26 08:03:43,523 epoch 57 - iter 12/25 - loss 2.73475238 - samples/sec: 127.31 - lr: 0.007812 +2021-03-26 08:03:44,454 epoch 57 - iter 14/25 - loss 2.73579529 - samples/sec: 137.72 - lr: 0.007812 +2021-03-26 08:03:45,423 epoch 57 - iter 16/25 - loss 2.73032914 - samples/sec: 132.24 - lr: 0.007812 +2021-03-26 08:03:46,456 epoch 57 - iter 18/25 - loss 2.72013121 - samples/sec: 124.20 - lr: 0.007812 +2021-03-26 08:03:47,330 epoch 57 - iter 20/25 - loss 2.70019326 - samples/sec: 146.77 - lr: 0.007812 +2021-03-26 08:03:48,316 epoch 57 - iter 22/25 - loss 2.70320966 - samples/sec: 129.97 - lr: 0.007812 +2021-03-26 08:03:49,287 epoch 57 - iter 24/25 - loss 2.71015591 - samples/sec: 132.05 - lr: 0.007812 +2021-03-26 08:03:49,677 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:03:49,677 EPOCH 57 done: loss 2.6899 - lr 0.0078125 +2021-03-26 08:03:50,422 DEV : loss 6.175540447235107 - score 0.9177 +2021-03-26 08:03:50,448 BAD EPOCHS (no improvement): 0 +2021-03-26 08:04:00,281 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:04:01,264 epoch 58 - iter 2/25 - loss 2.83420646 - samples/sec: 130.48 - lr: 0.007812 +2021-03-26 08:04:02,184 epoch 58 - iter 4/25 - loss 2.72172236 - samples/sec: 139.39 - lr: 0.007812 +2021-03-26 08:04:03,100 epoch 58 - iter 6/25 - loss 2.92447762 - samples/sec: 139.93 - lr: 0.007812 +2021-03-26 08:04:04,046 epoch 58 - iter 8/25 - loss 2.98371637 - samples/sec: 135.51 - lr: 0.007812 +2021-03-26 08:04:05,063 epoch 58 - iter 10/25 - loss 2.98575375 - samples/sec: 126.08 - lr: 0.007812 +2021-03-26 08:04:06,094 epoch 58 - iter 12/25 - loss 2.91445754 - samples/sec: 124.33 - lr: 0.007812 +2021-03-26 08:04:07,082 epoch 58 - iter 14/25 - loss 2.87455147 - samples/sec: 129.64 - lr: 0.007812 +2021-03-26 08:04:08,014 epoch 58 - iter 16/25 - loss 2.85320154 - samples/sec: 137.54 - lr: 0.007812 +2021-03-26 08:04:08,985 epoch 58 - iter 18/25 - loss 2.81956890 - samples/sec: 132.11 - lr: 0.007812 +2021-03-26 08:04:10,017 epoch 58 - iter 20/25 - loss 2.76832836 - samples/sec: 124.16 - lr: 0.007812 +2021-03-26 08:04:11,068 epoch 58 - iter 22/25 - loss 2.76364282 - samples/sec: 121.88 - lr: 0.007812 +2021-03-26 08:04:12,067 epoch 58 - iter 24/25 - loss 2.77961952 - samples/sec: 128.42 - lr: 0.007812 +2021-03-26 08:04:12,445 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:04:12,446 EPOCH 58 done: loss 2.7634 - lr 0.0078125 +2021-03-26 08:04:13,186 DEV : loss 6.17774772644043 - score 0.9173 +2021-03-26 08:04:13,209 BAD EPOCHS (no improvement): 1 +2021-03-26 08:04:13,211 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:04:14,205 epoch 59 - iter 2/25 - loss 2.55935884 - samples/sec: 128.95 - lr: 0.007812 +2021-03-26 08:04:15,168 epoch 59 - iter 4/25 - loss 2.47871357 - samples/sec: 133.11 - lr: 0.007812 +2021-03-26 08:04:16,134 epoch 59 - iter 6/25 - loss 2.69761129 - samples/sec: 132.82 - lr: 0.007812 +2021-03-26 08:04:17,298 epoch 59 - iter 8/25 - loss 2.87344724 - samples/sec: 110.09 - lr: 0.007812 +2021-03-26 08:04:18,418 epoch 59 - iter 10/25 - loss 2.79624567 - samples/sec: 114.41 - lr: 0.007812 +2021-03-26 08:04:19,577 epoch 59 - iter 12/25 - loss 2.82552556 - samples/sec: 111.41 - lr: 0.007812 +2021-03-26 08:04:20,704 epoch 59 - iter 14/25 - loss 2.79570598 - samples/sec: 114.87 - lr: 0.007812 +2021-03-26 08:04:21,623 epoch 59 - iter 16/25 - loss 2.77979054 - samples/sec: 140.65 - lr: 0.007812 +2021-03-26 08:04:22,655 epoch 59 - iter 18/25 - loss 2.79154040 - samples/sec: 124.19 - lr: 0.007812 +2021-03-26 08:04:23,594 epoch 59 - iter 20/25 - loss 2.77637383 - samples/sec: 136.43 - lr: 0.007812 +2021-03-26 08:04:24,473 epoch 59 - iter 22/25 - loss 2.73464512 - samples/sec: 145.86 - lr: 0.007812 +2021-03-26 08:04:25,341 epoch 59 - iter 24/25 - loss 2.72856198 - samples/sec: 147.73 - lr: 0.007812 +2021-03-26 08:04:25,719 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:04:25,721 EPOCH 59 done: loss 2.7133 - lr 0.0078125 +2021-03-26 08:04:26,418 DEV : loss 6.174241542816162 - score 0.9181 +2021-03-26 08:04:26,440 BAD EPOCHS (no improvement): 0 +2021-03-26 08:04:35,908 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:04:36,881 epoch 60 - iter 2/25 - loss 2.56185210 - samples/sec: 131.81 - lr: 0.007812 +2021-03-26 08:04:37,874 epoch 60 - iter 4/25 - loss 2.69779313 - samples/sec: 129.08 - lr: 0.007812 +2021-03-26 08:04:38,881 epoch 60 - iter 6/25 - loss 2.75501200 - samples/sec: 127.33 - lr: 0.007812 +2021-03-26 08:04:39,931 epoch 60 - iter 8/25 - loss 2.73785749 - samples/sec: 122.05 - lr: 0.007812 +2021-03-26 08:04:40,897 epoch 60 - iter 10/25 - loss 2.72844467 - samples/sec: 132.60 - lr: 0.007812 +2021-03-26 08:04:41,948 epoch 60 - iter 12/25 - loss 2.71255245 - samples/sec: 121.99 - lr: 0.007812 +2021-03-26 08:04:42,874 epoch 60 - iter 14/25 - loss 2.69912761 - samples/sec: 138.62 - lr: 0.007812 +2021-03-26 08:04:43,868 epoch 60 - iter 16/25 - loss 2.73890451 - samples/sec: 128.92 - lr: 0.007812 +2021-03-26 08:04:44,820 epoch 60 - iter 18/25 - loss 2.72212456 - samples/sec: 134.59 - lr: 0.007812 +2021-03-26 08:04:45,783 epoch 60 - iter 20/25 - loss 2.72613677 - samples/sec: 133.27 - lr: 0.007812 +2021-03-26 08:04:46,781 epoch 60 - iter 22/25 - loss 2.74840914 - samples/sec: 128.40 - lr: 0.007812 +2021-03-26 08:04:47,710 epoch 60 - iter 24/25 - loss 2.78334759 - samples/sec: 138.02 - lr: 0.007812 +2021-03-26 08:04:48,101 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:04:48,102 EPOCH 60 done: loss 2.7870 - lr 0.0078125 +2021-03-26 08:04:48,803 DEV : loss 6.180110454559326 - score 0.9177 +2021-03-26 08:04:48,825 BAD EPOCHS (no improvement): 1 +2021-03-26 08:04:48,826 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:04:49,736 epoch 61 - iter 2/25 - loss 2.79055202 - samples/sec: 141.03 - lr: 0.007812 +2021-03-26 08:04:50,681 epoch 61 - iter 4/25 - loss 2.93640441 - samples/sec: 135.57 - lr: 0.007812 +2021-03-26 08:04:51,615 epoch 61 - iter 6/25 - loss 2.99876702 - samples/sec: 137.22 - lr: 0.007812 +2021-03-26 08:04:52,511 epoch 61 - iter 8/25 - loss 2.97727868 - samples/sec: 143.18 - lr: 0.007812 +2021-03-26 08:04:53,917 epoch 61 - iter 10/25 - loss 2.98115492 - samples/sec: 91.08 - lr: 0.007812 +2021-03-26 08:04:54,899 epoch 61 - iter 12/25 - loss 2.97442589 - samples/sec: 130.56 - lr: 0.007812 +2021-03-26 08:04:55,853 epoch 61 - iter 14/25 - loss 2.91914528 - samples/sec: 134.43 - lr: 0.007812 +2021-03-26 08:04:56,908 epoch 61 - iter 16/25 - loss 2.90313707 - samples/sec: 121.50 - lr: 0.007812 +2021-03-26 08:04:57,821 epoch 61 - iter 18/25 - loss 2.86542445 - samples/sec: 140.31 - lr: 0.007812 +2021-03-26 08:04:58,881 epoch 61 - iter 20/25 - loss 2.82913233 - samples/sec: 121.06 - lr: 0.007812 +2021-03-26 08:04:59,827 epoch 61 - iter 22/25 - loss 2.79248063 - samples/sec: 135.50 - lr: 0.007812 +2021-03-26 08:05:00,801 epoch 61 - iter 24/25 - loss 2.83465045 - samples/sec: 131.53 - lr: 0.007812 +2021-03-26 08:05:01,418 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:05:01,419 EPOCH 61 done: loss 2.8220 - lr 0.0078125 +2021-03-26 08:05:02,124 DEV : loss 6.179595470428467 - score 0.9173 +2021-03-26 08:05:02,147 BAD EPOCHS (no improvement): 2 +2021-03-26 08:05:02,147 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:05:03,143 epoch 62 - iter 2/25 - loss 2.05444831 - samples/sec: 128.83 - lr: 0.007812 +2021-03-26 08:05:04,275 epoch 62 - iter 4/25 - loss 2.62164161 - samples/sec: 113.15 - lr: 0.007812 +2021-03-26 08:05:05,260 epoch 62 - iter 6/25 - loss 2.69296275 - samples/sec: 130.27 - lr: 0.007812 +2021-03-26 08:05:06,405 epoch 62 - iter 8/25 - loss 2.83039902 - samples/sec: 111.95 - lr: 0.007812 +2021-03-26 08:05:07,497 epoch 62 - iter 10/25 - loss 2.78501257 - samples/sec: 117.43 - lr: 0.007812 +2021-03-26 08:05:08,463 epoch 62 - iter 12/25 - loss 2.81995478 - samples/sec: 132.75 - lr: 0.007812 +2021-03-26 08:05:09,451 epoch 62 - iter 14/25 - loss 2.88364232 - samples/sec: 129.73 - lr: 0.007812 +2021-03-26 08:05:10,516 epoch 62 - iter 16/25 - loss 2.83740675 - samples/sec: 120.32 - lr: 0.007812 +2021-03-26 08:05:11,445 epoch 62 - iter 18/25 - loss 2.83892163 - samples/sec: 138.15 - lr: 0.007812 +2021-03-26 08:05:12,365 epoch 62 - iter 20/25 - loss 2.78212218 - samples/sec: 139.29 - lr: 0.007812 +2021-03-26 08:05:13,385 epoch 62 - iter 22/25 - loss 2.78157476 - samples/sec: 125.78 - lr: 0.007812 +2021-03-26 08:05:14,403 epoch 62 - iter 24/25 - loss 2.81738104 - samples/sec: 125.90 - lr: 0.007812 +2021-03-26 08:05:14,776 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:05:14,777 EPOCH 62 done: loss 2.8339 - lr 0.0078125 +2021-03-26 08:05:15,491 DEV : loss 6.171009063720703 - score 0.9181 +2021-03-26 08:05:15,514 BAD EPOCHS (no improvement): 0 +2021-03-26 08:05:24,932 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:05:25,826 epoch 63 - iter 2/25 - loss 2.51848078 - samples/sec: 143.50 - lr: 0.007812 +2021-03-26 08:05:26,829 epoch 63 - iter 4/25 - loss 2.60838878 - samples/sec: 127.81 - lr: 0.007812 +2021-03-26 08:05:27,777 epoch 63 - iter 6/25 - loss 2.60025974 - samples/sec: 135.22 - lr: 0.007812 +2021-03-26 08:05:28,670 epoch 63 - iter 8/25 - loss 2.63403827 - samples/sec: 143.40 - lr: 0.007812 +2021-03-26 08:05:29,666 epoch 63 - iter 10/25 - loss 2.80634141 - samples/sec: 128.72 - lr: 0.007812 +2021-03-26 08:05:30,645 epoch 63 - iter 12/25 - loss 2.77708697 - samples/sec: 131.04 - lr: 0.007812 +2021-03-26 08:05:31,602 epoch 63 - iter 14/25 - loss 2.71201353 - samples/sec: 134.04 - lr: 0.007812 +2021-03-26 08:05:32,581 epoch 63 - iter 16/25 - loss 2.72835907 - samples/sec: 130.96 - lr: 0.007812 +2021-03-26 08:05:33,721 epoch 63 - iter 18/25 - loss 2.72997316 - samples/sec: 112.34 - lr: 0.007812 +2021-03-26 08:05:34,621 epoch 63 - iter 20/25 - loss 2.71590858 - samples/sec: 142.39 - lr: 0.007812 +2021-03-26 08:05:35,778 epoch 63 - iter 22/25 - loss 2.73422451 - samples/sec: 110.81 - lr: 0.007812 +2021-03-26 08:05:36,865 epoch 63 - iter 24/25 - loss 2.73682607 - samples/sec: 117.96 - lr: 0.007812 +2021-03-26 08:05:37,286 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:05:37,287 EPOCH 63 done: loss 2.7471 - lr 0.0078125 +2021-03-26 08:05:38,067 DEV : loss 6.168983459472656 - score 0.9173 +2021-03-26 08:05:38,085 BAD EPOCHS (no improvement): 1 +2021-03-26 08:05:38,085 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:05:39,141 epoch 64 - iter 2/25 - loss 3.04874635 - samples/sec: 121.39 - lr: 0.007812 +2021-03-26 08:05:40,160 epoch 64 - iter 4/25 - loss 3.02451402 - samples/sec: 125.84 - lr: 0.007812 +2021-03-26 08:05:41,141 epoch 64 - iter 6/25 - loss 3.02157851 - samples/sec: 130.54 - lr: 0.007812 +2021-03-26 08:05:42,156 epoch 64 - iter 8/25 - loss 3.11041513 - samples/sec: 126.40 - lr: 0.007812 +2021-03-26 08:05:43,227 epoch 64 - iter 10/25 - loss 3.03794565 - samples/sec: 119.70 - lr: 0.007812 +2021-03-26 08:05:44,173 epoch 64 - iter 12/25 - loss 2.95237716 - samples/sec: 135.52 - lr: 0.007812 +2021-03-26 08:05:45,163 epoch 64 - iter 14/25 - loss 2.82234809 - samples/sec: 129.54 - lr: 0.007812 +2021-03-26 08:05:46,188 epoch 64 - iter 16/25 - loss 2.79835985 - samples/sec: 124.99 - lr: 0.007812 +2021-03-26 08:05:47,164 epoch 64 - iter 18/25 - loss 2.83342325 - samples/sec: 131.35 - lr: 0.007812 +2021-03-26 08:05:48,081 epoch 64 - iter 20/25 - loss 2.77793549 - samples/sec: 139.74 - lr: 0.007812 +2021-03-26 08:05:48,993 epoch 64 - iter 22/25 - loss 2.76137200 - samples/sec: 140.61 - lr: 0.007812 +2021-03-26 08:05:50,009 epoch 64 - iter 24/25 - loss 2.81117615 - samples/sec: 126.14 - lr: 0.007812 +2021-03-26 08:05:50,467 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:05:50,468 EPOCH 64 done: loss 2.8223 - lr 0.0078125 +2021-03-26 08:05:51,173 DEV : loss 6.175807952880859 - score 0.9173 +2021-03-26 08:05:51,189 BAD EPOCHS (no improvement): 2 +2021-03-26 08:05:51,190 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:05:52,179 epoch 65 - iter 2/25 - loss 2.65639818 - samples/sec: 129.64 - lr: 0.007812 +2021-03-26 08:05:53,178 epoch 65 - iter 4/25 - loss 2.82343382 - samples/sec: 128.24 - lr: 0.007812 +2021-03-26 08:05:54,222 epoch 65 - iter 6/25 - loss 2.72004970 - samples/sec: 122.73 - lr: 0.007812 +2021-03-26 08:05:55,260 epoch 65 - iter 8/25 - loss 2.67372179 - samples/sec: 123.52 - lr: 0.007812 +2021-03-26 08:05:56,283 epoch 65 - iter 10/25 - loss 2.58629749 - samples/sec: 125.36 - lr: 0.007812 +2021-03-26 08:05:57,308 epoch 65 - iter 12/25 - loss 2.52548907 - samples/sec: 124.96 - lr: 0.007812 +2021-03-26 08:05:58,310 epoch 65 - iter 14/25 - loss 2.55922290 - samples/sec: 127.94 - lr: 0.007812 +2021-03-26 08:05:59,198 epoch 65 - iter 16/25 - loss 2.51116529 - samples/sec: 144.34 - lr: 0.007812 +2021-03-26 08:06:00,151 epoch 65 - iter 18/25 - loss 2.54416679 - samples/sec: 134.58 - lr: 0.007812 +2021-03-26 08:06:01,081 epoch 65 - iter 20/25 - loss 2.59224055 - samples/sec: 137.88 - lr: 0.007812 +2021-03-26 08:06:02,018 epoch 65 - iter 22/25 - loss 2.63312927 - samples/sec: 136.79 - lr: 0.007812 +2021-03-26 08:06:03,002 epoch 65 - iter 24/25 - loss 2.68356836 - samples/sec: 130.23 - lr: 0.007812 +2021-03-26 08:06:03,393 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:06:03,393 EPOCH 65 done: loss 2.6666 - lr 0.0078125 +2021-03-26 08:06:04,119 DEV : loss 6.184266090393066 - score 0.9177 +2021-03-26 08:06:04,141 BAD EPOCHS (no improvement): 3 +2021-03-26 08:06:04,142 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:06:05,166 epoch 66 - iter 2/25 - loss 3.00271451 - samples/sec: 125.29 - lr: 0.007812 +2021-03-26 08:06:06,190 epoch 66 - iter 4/25 - loss 2.93223822 - samples/sec: 125.16 - lr: 0.007812 +2021-03-26 08:06:07,112 epoch 66 - iter 6/25 - loss 2.83356301 - samples/sec: 138.97 - lr: 0.007812 +2021-03-26 08:06:08,284 epoch 66 - iter 8/25 - loss 2.68446249 - samples/sec: 109.43 - lr: 0.007812 +2021-03-26 08:06:09,288 epoch 66 - iter 10/25 - loss 2.81322050 - samples/sec: 127.55 - lr: 0.007812 +2021-03-26 08:06:10,237 epoch 66 - iter 12/25 - loss 2.79783382 - samples/sec: 135.02 - lr: 0.007812 +2021-03-26 08:06:11,231 epoch 66 - iter 14/25 - loss 2.82855047 - samples/sec: 129.01 - lr: 0.007812 +2021-03-26 08:06:12,211 epoch 66 - iter 16/25 - loss 2.82480106 - samples/sec: 130.92 - lr: 0.007812 +2021-03-26 08:06:13,238 epoch 66 - iter 18/25 - loss 2.79299916 - samples/sec: 124.73 - lr: 0.007812 +2021-03-26 08:06:14,184 epoch 66 - iter 20/25 - loss 2.84182227 - samples/sec: 135.54 - lr: 0.007812 +2021-03-26 08:06:15,108 epoch 66 - iter 22/25 - loss 2.86088503 - samples/sec: 138.71 - lr: 0.007812 +2021-03-26 08:06:16,117 epoch 66 - iter 24/25 - loss 2.83759779 - samples/sec: 127.06 - lr: 0.007812 +2021-03-26 08:06:16,548 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:06:16,548 EPOCH 66 done: loss 2.8567 - lr 0.0078125 +2021-03-26 08:06:17,275 DEV : loss 6.17871618270874 - score 0.9173 +2021-03-26 08:06:17,291 BAD EPOCHS (no improvement): 4 +2021-03-26 08:06:17,291 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:06:18,335 epoch 67 - iter 2/25 - loss 2.24417460 - samples/sec: 122.80 - lr: 0.003906 +2021-03-26 08:06:19,355 epoch 67 - iter 4/25 - loss 2.76672024 - samples/sec: 125.73 - lr: 0.003906 +2021-03-26 08:06:20,368 epoch 67 - iter 6/25 - loss 2.68564645 - samples/sec: 126.53 - lr: 0.003906 +2021-03-26 08:06:21,420 epoch 67 - iter 8/25 - loss 2.79277220 - samples/sec: 121.91 - lr: 0.003906 +2021-03-26 08:06:22,371 epoch 67 - iter 10/25 - loss 2.75309818 - samples/sec: 134.78 - lr: 0.003906 +2021-03-26 08:06:23,376 epoch 67 - iter 12/25 - loss 2.65836643 - samples/sec: 127.70 - lr: 0.003906 +2021-03-26 08:06:24,344 epoch 67 - iter 14/25 - loss 2.62789102 - samples/sec: 132.35 - lr: 0.003906 +2021-03-26 08:06:25,422 epoch 67 - iter 16/25 - loss 2.64878156 - samples/sec: 118.85 - lr: 0.003906 +2021-03-26 08:06:26,483 epoch 67 - iter 18/25 - loss 2.72514617 - samples/sec: 120.85 - lr: 0.003906 +2021-03-26 08:06:27,484 epoch 67 - iter 20/25 - loss 2.79397482 - samples/sec: 127.98 - lr: 0.003906 +2021-03-26 08:06:28,437 epoch 67 - iter 22/25 - loss 2.76995759 - samples/sec: 134.53 - lr: 0.003906 +2021-03-26 08:06:29,334 epoch 67 - iter 24/25 - loss 2.79965747 - samples/sec: 142.84 - lr: 0.003906 +2021-03-26 08:06:29,777 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:06:29,778 EPOCH 67 done: loss 2.7726 - lr 0.0039062 +2021-03-26 08:06:30,502 DEV : loss 6.179577827453613 - score 0.9173 +2021-03-26 08:06:30,523 BAD EPOCHS (no improvement): 1 +2021-03-26 08:06:30,524 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:06:31,504 epoch 68 - iter 2/25 - loss 3.11455142 - samples/sec: 131.01 - lr: 0.003906 +2021-03-26 08:06:32,431 epoch 68 - iter 4/25 - loss 2.82809067 - samples/sec: 138.15 - lr: 0.003906 +2021-03-26 08:06:33,366 epoch 68 - iter 6/25 - loss 2.72769392 - samples/sec: 137.20 - lr: 0.003906 +2021-03-26 08:06:34,286 epoch 68 - iter 8/25 - loss 2.65765330 - samples/sec: 139.28 - lr: 0.003906 +2021-03-26 08:06:35,218 epoch 68 - iter 10/25 - loss 2.73607638 - samples/sec: 137.49 - lr: 0.003906 +2021-03-26 08:06:36,254 epoch 68 - iter 12/25 - loss 2.71418903 - samples/sec: 123.80 - lr: 0.003906 +2021-03-26 08:06:37,189 epoch 68 - iter 14/25 - loss 2.77073138 - samples/sec: 137.11 - lr: 0.003906 +2021-03-26 08:06:38,260 epoch 68 - iter 16/25 - loss 2.78171870 - samples/sec: 119.64 - lr: 0.003906 +2021-03-26 08:06:39,270 epoch 68 - iter 18/25 - loss 2.77806143 - samples/sec: 126.91 - lr: 0.003906 +2021-03-26 08:06:40,250 epoch 68 - iter 20/25 - loss 2.75635051 - samples/sec: 130.87 - lr: 0.003906 +2021-03-26 08:06:41,216 epoch 68 - iter 22/25 - loss 2.78475171 - samples/sec: 132.71 - lr: 0.003906 +2021-03-26 08:06:42,157 epoch 68 - iter 24/25 - loss 2.78028412 - samples/sec: 136.27 - lr: 0.003906 +2021-03-26 08:06:42,559 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:06:42,560 EPOCH 68 done: loss 2.8145 - lr 0.0039062 +2021-03-26 08:06:43,266 DEV : loss 6.189845085144043 - score 0.9177 +2021-03-26 08:06:43,286 BAD EPOCHS (no improvement): 2 +2021-03-26 08:06:43,286 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:06:44,309 epoch 69 - iter 2/25 - loss 2.54150748 - samples/sec: 125.31 - lr: 0.003906 +2021-03-26 08:06:45,311 epoch 69 - iter 4/25 - loss 2.69912976 - samples/sec: 127.93 - lr: 0.003906 +2021-03-26 08:06:46,286 epoch 69 - iter 6/25 - loss 2.71940128 - samples/sec: 131.57 - lr: 0.003906 +2021-03-26 08:06:47,311 epoch 69 - iter 8/25 - loss 2.73294532 - samples/sec: 125.02 - lr: 0.003906 +2021-03-26 08:06:48,291 epoch 69 - iter 10/25 - loss 2.77171984 - samples/sec: 130.84 - lr: 0.003906 +2021-03-26 08:06:49,182 epoch 69 - iter 12/25 - loss 2.85729210 - samples/sec: 143.80 - lr: 0.003906 +2021-03-26 08:06:50,113 epoch 69 - iter 14/25 - loss 2.84100824 - samples/sec: 137.66 - lr: 0.003906 +2021-03-26 08:06:51,160 epoch 69 - iter 16/25 - loss 2.81065214 - samples/sec: 122.42 - lr: 0.003906 +2021-03-26 08:06:52,056 epoch 69 - iter 18/25 - loss 2.79097626 - samples/sec: 143.06 - lr: 0.003906 +2021-03-26 08:06:53,085 epoch 69 - iter 20/25 - loss 2.80775127 - samples/sec: 124.57 - lr: 0.003906 +2021-03-26 08:06:53,990 epoch 69 - iter 22/25 - loss 2.77170913 - samples/sec: 141.60 - lr: 0.003906 +2021-03-26 08:06:55,044 epoch 69 - iter 24/25 - loss 2.81093141 - samples/sec: 121.61 - lr: 0.003906 +2021-03-26 08:06:55,481 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:06:55,483 EPOCH 69 done: loss 2.8364 - lr 0.0039062 +2021-03-26 08:06:56,219 DEV : loss 6.186550140380859 - score 0.9177 +2021-03-26 08:06:56,242 BAD EPOCHS (no improvement): 3 +2021-03-26 08:06:56,243 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:06:57,225 epoch 70 - iter 2/25 - loss 2.53426838 - samples/sec: 130.52 - lr: 0.003906 +2021-03-26 08:06:58,243 epoch 70 - iter 4/25 - loss 2.63942099 - samples/sec: 125.89 - lr: 0.003906 +2021-03-26 08:06:59,312 epoch 70 - iter 6/25 - loss 2.77118782 - samples/sec: 119.87 - lr: 0.003906 +2021-03-26 08:07:00,331 epoch 70 - iter 8/25 - loss 2.83780730 - samples/sec: 125.88 - lr: 0.003906 +2021-03-26 08:07:01,269 epoch 70 - iter 10/25 - loss 2.77150600 - samples/sec: 136.62 - lr: 0.003906 +2021-03-26 08:07:02,208 epoch 70 - iter 12/25 - loss 2.68753244 - samples/sec: 136.58 - lr: 0.003906 +2021-03-26 08:07:03,134 epoch 70 - iter 14/25 - loss 2.68498324 - samples/sec: 138.39 - lr: 0.003906 +2021-03-26 08:07:04,030 epoch 70 - iter 16/25 - loss 2.81244020 - samples/sec: 143.03 - lr: 0.003906 +2021-03-26 08:07:05,001 epoch 70 - iter 18/25 - loss 2.85790327 - samples/sec: 132.05 - lr: 0.003906 +2021-03-26 08:07:05,958 epoch 70 - iter 20/25 - loss 2.80446895 - samples/sec: 134.12 - lr: 0.003906 +2021-03-26 08:07:06,852 epoch 70 - iter 22/25 - loss 2.78271670 - samples/sec: 143.38 - lr: 0.003906 +2021-03-26 08:07:07,821 epoch 70 - iter 24/25 - loss 2.80982100 - samples/sec: 132.17 - lr: 0.003906 +2021-03-26 08:07:08,261 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:07:08,262 EPOCH 70 done: loss 2.8286 - lr 0.0039062 +2021-03-26 08:07:09,004 DEV : loss 6.186135768890381 - score 0.9177 +2021-03-26 08:07:09,027 BAD EPOCHS (no improvement): 4 +2021-03-26 08:07:09,028 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:07:10,094 epoch 71 - iter 2/25 - loss 2.33365417 - samples/sec: 120.27 - lr: 0.001953 +2021-03-26 08:07:11,122 epoch 71 - iter 4/25 - loss 2.40433973 - samples/sec: 124.67 - lr: 0.001953 +2021-03-26 08:07:12,127 epoch 71 - iter 6/25 - loss 2.42733010 - samples/sec: 127.63 - lr: 0.001953 +2021-03-26 08:07:13,177 epoch 71 - iter 8/25 - loss 2.58082166 - samples/sec: 122.06 - lr: 0.001953 +2021-03-26 08:07:14,222 epoch 71 - iter 10/25 - loss 2.59674971 - samples/sec: 122.72 - lr: 0.001953 +2021-03-26 08:07:15,215 epoch 71 - iter 12/25 - loss 2.59080458 - samples/sec: 129.06 - lr: 0.001953 +2021-03-26 08:07:16,220 epoch 71 - iter 14/25 - loss 2.58698351 - samples/sec: 127.62 - lr: 0.001953 +2021-03-26 08:07:17,080 epoch 71 - iter 16/25 - loss 2.55493875 - samples/sec: 149.07 - lr: 0.001953 +2021-03-26 08:07:18,053 epoch 71 - iter 18/25 - loss 2.53936093 - samples/sec: 131.78 - lr: 0.001953 +2021-03-26 08:07:19,048 epoch 71 - iter 20/25 - loss 2.60882049 - samples/sec: 128.90 - lr: 0.001953 +2021-03-26 08:07:20,112 epoch 71 - iter 22/25 - loss 2.62530265 - samples/sec: 120.51 - lr: 0.001953 +2021-03-26 08:07:21,064 epoch 71 - iter 24/25 - loss 2.63414593 - samples/sec: 134.58 - lr: 0.001953 +2021-03-26 08:07:21,479 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:07:21,480 EPOCH 71 done: loss 2.6570 - lr 0.0019531 +2021-03-26 08:07:22,193 DEV : loss 6.187526226043701 - score 0.9177 +2021-03-26 08:07:22,215 BAD EPOCHS (no improvement): 1 +2021-03-26 08:07:22,215 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:07:23,173 epoch 72 - iter 2/25 - loss 2.70109022 - samples/sec: 133.91 - lr: 0.001953 +2021-03-26 08:07:24,108 epoch 72 - iter 4/25 - loss 2.65733844 - samples/sec: 137.20 - lr: 0.001953 +2021-03-26 08:07:25,165 epoch 72 - iter 6/25 - loss 2.72642616 - samples/sec: 121.24 - lr: 0.001953 +2021-03-26 08:07:26,224 epoch 72 - iter 8/25 - loss 2.78701332 - samples/sec: 121.09 - lr: 0.001953 +2021-03-26 08:07:27,335 epoch 72 - iter 10/25 - loss 2.74920893 - samples/sec: 115.34 - lr: 0.001953 +2021-03-26 08:07:28,286 epoch 72 - iter 12/25 - loss 2.75116277 - samples/sec: 134.73 - lr: 0.001953 +2021-03-26 08:07:29,419 epoch 72 - iter 14/25 - loss 2.75593237 - samples/sec: 113.16 - lr: 0.001953 +2021-03-26 08:07:30,383 epoch 72 - iter 16/25 - loss 2.71642062 - samples/sec: 132.93 - lr: 0.001953 +2021-03-26 08:07:31,441 epoch 72 - iter 18/25 - loss 2.77816828 - samples/sec: 121.12 - lr: 0.001953 +2021-03-26 08:07:32,400 epoch 72 - iter 20/25 - loss 2.77651918 - samples/sec: 133.69 - lr: 0.001953 +2021-03-26 08:07:33,476 epoch 72 - iter 22/25 - loss 2.75537030 - samples/sec: 119.09 - lr: 0.001953 +2021-03-26 08:07:34,482 epoch 72 - iter 24/25 - loss 2.74449572 - samples/sec: 127.49 - lr: 0.001953 +2021-03-26 08:07:34,909 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:07:34,910 EPOCH 72 done: loss 2.6977 - lr 0.0019531 +2021-03-26 08:07:35,639 DEV : loss 6.187828063964844 - score 0.9177 +2021-03-26 08:07:35,662 BAD EPOCHS (no improvement): 2 +2021-03-26 08:07:35,663 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:07:36,711 epoch 73 - iter 2/25 - loss 2.32378674 - samples/sec: 122.32 - lr: 0.001953 +2021-03-26 08:07:37,684 epoch 73 - iter 4/25 - loss 2.64039457 - samples/sec: 131.82 - lr: 0.001953 +2021-03-26 08:07:38,673 epoch 73 - iter 6/25 - loss 2.71906400 - samples/sec: 129.55 - lr: 0.001953 +2021-03-26 08:07:39,577 epoch 73 - iter 8/25 - loss 2.76447648 - samples/sec: 141.91 - lr: 0.001953 +2021-03-26 08:07:40,511 epoch 73 - iter 10/25 - loss 2.75839403 - samples/sec: 137.37 - lr: 0.001953 +2021-03-26 08:07:41,443 epoch 73 - iter 12/25 - loss 2.68266495 - samples/sec: 137.44 - lr: 0.001953 +2021-03-26 08:07:42,503 epoch 73 - iter 14/25 - loss 2.70562822 - samples/sec: 120.99 - lr: 0.001953 +2021-03-26 08:07:43,378 epoch 73 - iter 16/25 - loss 2.63112544 - samples/sec: 146.44 - lr: 0.001953 +2021-03-26 08:07:44,439 epoch 73 - iter 18/25 - loss 2.65143849 - samples/sec: 120.82 - lr: 0.001953 +2021-03-26 08:07:45,419 epoch 73 - iter 20/25 - loss 2.64402987 - samples/sec: 130.89 - lr: 0.001953 +2021-03-26 08:07:46,403 epoch 73 - iter 22/25 - loss 2.63108562 - samples/sec: 130.37 - lr: 0.001953 +2021-03-26 08:07:47,380 epoch 73 - iter 24/25 - loss 2.65545537 - samples/sec: 131.20 - lr: 0.001953 +2021-03-26 08:07:47,805 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:07:47,806 EPOCH 73 done: loss 2.6644 - lr 0.0019531 +2021-03-26 08:07:48,505 DEV : loss 6.188020706176758 - score 0.9177 +2021-03-26 08:07:48,528 BAD EPOCHS (no improvement): 3 +2021-03-26 08:07:48,528 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:07:49,569 epoch 74 - iter 2/25 - loss 2.36066926 - samples/sec: 123.27 - lr: 0.001953 +2021-03-26 08:07:50,563 epoch 74 - iter 4/25 - loss 2.84606272 - samples/sec: 128.86 - lr: 0.001953 +2021-03-26 08:07:51,476 epoch 74 - iter 6/25 - loss 2.78872546 - samples/sec: 140.41 - lr: 0.001953 +2021-03-26 08:07:52,518 epoch 74 - iter 8/25 - loss 2.83456469 - samples/sec: 123.13 - lr: 0.001953 +2021-03-26 08:07:53,486 epoch 74 - iter 10/25 - loss 2.81370616 - samples/sec: 132.30 - lr: 0.001953 +2021-03-26 08:07:54,524 epoch 74 - iter 12/25 - loss 2.74629303 - samples/sec: 123.59 - lr: 0.001953 +2021-03-26 08:07:55,514 epoch 74 - iter 14/25 - loss 2.76375324 - samples/sec: 129.46 - lr: 0.001953 +2021-03-26 08:07:56,490 epoch 74 - iter 16/25 - loss 2.70198634 - samples/sec: 131.30 - lr: 0.001953 +2021-03-26 08:07:57,435 epoch 74 - iter 18/25 - loss 2.71382519 - samples/sec: 135.74 - lr: 0.001953 +2021-03-26 08:07:58,519 epoch 74 - iter 20/25 - loss 2.77547271 - samples/sec: 118.26 - lr: 0.001953 +2021-03-26 08:07:59,485 epoch 74 - iter 22/25 - loss 2.77600351 - samples/sec: 132.73 - lr: 0.001953 +2021-03-26 08:08:00,557 epoch 74 - iter 24/25 - loss 2.77583278 - samples/sec: 119.49 - lr: 0.001953 +2021-03-26 08:08:00,907 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:08:00,908 EPOCH 74 done: loss 2.7614 - lr 0.0019531 +2021-03-26 08:08:01,613 DEV : loss 6.188040733337402 - score 0.9173 +2021-03-26 08:08:01,636 BAD EPOCHS (no improvement): 4 +2021-03-26 08:08:01,637 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:08:02,603 epoch 75 - iter 2/25 - loss 2.41651249 - samples/sec: 132.80 - lr: 0.000977 +2021-03-26 08:08:03,626 epoch 75 - iter 4/25 - loss 2.59921521 - samples/sec: 125.28 - lr: 0.000977 +2021-03-26 08:08:04,561 epoch 75 - iter 6/25 - loss 2.52756298 - samples/sec: 137.11 - lr: 0.000977 +2021-03-26 08:08:05,673 epoch 75 - iter 8/25 - loss 2.70598119 - samples/sec: 115.34 - lr: 0.000977 +2021-03-26 08:08:06,639 epoch 75 - iter 10/25 - loss 2.77697103 - samples/sec: 132.71 - lr: 0.000977 +2021-03-26 08:08:07,572 epoch 75 - iter 12/25 - loss 2.78852485 - samples/sec: 137.43 - lr: 0.000977 +2021-03-26 08:08:08,576 epoch 75 - iter 14/25 - loss 2.77042481 - samples/sec: 127.69 - lr: 0.000977 +2021-03-26 08:08:09,662 epoch 75 - iter 16/25 - loss 2.73992635 - samples/sec: 118.18 - lr: 0.000977 +2021-03-26 08:08:10,681 epoch 75 - iter 18/25 - loss 2.74360679 - samples/sec: 125.79 - lr: 0.000977 +2021-03-26 08:08:11,594 epoch 75 - iter 20/25 - loss 2.77423202 - samples/sec: 140.46 - lr: 0.000977 +2021-03-26 08:08:12,533 epoch 75 - iter 22/25 - loss 2.75198400 - samples/sec: 136.51 - lr: 0.000977 +2021-03-26 08:08:13,464 epoch 75 - iter 24/25 - loss 2.76625696 - samples/sec: 137.79 - lr: 0.000977 +2021-03-26 08:08:13,907 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:08:13,907 EPOCH 75 done: loss 2.7788 - lr 0.0009766 +2021-03-26 08:08:14,631 DEV : loss 6.188265800476074 - score 0.9173 +2021-03-26 08:08:14,654 BAD EPOCHS (no improvement): 1 +2021-03-26 08:08:14,655 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:08:15,657 epoch 76 - iter 2/25 - loss 3.10712671 - samples/sec: 127.94 - lr: 0.000977 +2021-03-26 08:08:16,666 epoch 76 - iter 4/25 - loss 3.13716650 - samples/sec: 127.06 - lr: 0.000977 +2021-03-26 08:08:17,714 epoch 76 - iter 6/25 - loss 3.12287060 - samples/sec: 122.41 - lr: 0.000977 +2021-03-26 08:08:18,718 epoch 76 - iter 8/25 - loss 2.90880463 - samples/sec: 127.72 - lr: 0.000977 +2021-03-26 08:08:19,676 epoch 76 - iter 10/25 - loss 2.83946962 - samples/sec: 133.82 - lr: 0.000977 +2021-03-26 08:08:20,712 epoch 76 - iter 12/25 - loss 2.73176597 - samples/sec: 123.82 - lr: 0.000977 +2021-03-26 08:08:21,641 epoch 76 - iter 14/25 - loss 2.76485353 - samples/sec: 138.04 - lr: 0.000977 +2021-03-26 08:08:22,618 epoch 76 - iter 16/25 - loss 2.73733263 - samples/sec: 131.22 - lr: 0.000977 +2021-03-26 08:08:23,532 epoch 76 - iter 18/25 - loss 2.75554327 - samples/sec: 140.35 - lr: 0.000977 +2021-03-26 08:08:24,485 epoch 76 - iter 20/25 - loss 2.71294504 - samples/sec: 134.43 - lr: 0.000977 +2021-03-26 08:08:25,534 epoch 76 - iter 22/25 - loss 2.72319514 - samples/sec: 122.20 - lr: 0.000977 +2021-03-26 08:08:26,518 epoch 76 - iter 24/25 - loss 2.73717165 - samples/sec: 130.24 - lr: 0.000977 +2021-03-26 08:08:26,921 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:08:26,922 EPOCH 76 done: loss 2.7566 - lr 0.0009766 +2021-03-26 08:08:27,667 DEV : loss 6.188060760498047 - score 0.9173 +2021-03-26 08:08:27,689 BAD EPOCHS (no improvement): 2 +2021-03-26 08:08:27,690 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:08:28,633 epoch 77 - iter 2/25 - loss 2.09625757 - samples/sec: 135.96 - lr: 0.000977 +2021-03-26 08:08:29,545 epoch 77 - iter 4/25 - loss 2.58931965 - samples/sec: 140.55 - lr: 0.000977 +2021-03-26 08:08:30,490 epoch 77 - iter 6/25 - loss 2.62068017 - samples/sec: 135.74 - lr: 0.000977 +2021-03-26 08:08:31,398 epoch 77 - iter 8/25 - loss 2.60470769 - samples/sec: 141.00 - lr: 0.000977 +2021-03-26 08:08:32,408 epoch 77 - iter 10/25 - loss 2.61880281 - samples/sec: 126.95 - lr: 0.000977 +2021-03-26 08:08:33,335 epoch 77 - iter 12/25 - loss 2.62026229 - samples/sec: 138.23 - lr: 0.000977 +2021-03-26 08:08:34,261 epoch 77 - iter 14/25 - loss 2.58505779 - samples/sec: 138.47 - lr: 0.000977 +2021-03-26 08:08:35,268 epoch 77 - iter 16/25 - loss 2.56365578 - samples/sec: 127.37 - lr: 0.000977 +2021-03-26 08:08:36,264 epoch 77 - iter 18/25 - loss 2.59711338 - samples/sec: 128.71 - lr: 0.000977 +2021-03-26 08:08:37,287 epoch 77 - iter 20/25 - loss 2.63529403 - samples/sec: 125.35 - lr: 0.000977 +2021-03-26 08:08:38,359 epoch 77 - iter 22/25 - loss 2.62909592 - samples/sec: 119.59 - lr: 0.000977 +2021-03-26 08:08:39,330 epoch 77 - iter 24/25 - loss 2.66551463 - samples/sec: 132.00 - lr: 0.000977 +2021-03-26 08:08:39,717 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:08:39,717 EPOCH 77 done: loss 2.6587 - lr 0.0009766 +2021-03-26 08:08:40,429 DEV : loss 6.187152862548828 - score 0.9173 +2021-03-26 08:08:40,452 BAD EPOCHS (no improvement): 3 +2021-03-26 08:08:40,452 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:08:41,430 epoch 78 - iter 2/25 - loss 3.36305547 - samples/sec: 131.20 - lr: 0.000977 +2021-03-26 08:08:42,365 epoch 78 - iter 4/25 - loss 2.86613524 - samples/sec: 137.06 - lr: 0.000977 +2021-03-26 08:08:43,425 epoch 78 - iter 6/25 - loss 2.85387758 - samples/sec: 121.04 - lr: 0.000977 +2021-03-26 08:08:44,449 epoch 78 - iter 8/25 - loss 2.87975633 - samples/sec: 125.30 - lr: 0.000977 +2021-03-26 08:08:45,368 epoch 78 - iter 10/25 - loss 2.78941748 - samples/sec: 139.38 - lr: 0.000977 +2021-03-26 08:08:46,296 epoch 78 - iter 12/25 - loss 2.83068611 - samples/sec: 138.15 - lr: 0.000977 +2021-03-26 08:08:47,323 epoch 78 - iter 14/25 - loss 2.80439939 - samples/sec: 124.87 - lr: 0.000977 +2021-03-26 08:08:48,368 epoch 78 - iter 16/25 - loss 2.82471086 - samples/sec: 122.62 - lr: 0.000977 +2021-03-26 08:08:49,325 epoch 78 - iter 18/25 - loss 2.83826003 - samples/sec: 134.05 - lr: 0.000977 +2021-03-26 08:08:50,269 epoch 78 - iter 20/25 - loss 2.84359343 - samples/sec: 135.87 - lr: 0.000977 +2021-03-26 08:08:51,337 epoch 78 - iter 22/25 - loss 2.87817531 - samples/sec: 120.06 - lr: 0.000977 +2021-03-26 08:08:52,373 epoch 78 - iter 24/25 - loss 2.82889307 - samples/sec: 123.81 - lr: 0.000977 +2021-03-26 08:08:52,754 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:08:52,755 EPOCH 78 done: loss 2.8359 - lr 0.0009766 +2021-03-26 08:08:53,457 DEV : loss 6.186553001403809 - score 0.9173 +2021-03-26 08:08:53,481 BAD EPOCHS (no improvement): 4 +2021-03-26 08:08:53,481 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:08:54,362 epoch 79 - iter 2/25 - loss 2.95909679 - samples/sec: 145.63 - lr: 0.000488 +2021-03-26 08:08:55,289 epoch 79 - iter 4/25 - loss 2.81746089 - samples/sec: 138.31 - lr: 0.000488 +2021-03-26 08:08:56,320 epoch 79 - iter 6/25 - loss 2.89012504 - samples/sec: 124.34 - lr: 0.000488 +2021-03-26 08:08:57,322 epoch 79 - iter 8/25 - loss 2.91810098 - samples/sec: 127.94 - lr: 0.000488 +2021-03-26 08:08:58,204 epoch 79 - iter 10/25 - loss 2.91408601 - samples/sec: 145.42 - lr: 0.000488 +2021-03-26 08:08:59,138 epoch 79 - iter 12/25 - loss 2.82381584 - samples/sec: 137.23 - lr: 0.000488 +2021-03-26 08:09:00,065 epoch 79 - iter 14/25 - loss 2.79630446 - samples/sec: 138.33 - lr: 0.000488 +2021-03-26 08:09:01,051 epoch 79 - iter 16/25 - loss 2.77844489 - samples/sec: 129.98 - lr: 0.000488 +2021-03-26 08:09:02,047 epoch 79 - iter 18/25 - loss 2.81718817 - samples/sec: 128.72 - lr: 0.000488 +2021-03-26 08:09:03,127 epoch 79 - iter 20/25 - loss 2.85397427 - samples/sec: 118.83 - lr: 0.000488 +2021-03-26 08:09:04,159 epoch 79 - iter 22/25 - loss 2.82800953 - samples/sec: 124.13 - lr: 0.000488 +2021-03-26 08:09:05,137 epoch 79 - iter 24/25 - loss 2.81754422 - samples/sec: 131.10 - lr: 0.000488 +2021-03-26 08:09:05,617 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:09:05,618 EPOCH 79 done: loss 2.8396 - lr 0.0004883 +2021-03-26 08:09:06,327 DEV : loss 6.186826229095459 - score 0.9173 +2021-03-26 08:09:06,350 BAD EPOCHS (no improvement): 1 +2021-03-26 08:09:06,351 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:09:07,377 epoch 80 - iter 2/25 - loss 2.64280188 - samples/sec: 125.05 - lr: 0.000488 +2021-03-26 08:09:08,318 epoch 80 - iter 4/25 - loss 2.53505731 - samples/sec: 136.12 - lr: 0.000488 +2021-03-26 08:09:09,308 epoch 80 - iter 6/25 - loss 2.48866999 - samples/sec: 129.48 - lr: 0.000488 +2021-03-26 08:09:10,281 epoch 80 - iter 8/25 - loss 2.47880760 - samples/sec: 131.83 - lr: 0.000488 +2021-03-26 08:09:11,304 epoch 80 - iter 10/25 - loss 2.54190221 - samples/sec: 125.28 - lr: 0.000488 +2021-03-26 08:09:12,611 epoch 80 - iter 12/25 - loss 2.59663198 - samples/sec: 97.98 - lr: 0.000488 +2021-03-26 08:09:13,537 epoch 80 - iter 14/25 - loss 2.53316179 - samples/sec: 138.56 - lr: 0.000488 +2021-03-26 08:09:14,619 epoch 80 - iter 16/25 - loss 2.58981419 - samples/sec: 118.51 - lr: 0.000488 +2021-03-26 08:09:15,665 epoch 80 - iter 18/25 - loss 2.62501939 - samples/sec: 122.58 - lr: 0.000488 +2021-03-26 08:09:16,645 epoch 80 - iter 20/25 - loss 2.58662164 - samples/sec: 130.87 - lr: 0.000488 +2021-03-26 08:09:17,617 epoch 80 - iter 22/25 - loss 2.58739001 - samples/sec: 131.94 - lr: 0.000488 +2021-03-26 08:09:18,684 epoch 80 - iter 24/25 - loss 2.58220779 - samples/sec: 120.15 - lr: 0.000488 +2021-03-26 08:09:19,086 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:09:19,087 EPOCH 80 done: loss 2.6174 - lr 0.0004883 +2021-03-26 08:09:19,790 DEV : loss 6.187230110168457 - score 0.9173 +2021-03-26 08:09:19,812 BAD EPOCHS (no improvement): 2 +2021-03-26 08:09:19,813 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:09:20,786 epoch 81 - iter 2/25 - loss 2.75670075 - samples/sec: 131.78 - lr: 0.000488 +2021-03-26 08:09:21,781 epoch 81 - iter 4/25 - loss 3.00184679 - samples/sec: 128.75 - lr: 0.000488 +2021-03-26 08:09:22,720 epoch 81 - iter 6/25 - loss 2.95591950 - samples/sec: 136.62 - lr: 0.000488 +2021-03-26 08:09:23,666 epoch 81 - iter 8/25 - loss 2.84314916 - samples/sec: 135.57 - lr: 0.000488 +2021-03-26 08:09:24,686 epoch 81 - iter 10/25 - loss 2.74002171 - samples/sec: 125.75 - lr: 0.000488 +2021-03-26 08:09:25,656 epoch 81 - iter 12/25 - loss 2.79702922 - samples/sec: 132.05 - lr: 0.000488 +2021-03-26 08:09:26,761 epoch 81 - iter 14/25 - loss 2.86702430 - samples/sec: 115.98 - lr: 0.000488 +2021-03-26 08:09:27,712 epoch 81 - iter 16/25 - loss 2.88608465 - samples/sec: 134.76 - lr: 0.000488 +2021-03-26 08:09:28,614 epoch 81 - iter 18/25 - loss 2.89608358 - samples/sec: 142.11 - lr: 0.000488 +2021-03-26 08:09:29,537 epoch 81 - iter 20/25 - loss 2.86903533 - samples/sec: 138.97 - lr: 0.000488 +2021-03-26 08:09:30,571 epoch 81 - iter 22/25 - loss 2.82412986 - samples/sec: 123.98 - lr: 0.000488 +2021-03-26 08:09:31,647 epoch 81 - iter 24/25 - loss 2.81362030 - samples/sec: 119.07 - lr: 0.000488 +2021-03-26 08:09:32,038 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:09:32,039 EPOCH 81 done: loss 2.8294 - lr 0.0004883 +2021-03-26 08:09:32,758 DEV : loss 6.185996055603027 - score 0.9173 +2021-03-26 08:09:32,782 BAD EPOCHS (no improvement): 3 +2021-03-26 08:09:32,783 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:09:33,724 epoch 82 - iter 2/25 - loss 2.84042430 - samples/sec: 136.20 - lr: 0.000488 +2021-03-26 08:09:34,657 epoch 82 - iter 4/25 - loss 2.65296417 - samples/sec: 137.43 - lr: 0.000488 +2021-03-26 08:09:35,609 epoch 82 - iter 6/25 - loss 2.70227369 - samples/sec: 134.63 - lr: 0.000488 +2021-03-26 08:09:36,600 epoch 82 - iter 8/25 - loss 2.78364819 - samples/sec: 129.28 - lr: 0.000488 +2021-03-26 08:09:37,611 epoch 82 - iter 10/25 - loss 2.66723704 - samples/sec: 126.85 - lr: 0.000488 +2021-03-26 08:09:38,583 epoch 82 - iter 12/25 - loss 2.66724110 - samples/sec: 131.91 - lr: 0.000488 +2021-03-26 08:09:39,528 epoch 82 - iter 14/25 - loss 2.66870518 - samples/sec: 135.71 - lr: 0.000488 +2021-03-26 08:09:40,516 epoch 82 - iter 16/25 - loss 2.75237661 - samples/sec: 129.75 - lr: 0.000488 +2021-03-26 08:09:41,504 epoch 82 - iter 18/25 - loss 2.71161309 - samples/sec: 129.70 - lr: 0.000488 +2021-03-26 08:09:42,489 epoch 82 - iter 20/25 - loss 2.73067624 - samples/sec: 130.12 - lr: 0.000488 +2021-03-26 08:09:43,507 epoch 82 - iter 22/25 - loss 2.74962510 - samples/sec: 125.94 - lr: 0.000488 +2021-03-26 08:09:44,513 epoch 82 - iter 24/25 - loss 2.71701292 - samples/sec: 127.32 - lr: 0.000488 +2021-03-26 08:09:44,967 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:09:44,968 EPOCH 82 done: loss 2.7060 - lr 0.0004883 +2021-03-26 08:09:45,706 DEV : loss 6.185783386230469 - score 0.9173 +2021-03-26 08:09:45,729 BAD EPOCHS (no improvement): 4 +2021-03-26 08:09:45,730 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:09:46,716 epoch 83 - iter 2/25 - loss 2.85962808 - samples/sec: 130.18 - lr: 0.000244 +2021-03-26 08:09:47,740 epoch 83 - iter 4/25 - loss 2.75480992 - samples/sec: 125.14 - lr: 0.000244 +2021-03-26 08:09:48,780 epoch 83 - iter 6/25 - loss 2.68297470 - samples/sec: 123.23 - lr: 0.000244 +2021-03-26 08:09:49,741 epoch 83 - iter 8/25 - loss 2.79082182 - samples/sec: 133.45 - lr: 0.000244 +2021-03-26 08:09:50,686 epoch 83 - iter 10/25 - loss 2.74289029 - samples/sec: 135.70 - lr: 0.000244 +2021-03-26 08:09:51,599 epoch 83 - iter 12/25 - loss 2.74736812 - samples/sec: 140.49 - lr: 0.000244 +2021-03-26 08:09:52,638 epoch 83 - iter 14/25 - loss 2.69184843 - samples/sec: 123.28 - lr: 0.000244 +2021-03-26 08:09:53,648 epoch 83 - iter 16/25 - loss 2.75959721 - samples/sec: 126.95 - lr: 0.000244 +2021-03-26 08:09:54,608 epoch 83 - iter 18/25 - loss 2.71115977 - samples/sec: 133.53 - lr: 0.000244 +2021-03-26 08:09:55,878 epoch 83 - iter 20/25 - loss 2.74285476 - samples/sec: 100.90 - lr: 0.000244 +2021-03-26 08:09:56,819 epoch 83 - iter 22/25 - loss 2.72497958 - samples/sec: 136.33 - lr: 0.000244 +2021-03-26 08:09:57,716 epoch 83 - iter 24/25 - loss 2.67853123 - samples/sec: 142.90 - lr: 0.000244 +2021-03-26 08:09:58,148 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:09:58,150 EPOCH 83 done: loss 2.6657 - lr 0.0002441 +2021-03-26 08:09:58,863 DEV : loss 6.185376167297363 - score 0.9173 +2021-03-26 08:09:58,884 BAD EPOCHS (no improvement): 1 +2021-03-26 08:09:58,885 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:09:59,810 epoch 84 - iter 2/25 - loss 2.58333921 - samples/sec: 138.61 - lr: 0.000244 +2021-03-26 08:10:00,749 epoch 84 - iter 4/25 - loss 2.76761711 - samples/sec: 136.47 - lr: 0.000244 +2021-03-26 08:10:01,781 epoch 84 - iter 6/25 - loss 2.71056978 - samples/sec: 124.26 - lr: 0.000244 +2021-03-26 08:10:02,830 epoch 84 - iter 8/25 - loss 2.72177193 - samples/sec: 122.24 - lr: 0.000244 +2021-03-26 08:10:03,801 epoch 84 - iter 10/25 - loss 2.75179241 - samples/sec: 132.03 - lr: 0.000244 +2021-03-26 08:10:04,863 epoch 84 - iter 12/25 - loss 2.76484060 - samples/sec: 120.77 - lr: 0.000244 +2021-03-26 08:10:05,896 epoch 84 - iter 14/25 - loss 2.70445476 - samples/sec: 124.09 - lr: 0.000244 +2021-03-26 08:10:07,880 epoch 84 - iter 16/25 - loss 2.75915471 - samples/sec: 64.55 - lr: 0.000244 +2021-03-26 08:10:08,782 epoch 84 - iter 18/25 - loss 2.67348589 - samples/sec: 142.25 - lr: 0.000244 +2021-03-26 08:10:09,901 epoch 84 - iter 20/25 - loss 2.65599443 - samples/sec: 114.49 - lr: 0.000244 +2021-03-26 08:10:10,908 epoch 84 - iter 22/25 - loss 2.68863658 - samples/sec: 127.38 - lr: 0.000244 +2021-03-26 08:10:11,912 epoch 84 - iter 24/25 - loss 2.72180267 - samples/sec: 127.64 - lr: 0.000244 +2021-03-26 08:10:12,348 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:10:12,348 EPOCH 84 done: loss 2.7303 - lr 0.0002441 +2021-03-26 08:10:13,110 DEV : loss 6.185603618621826 - score 0.9173 +2021-03-26 08:10:13,142 BAD EPOCHS (no improvement): 2 +2021-03-26 08:10:13,143 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:10:14,276 epoch 85 - iter 2/25 - loss 2.66001129 - samples/sec: 113.17 - lr: 0.000244 +2021-03-26 08:10:15,390 epoch 85 - iter 4/25 - loss 2.54920405 - samples/sec: 115.13 - lr: 0.000244 +2021-03-26 08:10:16,350 epoch 85 - iter 6/25 - loss 2.38967019 - samples/sec: 133.48 - lr: 0.000244 +2021-03-26 08:10:17,285 epoch 85 - iter 8/25 - loss 2.52254377 - samples/sec: 137.09 - lr: 0.000244 +2021-03-26 08:10:18,137 epoch 85 - iter 10/25 - loss 2.48643364 - samples/sec: 150.34 - lr: 0.000244 +2021-03-26 08:10:19,155 epoch 85 - iter 12/25 - loss 2.50120088 - samples/sec: 125.91 - lr: 0.000244 +2021-03-26 08:10:20,085 epoch 85 - iter 14/25 - loss 2.58942009 - samples/sec: 138.04 - lr: 0.000244 +2021-03-26 08:10:21,044 epoch 85 - iter 16/25 - loss 2.71086637 - samples/sec: 133.54 - lr: 0.000244 +2021-03-26 08:10:22,044 epoch 85 - iter 18/25 - loss 2.73113488 - samples/sec: 128.27 - lr: 0.000244 +2021-03-26 08:10:23,144 epoch 85 - iter 20/25 - loss 2.73964195 - samples/sec: 116.51 - lr: 0.000244 +2021-03-26 08:10:24,110 epoch 85 - iter 22/25 - loss 2.72373083 - samples/sec: 132.59 - lr: 0.000244 +2021-03-26 08:10:25,135 epoch 85 - iter 24/25 - loss 2.71947557 - samples/sec: 125.07 - lr: 0.000244 +2021-03-26 08:10:25,558 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:10:25,559 EPOCH 85 done: loss 2.6878 - lr 0.0002441 +2021-03-26 08:10:26,268 DEV : loss 6.186032772064209 - score 0.9173 +2021-03-26 08:10:26,287 BAD EPOCHS (no improvement): 3 +2021-03-26 08:10:26,288 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:10:27,381 epoch 86 - iter 2/25 - loss 2.83442032 - samples/sec: 117.18 - lr: 0.000244 +2021-03-26 08:10:28,427 epoch 86 - iter 4/25 - loss 3.07467341 - samples/sec: 122.58 - lr: 0.000244 +2021-03-26 08:10:29,412 epoch 86 - iter 6/25 - loss 3.05214135 - samples/sec: 130.16 - lr: 0.000244 +2021-03-26 08:10:30,469 epoch 86 - iter 8/25 - loss 2.90568414 - samples/sec: 121.27 - lr: 0.000244 +2021-03-26 08:10:31,431 epoch 86 - iter 10/25 - loss 2.91739962 - samples/sec: 133.22 - lr: 0.000244 +2021-03-26 08:10:32,462 epoch 86 - iter 12/25 - loss 2.81895500 - samples/sec: 124.39 - lr: 0.000244 +2021-03-26 08:10:33,395 epoch 86 - iter 14/25 - loss 2.82476473 - samples/sec: 137.33 - lr: 0.000244 +2021-03-26 08:10:34,277 epoch 86 - iter 16/25 - loss 2.78650875 - samples/sec: 145.50 - lr: 0.000244 +2021-03-26 08:10:35,278 epoch 86 - iter 18/25 - loss 2.71929197 - samples/sec: 128.04 - lr: 0.000244 +2021-03-26 08:10:36,220 epoch 86 - iter 20/25 - loss 2.72559129 - samples/sec: 136.11 - lr: 0.000244 +2021-03-26 08:10:37,262 epoch 86 - iter 22/25 - loss 2.68120156 - samples/sec: 123.07 - lr: 0.000244 +2021-03-26 08:10:38,257 epoch 86 - iter 24/25 - loss 2.71939340 - samples/sec: 128.85 - lr: 0.000244 +2021-03-26 08:10:38,695 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:10:38,696 EPOCH 86 done: loss 2.7590 - lr 0.0002441 +2021-03-26 08:10:39,418 DEV : loss 6.185995101928711 - score 0.9173 +2021-03-26 08:10:39,441 BAD EPOCHS (no improvement): 4 +2021-03-26 08:10:39,441 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:10:40,535 epoch 87 - iter 2/25 - loss 3.12590420 - samples/sec: 117.25 - lr: 0.000122 +2021-03-26 08:10:41,574 epoch 87 - iter 4/25 - loss 3.28620195 - samples/sec: 123.43 - lr: 0.000122 +2021-03-26 08:10:42,655 epoch 87 - iter 6/25 - loss 3.30520022 - samples/sec: 118.61 - lr: 0.000122 +2021-03-26 08:10:43,690 epoch 87 - iter 8/25 - loss 3.32119605 - samples/sec: 123.86 - lr: 0.000122 +2021-03-26 08:10:44,668 epoch 87 - iter 10/25 - loss 3.14544759 - samples/sec: 131.02 - lr: 0.000122 +2021-03-26 08:10:45,613 epoch 87 - iter 12/25 - loss 3.15682425 - samples/sec: 135.74 - lr: 0.000122 +2021-03-26 08:10:46,555 epoch 87 - iter 14/25 - loss 3.06249428 - samples/sec: 136.06 - lr: 0.000122 +2021-03-26 08:10:47,534 epoch 87 - iter 16/25 - loss 3.03412677 - samples/sec: 130.96 - lr: 0.000122 +2021-03-26 08:10:48,552 epoch 87 - iter 18/25 - loss 2.96904975 - samples/sec: 125.98 - lr: 0.000122 +2021-03-26 08:10:49,470 epoch 87 - iter 20/25 - loss 2.94991877 - samples/sec: 139.62 - lr: 0.000122 +2021-03-26 08:10:50,427 epoch 87 - iter 22/25 - loss 2.88184339 - samples/sec: 134.01 - lr: 0.000122 +2021-03-26 08:10:51,340 epoch 87 - iter 24/25 - loss 2.85626723 - samples/sec: 140.36 - lr: 0.000122 +2021-03-26 08:10:51,784 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:10:51,784 EPOCH 87 done: loss 2.8113 - lr 0.0001221 +2021-03-26 08:10:52,491 DEV : loss 6.186027526855469 - score 0.9173 +2021-03-26 08:10:52,513 BAD EPOCHS (no improvement): 1 +2021-03-26 08:10:52,514 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:10:53,566 epoch 88 - iter 2/25 - loss 2.68268955 - samples/sec: 121.82 - lr: 0.000122 +2021-03-26 08:10:54,524 epoch 88 - iter 4/25 - loss 2.62284821 - samples/sec: 133.81 - lr: 0.000122 +2021-03-26 08:10:55,522 epoch 88 - iter 6/25 - loss 2.53554523 - samples/sec: 128.54 - lr: 0.000122 +2021-03-26 08:10:56,537 epoch 88 - iter 8/25 - loss 2.52243441 - samples/sec: 126.30 - lr: 0.000122 +2021-03-26 08:10:57,455 epoch 88 - iter 10/25 - loss 2.47092564 - samples/sec: 139.73 - lr: 0.000122 +2021-03-26 08:10:58,376 epoch 88 - iter 12/25 - loss 2.52259127 - samples/sec: 139.19 - lr: 0.000122 +2021-03-26 08:10:59,424 epoch 88 - iter 14/25 - loss 2.52517985 - samples/sec: 122.39 - lr: 0.000122 +2021-03-26 08:11:00,399 epoch 88 - iter 16/25 - loss 2.61265714 - samples/sec: 131.46 - lr: 0.000122 +2021-03-26 08:11:01,390 epoch 88 - iter 18/25 - loss 2.59087091 - samples/sec: 129.34 - lr: 0.000122 +2021-03-26 08:11:02,398 epoch 88 - iter 20/25 - loss 2.61029449 - samples/sec: 127.06 - lr: 0.000122 +2021-03-26 08:11:03,410 epoch 88 - iter 22/25 - loss 2.63463608 - samples/sec: 126.76 - lr: 0.000122 +2021-03-26 08:11:04,419 epoch 88 - iter 24/25 - loss 2.66323133 - samples/sec: 127.02 - lr: 0.000122 +2021-03-26 08:11:04,836 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:11:04,837 EPOCH 88 done: loss 2.6686 - lr 0.0001221 +2021-03-26 08:11:05,616 DEV : loss 6.185815811157227 - score 0.9173 +2021-03-26 08:11:05,639 BAD EPOCHS (no improvement): 2 +2021-03-26 08:11:05,640 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:11:06,667 epoch 89 - iter 2/25 - loss 2.97890735 - samples/sec: 124.79 - lr: 0.000122 +2021-03-26 08:11:07,650 epoch 89 - iter 4/25 - loss 2.86776453 - samples/sec: 130.51 - lr: 0.000122 +2021-03-26 08:11:08,646 epoch 89 - iter 6/25 - loss 2.75692447 - samples/sec: 128.62 - lr: 0.000122 +2021-03-26 08:11:09,654 epoch 89 - iter 8/25 - loss 2.83591112 - samples/sec: 127.24 - lr: 0.000122 +2021-03-26 08:11:10,678 epoch 89 - iter 10/25 - loss 2.87064221 - samples/sec: 125.15 - lr: 0.000122 +2021-03-26 08:11:11,616 epoch 89 - iter 12/25 - loss 2.92312527 - samples/sec: 136.58 - lr: 0.000122 +2021-03-26 08:11:12,512 epoch 89 - iter 14/25 - loss 2.88578798 - samples/sec: 143.21 - lr: 0.000122 +2021-03-26 08:11:13,489 epoch 89 - iter 16/25 - loss 2.84937823 - samples/sec: 131.21 - lr: 0.000122 +2021-03-26 08:11:14,552 epoch 89 - iter 18/25 - loss 2.79902322 - samples/sec: 120.52 - lr: 0.000122 +2021-03-26 08:11:15,521 epoch 89 - iter 20/25 - loss 2.76434165 - samples/sec: 132.20 - lr: 0.000122 +2021-03-26 08:11:16,547 epoch 89 - iter 22/25 - loss 2.79289429 - samples/sec: 125.08 - lr: 0.000122 +2021-03-26 08:11:17,647 epoch 89 - iter 24/25 - loss 2.79572803 - samples/sec: 117.01 - lr: 0.000122 +2021-03-26 08:11:18,005 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:11:18,006 EPOCH 89 done: loss 2.7775 - lr 0.0001221 +2021-03-26 08:11:18,728 DEV : loss 6.185563564300537 - score 0.9173 +2021-03-26 08:11:18,750 BAD EPOCHS (no improvement): 3 +2021-03-26 08:11:18,751 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:11:19,745 epoch 90 - iter 2/25 - loss 2.81466031 - samples/sec: 129.08 - lr: 0.000122 +2021-03-26 08:11:20,923 epoch 90 - iter 4/25 - loss 2.73348701 - samples/sec: 108.76 - lr: 0.000122 +2021-03-26 08:11:21,877 epoch 90 - iter 6/25 - loss 2.85376092 - samples/sec: 134.40 - lr: 0.000122 +2021-03-26 08:11:22,931 epoch 90 - iter 8/25 - loss 2.80158153 - samples/sec: 121.54 - lr: 0.000122 +2021-03-26 08:11:23,805 epoch 90 - iter 10/25 - loss 2.70515602 - samples/sec: 146.80 - lr: 0.000122 +2021-03-26 08:11:24,826 epoch 90 - iter 12/25 - loss 2.72229065 - samples/sec: 125.47 - lr: 0.000122 +2021-03-26 08:11:25,868 epoch 90 - iter 14/25 - loss 2.70650598 - samples/sec: 123.07 - lr: 0.000122 +2021-03-26 08:11:26,812 epoch 90 - iter 16/25 - loss 2.66336836 - samples/sec: 135.89 - lr: 0.000122 +2021-03-26 08:11:27,916 epoch 90 - iter 18/25 - loss 2.60368521 - samples/sec: 115.98 - lr: 0.000122 +2021-03-26 08:11:28,785 epoch 90 - iter 20/25 - loss 2.59294172 - samples/sec: 147.67 - lr: 0.000122 +2021-03-26 08:11:29,915 epoch 90 - iter 22/25 - loss 2.57639271 - samples/sec: 113.35 - lr: 0.000122 +2021-03-26 08:11:30,898 epoch 90 - iter 24/25 - loss 2.62383321 - samples/sec: 130.41 - lr: 0.000122 +2021-03-26 08:11:31,369 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:11:31,369 EPOCH 90 done: loss 2.6541 - lr 0.0001221 +2021-03-26 08:11:32,111 DEV : loss 6.185521602630615 - score 0.9173 +2021-03-26 08:11:32,134 BAD EPOCHS (no improvement): 4 +2021-03-26 08:11:32,135 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:11:32,135 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:11:32,135 learning rate too small - quitting training! +2021-03-26 08:11:32,136 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:11:41,404 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:11:41,405 Testing using best model ... +2021-03-26 08:11:41,406 loading file /home/tmp/megahedm/models/multipos/multipos_UDMADAR_4Diale-LEV_EGY_GLF_MGR__fasttext_flairbwfw__64__0.5_202103260747/best-model.pt +2021-03-26 08:11:48,677 0.911 +2021-03-26 08:11:48,678 +Results: +- F-score (micro): 0.9077 +- F-score (macro): 0.6024 +- Accuracy (incl. no class): 0.911 + +By class: + precision recall f1-score support + + NOUN 0.9227 0.9363 0.9294 408 + NUM 0.9333 0.7778 0.8485 18 + ADJ 0.8913 0.8367 0.8632 98 + CCONJ 1.0000 0.9342 0.9660 76 + PRON 0.9609 0.9773 0.9690 176 + ADV 0.8776 0.8350 0.8557 103 + PART 0.9554 0.9772 0.9661 219 + VERB 0.9375 0.9507 0.9441 142 + ADP 0.9811 0.9541 0.9674 109 + DET 0.9206 0.9667 0.9431 60 + PUNCT 1.0000 1.0000 1.0000 30 + SCONJ 0.8788 0.9355 0.9062 31 + PROPN 0.8077 0.8400 0.8235 25 + AUX 0.9250 0.9487 0.9367 39 + INTJ 0.9412 0.9412 0.9412 17 + DET+NOUN 0.9167 0.9429 0.9296 70 + CONJ+DET+NOUN 1.0000 0.8333 0.9091 6 + CONJ+DET+ADJ 0.0000 1.0000 0.0000 0 + PREP+DET+NOUN+NSUFF 0.3333 0.3333 0.3333 3 + PREP+NOUN 0.7500 0.8333 0.7895 18 + PREP+DET+NOUN 0.7778 1.0000 0.8750 7 + PUNC 1.0000 1.0000 1.0000 148 + DET+NOUN+NSUFF 0.8846 0.8846 0.8846 26 + PROG_PART+V 0.8140 0.9459 0.8750 37 + PART+PRON 1.0000 0.8824 0.9375 17 + V+PRON 0.8462 0.7586 0.8000 58 + PREP+PART+PRON 1.0000 1.0000 1.0000 4 + URL 1.0000 1.0000 1.0000 3 + EOS 1.0000 1.0000 1.0000 70 + PRON+DET+NOUN+NSUFF 1.0000 0.0000 0.0000 4 + PRON+DET+NOUN 0.0000 1.0000 0.0000 0 + PROG_PART+V+PREP 1.0000 0.0000 0.0000 1 + PREP 0.9545 0.9844 0.9692 64 + CONJ+PART 1.0000 0.9333 0.9655 15 + CONJ+PROG_PART+V 0.6667 0.6667 0.6667 3 + CONJ+NOUN 0.7273 0.8000 0.7619 10 + PROG_PART+V+PRON 0.8421 0.8889 0.8649 18 + PREP+PRON 0.7931 0.9583 0.8679 24 + ADJ+NSUFF 0.8571 0.7317 0.7895 41 + NOUN+NSUFF 0.7353 0.8772 0.8000 57 + V 0.8532 0.8942 0.8732 104 + PART+V+NEG_PART 0.5000 0.6667 0.5714 3 + NOUN+PRON 0.8841 0.8243 0.8531 74 + PREP+NOUN+NSUFF+PRON 1.0000 0.2500 0.4000 4 + PREP+NOUN+PRON 0.0000 0.0000 0.0000 2 + FUT_PART+V 1.0000 0.6667 0.8000 12 + FOREIGN 1.0000 0.6667 0.8000 3 + MENTION 0.9565 1.0000 0.9778 22 + CONJ+ADJ 0.0000 0.0000 0.0000 2 + CONJ+PRON 1.0000 1.0000 1.0000 7 + ADJ+PRON 0.5714 0.8000 0.6667 5 + HASH 1.0000 0.9231 0.9600 13 + CONJ 0.9722 0.9722 0.9722 36 + DET+ADJ 1.0000 0.6250 0.7692 8 + DET+NUM 0.0000 1.0000 0.0000 0 + ADJ+PREP+PRON 0.0000 0.0000 0.0000 1 + PROG_PART+V+PREP+PRON 0.0000 1.0000 0.0000 0 + NOUN+NSUFF+PRON 0.5238 0.9167 0.6667 12 + PREP+NOUN+NSUFF 0.0000 0.0000 0.0000 4 + PART+V 1.0000 0.0000 0.0000 2 + EMOT 0.9500 1.0000 0.9744 19 + NOUN+NSUFF+NSUFF 1.0000 0.0000 0.0000 1 + V+PRON+PRON 1.0000 0.4000 0.5714 10 + FUT_PART 1.0000 1.0000 1.0000 11 + DET+ADJ+NSUFF 0.5000 0.5000 0.5000 2 + PREP+DET+NUM+NSUFF 1.0000 0.0000 0.0000 1 + NOUN+CASE 0.8000 0.8000 0.8000 5 + PREP+PART 1.0000 1.0000 1.0000 1 + PART+NOUN 0.5000 0.5000 0.5000 2 + NOUN+PRON+PRON 1.0000 0.0000 0.0000 2 + PART+NOUN+PRON 1.0000 0.0000 0.0000 2 + CONJ+FUT_PART+V 1.0000 0.0000 0.0000 2 + FUT_PART+V+PRON 0.4000 1.0000 0.5714 2 + CONJ+V 0.7778 1.0000 0.8750 7 + CONJ+V+PREP+PRON 0.5000 1.0000 0.6667 1 + PROG_PART+V+NEG_PART 1.0000 1.0000 1.0000 1 + CONJ+PREP+PRON 1.0000 1.0000 1.0000 1 + ADJ+NSUFF+PRON 1.0000 0.0000 0.0000 2 + PREP+DET 1.0000 0.0000 0.0000 1 + CONJ+NOUN+PRON 0.5000 1.0000 0.6667 2 + CONJ+PART+V+PRON 1.0000 0.0000 0.0000 1 + V+PREP+PRON 0.3333 0.3333 0.3333 3 + PART+V+PRON+NEG_PART 0.3333 1.0000 0.5000 2 + V+NOUN 1.0000 0.0000 0.0000 1 + CONJ+NOUN+NSUFF 1.0000 1.0000 1.0000 2 + ADV+NSUFF 1.0000 1.0000 1.0000 2 +PART+V+PRON+PRON+NEG_PART 1.0000 1.0000 1.0000 1 + PART+PREP+PRON+NEG_PART 1.0000 0.3333 0.5000 3 + CONJ+ADV 1.0000 1.0000 1.0000 2 + NOUN+CASE+PRON 1.0000 0.0000 0.0000 1 + PART+PREP+NEG_PART 1.0000 1.0000 1.0000 3 + PART+NOUN+NEG_PART 1.0000 1.0000 1.0000 1 + CONJ+PROG_PART+V+PRON 1.0000 1.0000 1.0000 1 + CONJ+V+PRON 1.0000 1.0000 1.0000 1 + PART+PART 1.0000 0.0000 0.0000 1 + CONJ+ADJ+NSUFF 1.0000 0.0000 0.0000 1 + CONJ+PREP 1.0000 1.0000 1.0000 1 + NUM+NSUFF 1.0000 0.0000 0.0000 1 + FUT_PART+V+PRON+PRON 1.0000 0.0000 0.0000 2 + FUT_PART+V+PREP+PRON 0.0000 1.0000 0.0000 0 + ADJ+CASE+PREP 1.0000 0.0000 0.0000 1 + PREP+ADV 1.0000 0.0000 0.0000 1 + PREP+DET+ADJ 0.0000 1.0000 0.0000 0 + NOUN+PRON+NEG_PART 1.0000 0.0000 0.0000 1 + + micro avg 0.9080 0.9074 0.9077 2677 + macro avg 0.8047 0.6744 0.6024 2677 + weighted avg 0.9161 0.9074 0.9036 2677 + +2021-03-26 08:11:48,678 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:11:48,678 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:11:52,326 Reading data from ../../Datasets_adhoc/CSCS_corpus-GUC +2021-03-26 08:11:52,327 Train: ../../Datasets_adhoc/CSCS_corpus-GUC/all_participants.conllu +2021-03-26 08:11:52,327 Dev: None +2021-03-26 08:11:52,327 Test: None +2021-03-26 08:11:54,235 Reading data from ../../Datasets_adhoc/UD_MADAR +2021-03-26 08:11:54,235 Train: ../../Datasets_adhoc/UD_MADAR/ajp_madar-ud-test-edit.conllu +2021-03-26 08:11:54,236 Dev: None +2021-03-26 08:11:54,236 Test: None +2021-03-26 08:11:54,280 Reading data from ../../Datasets_adhoc/dialectal_arabic_resources-master +2021-03-26 08:11:54,281 Train: ../../Datasets_adhoc/dialectal_arabic_resources-master/seg_plus_pos_lev.txt +2021-03-26 08:11:54,281 Dev: None +2021-03-26 08:11:54,282 Test: None +2021-03-26 08:11:54,437 Reading data from ../../Datasets_adhoc/dialectal_arabic_resources-master +2021-03-26 08:11:54,437 Train: ../../Datasets_adhoc/dialectal_arabic_resources-master/seg_plus_pos_egy.txt +2021-03-26 08:11:54,438 Dev: None +2021-03-26 08:11:54,438 Test: None +2021-03-26 08:11:54,605 Reading data from ../../Datasets_adhoc/dialectal_arabic_resources-master +2021-03-26 08:11:54,605 Train: ../../Datasets_adhoc/dialectal_arabic_resources-master/seg_plus_pos_glf.txt +2021-03-26 08:11:54,606 Dev: None +2021-03-26 08:11:54,606 Test: None +2021-03-26 08:11:54,757 Reading data from ../../Datasets_adhoc/dialectal_arabic_resources-master +2021-03-26 08:11:54,757 Train: ../../Datasets_adhoc/dialectal_arabic_resources-master/seg_plus_pos_mgr.txt +2021-03-26 08:11:54,758 Dev: None +2021-03-26 08:11:54,758 Test: None +2021-03-26 08:11:54,897 Filtering long sentences +2021-03-26 08:11:54,935 MultiCorpus: 1574 train + 176 dev + 194 test sentences + - ColumnCorpus Corpus: 934 train + 104 dev + 115 test sentences + - ColumnCorpus Corpus: 81 train + 9 dev + 10 test sentences + - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences + - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences +2021-03-26 08:11:55,329 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:11:55,330 Model: "SequenceTagger( + (embeddings): StackedEmbeddings( + (list_embedding_0): WordEmbeddings('ar') + (list_embedding_1): FlairEmbeddings( + (lm): LanguageModel( + (drop): Dropout(p=0.1, inplace=False) + (encoder): Embedding(7125, 100) + (rnn): LSTM(100, 2048) + (decoder): Linear(in_features=2048, out_features=7125, bias=True) + ) + ) + (list_embedding_2): FlairEmbeddings( + (lm): LanguageModel( + (drop): Dropout(p=0.1, inplace=False) + (encoder): Embedding(7125, 100) + (rnn): LSTM(100, 2048) + (decoder): Linear(in_features=2048, out_features=7125, bias=True) + ) + ) + ) + (word_dropout): WordDropout(p=0.05) + (locked_dropout): LockedDropout(p=0.5) + (embedding2nn): Linear(in_features=4396, out_features=4396, bias=True) + (rnn): LSTM(4396, 256, batch_first=True, bidirectional=True) + (linear): Linear(in_features=512, out_features=206, bias=True) + (beta): 1.0 + (weights): None + (weight_tensor) None +)" +2021-03-26 08:11:55,330 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:11:55,330 Corpus: "MultiCorpus: 1574 train + 176 dev + 194 test sentences + - ColumnCorpus Corpus: 934 train + 104 dev + 115 test sentences + - ColumnCorpus Corpus: 81 train + 9 dev + 10 test sentences + - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences + - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences" +2021-03-26 08:11:55,331 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:11:55,331 Parameters: +2021-03-26 08:11:55,331 - learning_rate: "0.5" +2021-03-26 08:11:55,332 - mini_batch_size: "64" +2021-03-26 08:11:55,332 - patience: "3" +2021-03-26 08:11:55,332 - anneal_factor: "0.5" +2021-03-26 08:11:55,333 - max_epochs: "150" +2021-03-26 08:11:55,333 - shuffle: "True" +2021-03-26 08:11:55,333 - train_with_dev: "False" +2021-03-26 08:11:55,334 - batch_growth_annealing: "False" +2021-03-26 08:11:55,334 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:11:55,334 Model training base path: "/home/tmp/megahedm/models/multipos/multipos_UDMADAR_4Diale-LEV_EGY_GLF_MGR__fasttext_flairbwfw__64__0.5_202103260811" +2021-03-26 08:11:55,335 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:11:55,335 Device: cuda:0 +2021-03-26 08:11:55,335 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:11:55,336 Embeddings storage mode: cpu +2021-03-26 08:11:55,338 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:11:57,116 epoch 1 - iter 2/25 - loss 82.69540787 - samples/sec: 72.03 - lr: 0.500000 +2021-03-26 08:11:58,395 epoch 1 - iter 4/25 - loss 85.90545464 - samples/sec: 100.22 - lr: 0.500000 +2021-03-26 08:11:59,684 epoch 1 - iter 6/25 - loss 79.33684158 - samples/sec: 99.42 - lr: 0.500000 +2021-03-26 08:12:00,911 epoch 1 - iter 8/25 - loss 74.56877232 - samples/sec: 104.38 - lr: 0.500000 +2021-03-26 08:12:02,163 epoch 1 - iter 10/25 - loss 73.09503098 - samples/sec: 102.38 - lr: 0.500000 +2021-03-26 08:12:03,374 epoch 1 - iter 12/25 - loss 70.68474229 - samples/sec: 105.82 - lr: 0.500000 +2021-03-26 08:12:04,713 epoch 1 - iter 14/25 - loss 69.38999694 - samples/sec: 95.69 - lr: 0.500000 +2021-03-26 08:12:05,969 epoch 1 - iter 16/25 - loss 68.14279413 - samples/sec: 102.00 - lr: 0.500000 +2021-03-26 08:12:07,290 epoch 1 - iter 18/25 - loss 67.24124633 - samples/sec: 97.01 - lr: 0.500000 +2021-03-26 08:12:08,635 epoch 1 - iter 20/25 - loss 65.85308533 - samples/sec: 95.27 - lr: 0.500000 +2021-03-26 08:12:10,008 epoch 1 - iter 22/25 - loss 64.25220333 - samples/sec: 93.28 - lr: 0.500000 +2021-03-26 08:12:11,403 epoch 1 - iter 24/25 - loss 62.78505087 - samples/sec: 91.84 - lr: 0.500000 +2021-03-26 08:12:11,908 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:12:11,909 EPOCH 1 done: loss 61.9633 - lr 0.5000000 +2021-03-26 08:12:13,113 DEV : loss 43.57872772216797 - score 0.2937 +2021-03-26 08:12:13,135 BAD EPOCHS (no improvement): 0 +2021-03-26 08:12:22,390 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:12:23,329 epoch 2 - iter 2/25 - loss 41.18096542 - samples/sec: 136.45 - lr: 0.500000 +2021-03-26 08:12:24,274 epoch 2 - iter 4/25 - loss 42.09041786 - samples/sec: 135.70 - lr: 0.500000 +2021-03-26 08:12:25,257 epoch 2 - iter 6/25 - loss 41.03157552 - samples/sec: 130.50 - lr: 0.500000 +2021-03-26 08:12:26,236 epoch 2 - iter 8/25 - loss 41.45999718 - samples/sec: 130.92 - lr: 0.500000 +2021-03-26 08:12:27,220 epoch 2 - iter 10/25 - loss 40.17591343 - samples/sec: 130.21 - lr: 0.500000 +2021-03-26 08:12:28,146 epoch 2 - iter 12/25 - loss 39.56078482 - samples/sec: 138.45 - lr: 0.500000 +2021-03-26 08:12:29,183 epoch 2 - iter 14/25 - loss 39.04572909 - samples/sec: 123.64 - lr: 0.500000 +2021-03-26 08:12:30,176 epoch 2 - iter 16/25 - loss 38.69135320 - samples/sec: 129.02 - lr: 0.500000 +2021-03-26 08:12:31,136 epoch 2 - iter 18/25 - loss 38.19763809 - samples/sec: 133.55 - lr: 0.500000 +2021-03-26 08:12:32,073 epoch 2 - iter 20/25 - loss 37.77106895 - samples/sec: 136.77 - lr: 0.500000 +2021-03-26 08:12:33,096 epoch 2 - iter 22/25 - loss 36.93832788 - samples/sec: 125.43 - lr: 0.500000 +2021-03-26 08:12:34,178 epoch 2 - iter 24/25 - loss 36.67819985 - samples/sec: 118.52 - lr: 0.500000 +2021-03-26 08:12:34,651 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:12:34,652 EPOCH 2 done: loss 36.9102 - lr 0.5000000 +2021-03-26 08:12:35,387 DEV : loss 26.169567108154297 - score 0.5504 +2021-03-26 08:12:35,405 BAD EPOCHS (no improvement): 0 +2021-03-26 08:12:45,052 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:12:46,079 epoch 3 - iter 2/25 - loss 27.94248581 - samples/sec: 124.85 - lr: 0.500000 +2021-03-26 08:12:47,115 epoch 3 - iter 4/25 - loss 29.98732948 - samples/sec: 123.76 - lr: 0.500000 +2021-03-26 08:12:48,117 epoch 3 - iter 6/25 - loss 30.17901389 - samples/sec: 127.85 - lr: 0.500000 +2021-03-26 08:12:49,017 epoch 3 - iter 8/25 - loss 29.07717657 - samples/sec: 142.45 - lr: 0.500000 +2021-03-26 08:12:50,013 epoch 3 - iter 10/25 - loss 28.96063709 - samples/sec: 128.80 - lr: 0.500000 +2021-03-26 08:12:50,969 epoch 3 - iter 12/25 - loss 27.79913012 - samples/sec: 134.09 - lr: 0.500000 +2021-03-26 08:12:51,853 epoch 3 - iter 14/25 - loss 26.78517042 - samples/sec: 145.13 - lr: 0.500000 +2021-03-26 08:12:52,818 epoch 3 - iter 16/25 - loss 26.49672520 - samples/sec: 132.85 - lr: 0.500000 +2021-03-26 08:12:53,792 epoch 3 - iter 18/25 - loss 26.31827492 - samples/sec: 131.53 - lr: 0.500000 +2021-03-26 08:12:54,721 epoch 3 - iter 20/25 - loss 26.07491808 - samples/sec: 138.08 - lr: 0.500000 +2021-03-26 08:12:55,655 epoch 3 - iter 22/25 - loss 25.77279256 - samples/sec: 137.18 - lr: 0.500000 +2021-03-26 08:12:56,711 epoch 3 - iter 24/25 - loss 25.54567377 - samples/sec: 121.40 - lr: 0.500000 +2021-03-26 08:12:57,090 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:12:57,091 EPOCH 3 done: loss 25.4674 - lr 0.5000000 +2021-03-26 08:12:57,824 DEV : loss 17.2449951171875 - score 0.6876 +2021-03-26 08:12:57,847 BAD EPOCHS (no improvement): 0 +2021-03-26 08:13:07,348 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:13:08,338 epoch 4 - iter 2/25 - loss 22.18989372 - samples/sec: 129.57 - lr: 0.500000 +2021-03-26 08:13:09,335 epoch 4 - iter 4/25 - loss 21.36251116 - samples/sec: 128.57 - lr: 0.500000 +2021-03-26 08:13:10,370 epoch 4 - iter 6/25 - loss 20.81579622 - samples/sec: 123.77 - lr: 0.500000 +2021-03-26 08:13:11,362 epoch 4 - iter 8/25 - loss 20.94697762 - samples/sec: 129.30 - lr: 0.500000 +2021-03-26 08:13:12,314 epoch 4 - iter 10/25 - loss 20.32834587 - samples/sec: 134.59 - lr: 0.500000 +2021-03-26 08:13:13,332 epoch 4 - iter 12/25 - loss 20.46576881 - samples/sec: 126.00 - lr: 0.500000 +2021-03-26 08:13:14,215 epoch 4 - iter 14/25 - loss 20.12338993 - samples/sec: 145.20 - lr: 0.500000 +2021-03-26 08:13:15,108 epoch 4 - iter 16/25 - loss 19.82321393 - samples/sec: 143.61 - lr: 0.500000 +2021-03-26 08:13:16,155 epoch 4 - iter 18/25 - loss 19.77442031 - samples/sec: 122.47 - lr: 0.500000 +2021-03-26 08:13:17,215 epoch 4 - iter 20/25 - loss 19.73856611 - samples/sec: 120.95 - lr: 0.500000 +2021-03-26 08:13:18,191 epoch 4 - iter 22/25 - loss 19.75611487 - samples/sec: 131.22 - lr: 0.500000 +2021-03-26 08:13:19,153 epoch 4 - iter 24/25 - loss 19.74460936 - samples/sec: 133.20 - lr: 0.500000 +2021-03-26 08:13:19,512 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:13:19,513 EPOCH 4 done: loss 19.7620 - lr 0.5000000 +2021-03-26 08:13:20,225 DEV : loss 14.096415519714355 - score 0.7539 +2021-03-26 08:13:20,247 BAD EPOCHS (no improvement): 0 +2021-03-26 08:13:29,660 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:13:30,708 epoch 5 - iter 2/25 - loss 15.49918652 - samples/sec: 122.31 - lr: 0.500000 +2021-03-26 08:13:31,707 epoch 5 - iter 4/25 - loss 16.34196806 - samples/sec: 128.31 - lr: 0.500000 +2021-03-26 08:13:32,629 epoch 5 - iter 6/25 - loss 16.47365252 - samples/sec: 139.06 - lr: 0.500000 +2021-03-26 08:13:33,588 epoch 5 - iter 8/25 - loss 16.72915411 - samples/sec: 133.72 - lr: 0.500000 +2021-03-26 08:13:34,628 epoch 5 - iter 10/25 - loss 16.65385647 - samples/sec: 123.11 - lr: 0.500000 +2021-03-26 08:13:35,643 epoch 5 - iter 12/25 - loss 16.54397535 - samples/sec: 126.39 - lr: 0.500000 +2021-03-26 08:13:36,664 epoch 5 - iter 14/25 - loss 16.43941729 - samples/sec: 125.44 - lr: 0.500000 +2021-03-26 08:13:37,627 epoch 5 - iter 16/25 - loss 16.51431048 - samples/sec: 133.18 - lr: 0.500000 +2021-03-26 08:13:38,664 epoch 5 - iter 18/25 - loss 16.42917135 - samples/sec: 123.62 - lr: 0.500000 +2021-03-26 08:13:39,605 epoch 5 - iter 20/25 - loss 16.32712135 - samples/sec: 136.30 - lr: 0.500000 +2021-03-26 08:13:40,631 epoch 5 - iter 22/25 - loss 16.20318218 - samples/sec: 124.99 - lr: 0.500000 +2021-03-26 08:13:41,558 epoch 5 - iter 24/25 - loss 16.25766639 - samples/sec: 138.34 - lr: 0.500000 +2021-03-26 08:13:41,957 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:13:41,957 EPOCH 5 done: loss 16.3372 - lr 0.5000000 +2021-03-26 08:13:42,681 DEV : loss 11.58500862121582 - score 0.7945 +2021-03-26 08:13:42,696 BAD EPOCHS (no improvement): 0 +2021-03-26 08:13:52,128 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:13:53,106 epoch 6 - iter 2/25 - loss 13.41134262 - samples/sec: 131.11 - lr: 0.500000 +2021-03-26 08:13:54,017 epoch 6 - iter 4/25 - loss 14.33909249 - samples/sec: 140.75 - lr: 0.500000 +2021-03-26 08:13:54,945 epoch 6 - iter 6/25 - loss 14.27938827 - samples/sec: 138.26 - lr: 0.500000 +2021-03-26 08:13:55,888 epoch 6 - iter 8/25 - loss 13.98244059 - samples/sec: 135.93 - lr: 0.500000 +2021-03-26 08:13:56,896 epoch 6 - iter 10/25 - loss 13.70569067 - samples/sec: 127.04 - lr: 0.500000 +2021-03-26 08:13:57,865 epoch 6 - iter 12/25 - loss 13.65978408 - samples/sec: 132.27 - lr: 0.500000 +2021-03-26 08:13:58,841 epoch 6 - iter 14/25 - loss 14.30981084 - samples/sec: 131.43 - lr: 0.500000 +2021-03-26 08:13:59,783 epoch 6 - iter 16/25 - loss 13.86719906 - samples/sec: 136.12 - lr: 0.500000 +2021-03-26 08:14:00,703 epoch 6 - iter 18/25 - loss 13.85862313 - samples/sec: 139.34 - lr: 0.500000 +2021-03-26 08:14:01,630 epoch 6 - iter 20/25 - loss 13.72562017 - samples/sec: 138.32 - lr: 0.500000 +2021-03-26 08:14:02,592 epoch 6 - iter 22/25 - loss 14.05992985 - samples/sec: 133.30 - lr: 0.500000 +2021-03-26 08:14:03,686 epoch 6 - iter 24/25 - loss 14.23950656 - samples/sec: 117.15 - lr: 0.500000 +2021-03-26 08:14:04,051 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:14:04,052 EPOCH 6 done: loss 14.1535 - lr 0.5000000 +2021-03-26 08:14:04,768 DEV : loss 10.297845840454102 - score 0.8135 +2021-03-26 08:14:04,787 BAD EPOCHS (no improvement): 0 +2021-03-26 08:14:14,125 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:14:15,363 epoch 7 - iter 2/25 - loss 12.14385891 - samples/sec: 103.55 - lr: 0.500000 +2021-03-26 08:14:16,472 epoch 7 - iter 4/25 - loss 12.05147791 - samples/sec: 115.63 - lr: 0.500000 +2021-03-26 08:14:17,559 epoch 7 - iter 6/25 - loss 12.24715741 - samples/sec: 117.85 - lr: 0.500000 +2021-03-26 08:14:18,512 epoch 7 - iter 8/25 - loss 12.49173355 - samples/sec: 134.65 - lr: 0.500000 +2021-03-26 08:14:19,444 epoch 7 - iter 10/25 - loss 12.60267935 - samples/sec: 137.58 - lr: 0.500000 +2021-03-26 08:14:20,450 epoch 7 - iter 12/25 - loss 12.57178235 - samples/sec: 127.45 - lr: 0.500000 +2021-03-26 08:14:21,419 epoch 7 - iter 14/25 - loss 12.35317836 - samples/sec: 132.30 - lr: 0.500000 +2021-03-26 08:14:22,418 epoch 7 - iter 16/25 - loss 12.54999465 - samples/sec: 128.27 - lr: 0.500000 +2021-03-26 08:14:23,487 epoch 7 - iter 18/25 - loss 12.62174733 - samples/sec: 120.03 - lr: 0.500000 +2021-03-26 08:14:24,452 epoch 7 - iter 20/25 - loss 12.63937850 - samples/sec: 132.68 - lr: 0.500000 +2021-03-26 08:14:25,480 epoch 7 - iter 22/25 - loss 12.64787314 - samples/sec: 124.74 - lr: 0.500000 +2021-03-26 08:14:26,472 epoch 7 - iter 24/25 - loss 12.69575051 - samples/sec: 129.18 - lr: 0.500000 +2021-03-26 08:14:26,884 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:14:26,885 EPOCH 7 done: loss 12.8097 - lr 0.5000000 +2021-03-26 08:14:27,615 DEV : loss 8.942085266113281 - score 0.8416 +2021-03-26 08:14:27,639 BAD EPOCHS (no improvement): 0 +2021-03-26 08:14:37,096 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:14:38,050 epoch 8 - iter 2/25 - loss 10.87221813 - samples/sec: 134.38 - lr: 0.500000 +2021-03-26 08:14:39,084 epoch 8 - iter 4/25 - loss 11.29472542 - samples/sec: 123.98 - lr: 0.500000 +2021-03-26 08:14:40,086 epoch 8 - iter 6/25 - loss 12.02312867 - samples/sec: 127.97 - lr: 0.500000 +2021-03-26 08:14:41,132 epoch 8 - iter 8/25 - loss 12.35515392 - samples/sec: 122.41 - lr: 0.500000 +2021-03-26 08:14:42,096 epoch 8 - iter 10/25 - loss 12.06548958 - samples/sec: 133.20 - lr: 0.500000 +2021-03-26 08:14:43,065 epoch 8 - iter 12/25 - loss 11.95635835 - samples/sec: 132.24 - lr: 0.500000 +2021-03-26 08:14:44,011 epoch 8 - iter 14/25 - loss 11.88751956 - samples/sec: 135.54 - lr: 0.500000 +2021-03-26 08:14:44,973 epoch 8 - iter 16/25 - loss 11.72202009 - samples/sec: 133.15 - lr: 0.500000 +2021-03-26 08:14:45,895 epoch 8 - iter 18/25 - loss 11.36020141 - samples/sec: 139.16 - lr: 0.500000 +2021-03-26 08:14:46,979 epoch 8 - iter 20/25 - loss 11.38508224 - samples/sec: 118.18 - lr: 0.500000 +2021-03-26 08:14:48,005 epoch 8 - iter 22/25 - loss 11.59132914 - samples/sec: 124.91 - lr: 0.500000 +2021-03-26 08:14:49,043 epoch 8 - iter 24/25 - loss 11.46632906 - samples/sec: 123.49 - lr: 0.500000 +2021-03-26 08:14:49,450 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:14:49,451 EPOCH 8 done: loss 11.4002 - lr 0.5000000 +2021-03-26 08:14:50,197 DEV : loss 8.089621543884277 - score 0.8515 +2021-03-26 08:14:50,219 BAD EPOCHS (no improvement): 0 +2021-03-26 08:14:59,764 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:15:00,849 epoch 9 - iter 2/25 - loss 10.32193851 - samples/sec: 118.26 - lr: 0.500000 +2021-03-26 08:15:01,921 epoch 9 - iter 4/25 - loss 10.57744074 - samples/sec: 119.55 - lr: 0.500000 +2021-03-26 08:15:02,953 epoch 9 - iter 6/25 - loss 11.03293610 - samples/sec: 124.21 - lr: 0.500000 +2021-03-26 08:15:03,914 epoch 9 - iter 8/25 - loss 10.46654230 - samples/sec: 133.41 - lr: 0.500000 +2021-03-26 08:15:05,033 epoch 9 - iter 10/25 - loss 10.52596345 - samples/sec: 114.52 - lr: 0.500000 +2021-03-26 08:15:06,008 epoch 9 - iter 12/25 - loss 10.55391188 - samples/sec: 131.49 - lr: 0.500000 +2021-03-26 08:15:07,056 epoch 9 - iter 14/25 - loss 10.63699726 - samples/sec: 122.31 - lr: 0.500000 +2021-03-26 08:15:07,977 epoch 9 - iter 16/25 - loss 10.61607608 - samples/sec: 139.16 - lr: 0.500000 +2021-03-26 08:15:08,969 epoch 9 - iter 18/25 - loss 10.64035514 - samples/sec: 129.31 - lr: 0.500000 +2021-03-26 08:15:09,888 epoch 9 - iter 20/25 - loss 10.61658571 - samples/sec: 139.45 - lr: 0.500000 +2021-03-26 08:15:10,802 epoch 9 - iter 22/25 - loss 10.39599518 - samples/sec: 140.27 - lr: 0.500000 +2021-03-26 08:15:11,783 epoch 9 - iter 24/25 - loss 10.44796216 - samples/sec: 130.70 - lr: 0.500000 +2021-03-26 08:15:12,143 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:15:12,144 EPOCH 9 done: loss 10.4188 - lr 0.5000000 +2021-03-26 08:15:12,914 DEV : loss 7.821622371673584 - score 0.8622 +2021-03-26 08:15:12,938 BAD EPOCHS (no improvement): 0 +2021-03-26 08:15:22,456 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:15:23,458 epoch 10 - iter 2/25 - loss 7.78988838 - samples/sec: 127.94 - lr: 0.500000 +2021-03-26 08:15:24,633 epoch 10 - iter 4/25 - loss 8.20438027 - samples/sec: 109.09 - lr: 0.500000 +2021-03-26 08:15:25,654 epoch 10 - iter 6/25 - loss 8.11983109 - samples/sec: 125.65 - lr: 0.500000 +2021-03-26 08:15:26,838 epoch 10 - iter 8/25 - loss 8.96964788 - samples/sec: 108.22 - lr: 0.500000 +2021-03-26 08:15:28,017 epoch 10 - iter 10/25 - loss 8.98561802 - samples/sec: 108.78 - lr: 0.500000 +2021-03-26 08:15:29,215 epoch 10 - iter 12/25 - loss 8.97878774 - samples/sec: 107.01 - lr: 0.500000 +2021-03-26 08:15:30,313 epoch 10 - iter 14/25 - loss 9.11639765 - samples/sec: 116.67 - lr: 0.500000 +2021-03-26 08:15:31,319 epoch 10 - iter 16/25 - loss 9.25904155 - samples/sec: 127.49 - lr: 0.500000 +2021-03-26 08:15:32,356 epoch 10 - iter 18/25 - loss 9.40983295 - samples/sec: 123.64 - lr: 0.500000 +2021-03-26 08:15:33,580 epoch 10 - iter 20/25 - loss 9.54233298 - samples/sec: 104.71 - lr: 0.500000 +2021-03-26 08:15:34,844 epoch 10 - iter 22/25 - loss 9.56605933 - samples/sec: 101.39 - lr: 0.500000 +2021-03-26 08:15:36,143 epoch 10 - iter 24/25 - loss 9.59460024 - samples/sec: 98.64 - lr: 0.500000 +2021-03-26 08:15:36,632 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:15:36,633 EPOCH 10 done: loss 9.5178 - lr 0.5000000 +2021-03-26 08:15:37,368 DEV : loss 7.125051021575928 - score 0.8689 +2021-03-26 08:15:37,390 BAD EPOCHS (no improvement): 0 +2021-03-26 08:15:47,068 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:15:48,091 epoch 11 - iter 2/25 - loss 8.83649921 - samples/sec: 125.38 - lr: 0.500000 +2021-03-26 08:15:49,052 epoch 11 - iter 4/25 - loss 9.08403540 - samples/sec: 133.36 - lr: 0.500000 +2021-03-26 08:15:50,009 epoch 11 - iter 6/25 - loss 8.58666507 - samples/sec: 133.97 - lr: 0.500000 +2021-03-26 08:15:50,960 epoch 11 - iter 8/25 - loss 9.06016153 - samples/sec: 134.83 - lr: 0.500000 +2021-03-26 08:15:52,063 epoch 11 - iter 10/25 - loss 8.88002706 - samples/sec: 116.22 - lr: 0.500000 +2021-03-26 08:15:53,103 epoch 11 - iter 12/25 - loss 8.69153821 - samples/sec: 123.25 - lr: 0.500000 +2021-03-26 08:15:54,112 epoch 11 - iter 14/25 - loss 8.90236327 - samples/sec: 127.00 - lr: 0.500000 +2021-03-26 08:15:55,164 epoch 11 - iter 16/25 - loss 8.69934401 - samples/sec: 121.84 - lr: 0.500000 +2021-03-26 08:15:56,431 epoch 11 - iter 18/25 - loss 8.75323354 - samples/sec: 101.24 - lr: 0.500000 +2021-03-26 08:15:57,514 epoch 11 - iter 20/25 - loss 8.71253555 - samples/sec: 118.28 - lr: 0.500000 +2021-03-26 08:15:58,470 epoch 11 - iter 22/25 - loss 8.70876809 - samples/sec: 134.25 - lr: 0.500000 +2021-03-26 08:15:59,486 epoch 11 - iter 24/25 - loss 8.68266873 - samples/sec: 126.18 - lr: 0.500000 +2021-03-26 08:15:59,912 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:15:59,913 EPOCH 11 done: loss 8.8185 - lr 0.5000000 +2021-03-26 08:16:00,629 DEV : loss 6.984427452087402 - score 0.8739 +2021-03-26 08:16:00,651 BAD EPOCHS (no improvement): 0 +2021-03-26 08:16:10,141 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:16:11,372 epoch 12 - iter 2/25 - loss 7.81654215 - samples/sec: 104.18 - lr: 0.500000 +2021-03-26 08:16:12,442 epoch 12 - iter 4/25 - loss 7.47270012 - samples/sec: 119.91 - lr: 0.500000 +2021-03-26 08:16:13,384 epoch 12 - iter 6/25 - loss 8.39132531 - samples/sec: 136.03 - lr: 0.500000 +2021-03-26 08:16:14,323 epoch 12 - iter 8/25 - loss 8.67520797 - samples/sec: 136.58 - lr: 0.500000 +2021-03-26 08:16:15,252 epoch 12 - iter 10/25 - loss 8.53146949 - samples/sec: 137.97 - lr: 0.500000 +2021-03-26 08:16:16,288 epoch 12 - iter 12/25 - loss 8.33992930 - samples/sec: 123.67 - lr: 0.500000 +2021-03-26 08:16:17,412 epoch 12 - iter 14/25 - loss 8.22010701 - samples/sec: 113.99 - lr: 0.500000 +2021-03-26 08:16:18,351 epoch 12 - iter 16/25 - loss 8.26404691 - samples/sec: 136.59 - lr: 0.500000 +2021-03-26 08:16:19,321 epoch 12 - iter 18/25 - loss 8.19568605 - samples/sec: 132.22 - lr: 0.500000 +2021-03-26 08:16:20,420 epoch 12 - iter 20/25 - loss 8.29496424 - samples/sec: 116.60 - lr: 0.500000 +2021-03-26 08:16:21,292 epoch 12 - iter 22/25 - loss 8.28023763 - samples/sec: 147.05 - lr: 0.500000 +2021-03-26 08:16:22,376 epoch 12 - iter 24/25 - loss 8.29219141 - samples/sec: 118.19 - lr: 0.500000 +2021-03-26 08:16:22,776 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:16:22,777 EPOCH 12 done: loss 8.4431 - lr 0.5000000 +2021-03-26 08:16:23,488 DEV : loss 7.6165900230407715 - score 0.868 +2021-03-26 08:16:23,505 BAD EPOCHS (no improvement): 1 +2021-03-26 08:16:23,506 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:16:24,580 epoch 13 - iter 2/25 - loss 8.72345257 - samples/sec: 119.32 - lr: 0.500000 +2021-03-26 08:16:25,643 epoch 13 - iter 4/25 - loss 7.85941982 - samples/sec: 120.66 - lr: 0.500000 +2021-03-26 08:16:26,713 epoch 13 - iter 6/25 - loss 7.69082999 - samples/sec: 119.86 - lr: 0.500000 +2021-03-26 08:16:27,671 epoch 13 - iter 8/25 - loss 7.97219425 - samples/sec: 134.40 - lr: 0.500000 +2021-03-26 08:16:28,664 epoch 13 - iter 10/25 - loss 7.87737551 - samples/sec: 129.00 - lr: 0.500000 +2021-03-26 08:16:29,764 epoch 13 - iter 12/25 - loss 7.69054703 - samples/sec: 116.50 - lr: 0.500000 +2021-03-26 08:16:30,805 epoch 13 - iter 14/25 - loss 7.75256617 - samples/sec: 123.24 - lr: 0.500000 +2021-03-26 08:16:31,828 epoch 13 - iter 16/25 - loss 7.90568480 - samples/sec: 125.28 - lr: 0.500000 +2021-03-26 08:16:32,788 epoch 13 - iter 18/25 - loss 7.90655682 - samples/sec: 133.51 - lr: 0.500000 +2021-03-26 08:16:33,932 epoch 13 - iter 20/25 - loss 7.88109205 - samples/sec: 111.97 - lr: 0.500000 +2021-03-26 08:16:34,959 epoch 13 - iter 22/25 - loss 8.02318389 - samples/sec: 124.85 - lr: 0.500000 +2021-03-26 08:16:35,870 epoch 13 - iter 24/25 - loss 7.88861491 - samples/sec: 140.80 - lr: 0.500000 +2021-03-26 08:16:36,256 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:16:36,257 EPOCH 13 done: loss 7.8336 - lr 0.5000000 +2021-03-26 08:16:36,986 DEV : loss 6.979231834411621 - score 0.871 +2021-03-26 08:16:37,001 BAD EPOCHS (no improvement): 2 +2021-03-26 08:16:37,002 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:16:37,970 epoch 14 - iter 2/25 - loss 10.96346951 - samples/sec: 132.45 - lr: 0.500000 +2021-03-26 08:16:39,003 epoch 14 - iter 4/25 - loss 10.93682742 - samples/sec: 124.15 - lr: 0.500000 +2021-03-26 08:16:40,104 epoch 14 - iter 6/25 - loss 9.74375391 - samples/sec: 116.45 - lr: 0.500000 +2021-03-26 08:16:41,013 epoch 14 - iter 8/25 - loss 9.09124070 - samples/sec: 140.95 - lr: 0.500000 +2021-03-26 08:16:41,977 epoch 14 - iter 10/25 - loss 8.89416289 - samples/sec: 133.06 - lr: 0.500000 +2021-03-26 08:16:43,015 epoch 14 - iter 12/25 - loss 8.60138631 - samples/sec: 123.50 - lr: 0.500000 +2021-03-26 08:16:43,950 epoch 14 - iter 14/25 - loss 8.30748442 - samples/sec: 137.80 - lr: 0.500000 +2021-03-26 08:16:44,987 epoch 14 - iter 16/25 - loss 8.26665404 - samples/sec: 123.64 - lr: 0.500000 +2021-03-26 08:16:46,067 epoch 14 - iter 18/25 - loss 8.19610063 - samples/sec: 118.69 - lr: 0.500000 +2021-03-26 08:16:46,993 epoch 14 - iter 20/25 - loss 8.22014797 - samples/sec: 138.49 - lr: 0.500000 +2021-03-26 08:16:48,144 epoch 14 - iter 22/25 - loss 8.11139623 - samples/sec: 111.27 - lr: 0.500000 +2021-03-26 08:16:49,092 epoch 14 - iter 24/25 - loss 8.05588663 - samples/sec: 135.35 - lr: 0.500000 +2021-03-26 08:16:49,531 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:16:49,531 EPOCH 14 done: loss 8.0288 - lr 0.5000000 +2021-03-26 08:16:50,260 DEV : loss 6.266796112060547 - score 0.8801 +2021-03-26 08:16:50,283 BAD EPOCHS (no improvement): 0 +2021-03-26 08:16:59,697 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:17:00,687 epoch 15 - iter 2/25 - loss 6.77654982 - samples/sec: 129.65 - lr: 0.500000 +2021-03-26 08:17:01,722 epoch 15 - iter 4/25 - loss 6.73212910 - samples/sec: 123.75 - lr: 0.500000 +2021-03-26 08:17:02,675 epoch 15 - iter 6/25 - loss 7.18263904 - samples/sec: 134.58 - lr: 0.500000 +2021-03-26 08:17:03,684 epoch 15 - iter 8/25 - loss 7.14034599 - samples/sec: 126.98 - lr: 0.500000 +2021-03-26 08:17:04,614 epoch 15 - iter 10/25 - loss 7.17819848 - samples/sec: 137.83 - lr: 0.500000 +2021-03-26 08:17:05,588 epoch 15 - iter 12/25 - loss 6.99284558 - samples/sec: 131.61 - lr: 0.500000 +2021-03-26 08:17:06,602 epoch 15 - iter 14/25 - loss 7.10860590 - samples/sec: 126.46 - lr: 0.500000 +2021-03-26 08:17:07,648 epoch 15 - iter 16/25 - loss 7.04990083 - samples/sec: 122.49 - lr: 0.500000 +2021-03-26 08:17:08,616 epoch 15 - iter 18/25 - loss 7.14585771 - samples/sec: 132.47 - lr: 0.500000 +2021-03-26 08:17:09,651 epoch 15 - iter 20/25 - loss 7.15546744 - samples/sec: 123.76 - lr: 0.500000 +2021-03-26 08:17:10,651 epoch 15 - iter 22/25 - loss 7.10891477 - samples/sec: 128.28 - lr: 0.500000 +2021-03-26 08:17:11,784 epoch 15 - iter 24/25 - loss 7.15305050 - samples/sec: 113.18 - lr: 0.500000 +2021-03-26 08:17:12,158 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:17:12,159 EPOCH 15 done: loss 7.1976 - lr 0.5000000 +2021-03-26 08:17:12,887 DEV : loss 6.324016571044922 - score 0.8895 +2021-03-26 08:17:12,911 BAD EPOCHS (no improvement): 0 +2021-03-26 08:17:22,753 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:17:23,828 epoch 16 - iter 2/25 - loss 6.31022072 - samples/sec: 119.38 - lr: 0.500000 +2021-03-26 08:17:24,818 epoch 16 - iter 4/25 - loss 6.49304485 - samples/sec: 129.56 - lr: 0.500000 +2021-03-26 08:17:25,780 epoch 16 - iter 6/25 - loss 6.19578926 - samples/sec: 133.36 - lr: 0.500000 +2021-03-26 08:17:26,857 epoch 16 - iter 8/25 - loss 5.96798402 - samples/sec: 119.01 - lr: 0.500000 +2021-03-26 08:17:27,781 epoch 16 - iter 10/25 - loss 6.17498813 - samples/sec: 138.72 - lr: 0.500000 +2021-03-26 08:17:28,737 epoch 16 - iter 12/25 - loss 6.21972493 - samples/sec: 134.08 - lr: 0.500000 +2021-03-26 08:17:30,084 epoch 16 - iter 14/25 - loss 6.37837097 - samples/sec: 95.18 - lr: 0.500000 +2021-03-26 08:17:31,199 epoch 16 - iter 16/25 - loss 6.60098591 - samples/sec: 114.97 - lr: 0.500000 +2021-03-26 08:17:32,145 epoch 16 - iter 18/25 - loss 6.65434315 - samples/sec: 135.50 - lr: 0.500000 +2021-03-26 08:17:33,129 epoch 16 - iter 20/25 - loss 6.71629548 - samples/sec: 130.24 - lr: 0.500000 +2021-03-26 08:17:34,075 epoch 16 - iter 22/25 - loss 6.70602003 - samples/sec: 135.45 - lr: 0.500000 +2021-03-26 08:17:35,083 epoch 16 - iter 24/25 - loss 6.80145276 - samples/sec: 127.22 - lr: 0.500000 +2021-03-26 08:17:35,425 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:17:35,426 EPOCH 16 done: loss 6.8080 - lr 0.5000000 +2021-03-26 08:17:36,127 DEV : loss 5.873531341552734 - score 0.8974 +2021-03-26 08:17:36,149 BAD EPOCHS (no improvement): 0 +2021-03-26 08:17:45,651 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:17:46,682 epoch 17 - iter 2/25 - loss 5.51331139 - samples/sec: 124.48 - lr: 0.500000 +2021-03-26 08:17:47,676 epoch 17 - iter 4/25 - loss 5.94864118 - samples/sec: 128.92 - lr: 0.500000 +2021-03-26 08:17:48,744 epoch 17 - iter 6/25 - loss 5.79273653 - samples/sec: 120.06 - lr: 0.500000 +2021-03-26 08:17:49,727 epoch 17 - iter 8/25 - loss 6.06304932 - samples/sec: 130.47 - lr: 0.500000 +2021-03-26 08:17:50,846 epoch 17 - iter 10/25 - loss 6.33874021 - samples/sec: 114.54 - lr: 0.500000 +2021-03-26 08:17:51,925 epoch 17 - iter 12/25 - loss 6.44700205 - samples/sec: 118.83 - lr: 0.500000 +2021-03-26 08:17:52,908 epoch 17 - iter 14/25 - loss 6.53073512 - samples/sec: 130.48 - lr: 0.500000 +2021-03-26 08:17:53,839 epoch 17 - iter 16/25 - loss 6.51271585 - samples/sec: 137.65 - lr: 0.500000 +2021-03-26 08:17:54,870 epoch 17 - iter 18/25 - loss 6.59723255 - samples/sec: 124.42 - lr: 0.500000 +2021-03-26 08:17:55,905 epoch 17 - iter 20/25 - loss 6.51246181 - samples/sec: 123.92 - lr: 0.500000 +2021-03-26 08:17:56,879 epoch 17 - iter 22/25 - loss 6.48072837 - samples/sec: 131.48 - lr: 0.500000 +2021-03-26 08:17:57,819 epoch 17 - iter 24/25 - loss 6.57046958 - samples/sec: 136.45 - lr: 0.500000 +2021-03-26 08:17:58,225 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:17:58,226 EPOCH 17 done: loss 6.5876 - lr 0.5000000 +2021-03-26 08:17:58,950 DEV : loss 5.898545265197754 - score 0.8946 +2021-03-26 08:17:58,973 BAD EPOCHS (no improvement): 1 +2021-03-26 08:17:58,974 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:17:59,948 epoch 18 - iter 2/25 - loss 5.93250608 - samples/sec: 131.67 - lr: 0.500000 +2021-03-26 08:18:00,989 epoch 18 - iter 4/25 - loss 6.21452951 - samples/sec: 123.26 - lr: 0.500000 +2021-03-26 08:18:01,997 epoch 18 - iter 6/25 - loss 6.38846231 - samples/sec: 127.17 - lr: 0.500000 +2021-03-26 08:18:02,914 epoch 18 - iter 8/25 - loss 6.20925385 - samples/sec: 139.71 - lr: 0.500000 +2021-03-26 08:18:03,951 epoch 18 - iter 10/25 - loss 6.16746583 - samples/sec: 123.66 - lr: 0.500000 +2021-03-26 08:18:04,918 epoch 18 - iter 12/25 - loss 6.01534859 - samples/sec: 132.65 - lr: 0.500000 +2021-03-26 08:18:06,023 epoch 18 - iter 14/25 - loss 6.08419527 - samples/sec: 116.01 - lr: 0.500000 +2021-03-26 08:18:07,122 epoch 18 - iter 16/25 - loss 6.03223971 - samples/sec: 116.67 - lr: 0.500000 +2021-03-26 08:18:08,136 epoch 18 - iter 18/25 - loss 6.11314792 - samples/sec: 126.53 - lr: 0.500000 +2021-03-26 08:18:09,157 epoch 18 - iter 20/25 - loss 6.26362159 - samples/sec: 125.59 - lr: 0.500000 +2021-03-26 08:18:10,088 epoch 18 - iter 22/25 - loss 6.25091089 - samples/sec: 137.61 - lr: 0.500000 +2021-03-26 08:18:11,026 epoch 18 - iter 24/25 - loss 6.25768552 - samples/sec: 136.61 - lr: 0.500000 +2021-03-26 08:18:11,418 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:18:11,419 EPOCH 18 done: loss 6.2439 - lr 0.5000000 +2021-03-26 08:18:12,122 DEV : loss 5.98818302154541 - score 0.8864 +2021-03-26 08:18:12,146 BAD EPOCHS (no improvement): 2 +2021-03-26 08:18:12,146 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:18:13,158 epoch 19 - iter 2/25 - loss 5.93176770 - samples/sec: 126.80 - lr: 0.500000 +2021-03-26 08:18:14,110 epoch 19 - iter 4/25 - loss 6.19001615 - samples/sec: 134.59 - lr: 0.500000 +2021-03-26 08:18:15,191 epoch 19 - iter 6/25 - loss 5.82322733 - samples/sec: 118.56 - lr: 0.500000 +2021-03-26 08:18:16,218 epoch 19 - iter 8/25 - loss 5.79541320 - samples/sec: 124.75 - lr: 0.500000 +2021-03-26 08:18:17,246 epoch 19 - iter 10/25 - loss 5.93873725 - samples/sec: 124.79 - lr: 0.500000 +2021-03-26 08:18:18,401 epoch 19 - iter 12/25 - loss 5.89532459 - samples/sec: 111.03 - lr: 0.500000 +2021-03-26 08:18:19,426 epoch 19 - iter 14/25 - loss 5.97058112 - samples/sec: 125.05 - lr: 0.500000 +2021-03-26 08:18:20,416 epoch 19 - iter 16/25 - loss 5.94304895 - samples/sec: 129.52 - lr: 0.500000 +2021-03-26 08:18:21,406 epoch 19 - iter 18/25 - loss 5.90629564 - samples/sec: 129.46 - lr: 0.500000 +2021-03-26 08:18:22,367 epoch 19 - iter 20/25 - loss 5.90371339 - samples/sec: 133.58 - lr: 0.500000 +2021-03-26 08:18:23,426 epoch 19 - iter 22/25 - loss 6.05487739 - samples/sec: 121.06 - lr: 0.500000 +2021-03-26 08:18:24,422 epoch 19 - iter 24/25 - loss 6.00732064 - samples/sec: 128.59 - lr: 0.500000 +2021-03-26 08:18:24,861 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:18:24,862 EPOCH 19 done: loss 6.0482 - lr 0.5000000 +2021-03-26 08:18:25,576 DEV : loss 5.909600734710693 - score 0.8913 +2021-03-26 08:18:25,594 BAD EPOCHS (no improvement): 3 +2021-03-26 08:18:25,595 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:18:26,604 epoch 20 - iter 2/25 - loss 5.59814882 - samples/sec: 126.97 - lr: 0.500000 +2021-03-26 08:18:27,785 epoch 20 - iter 4/25 - loss 5.75239575 - samples/sec: 108.51 - lr: 0.500000 +2021-03-26 08:18:28,743 epoch 20 - iter 6/25 - loss 5.82471029 - samples/sec: 133.87 - lr: 0.500000 +2021-03-26 08:18:29,809 epoch 20 - iter 8/25 - loss 5.66427326 - samples/sec: 120.18 - lr: 0.500000 +2021-03-26 08:18:30,758 epoch 20 - iter 10/25 - loss 5.80760927 - samples/sec: 135.17 - lr: 0.500000 +2021-03-26 08:18:31,810 epoch 20 - iter 12/25 - loss 5.71123417 - samples/sec: 121.93 - lr: 0.500000 +2021-03-26 08:18:32,776 epoch 20 - iter 14/25 - loss 5.66636910 - samples/sec: 132.71 - lr: 0.500000 +2021-03-26 08:18:34,805 epoch 20 - iter 16/25 - loss 5.60936448 - samples/sec: 63.13 - lr: 0.500000 +2021-03-26 08:18:35,716 epoch 20 - iter 18/25 - loss 5.57157196 - samples/sec: 140.69 - lr: 0.500000 +2021-03-26 08:18:36,716 epoch 20 - iter 20/25 - loss 5.68519535 - samples/sec: 128.27 - lr: 0.500000 +2021-03-26 08:18:37,716 epoch 20 - iter 22/25 - loss 5.76139723 - samples/sec: 128.10 - lr: 0.500000 +2021-03-26 08:18:38,714 epoch 20 - iter 24/25 - loss 5.81321446 - samples/sec: 129.07 - lr: 0.500000 +2021-03-26 08:18:39,177 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:18:39,178 EPOCH 20 done: loss 5.8133 - lr 0.5000000 +2021-03-26 08:18:39,875 DEV : loss 5.780816555023193 - score 0.8964 +2021-03-26 08:18:39,892 BAD EPOCHS (no improvement): 4 +2021-03-26 08:18:39,892 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:18:40,893 epoch 21 - iter 2/25 - loss 5.82937956 - samples/sec: 128.05 - lr: 0.250000 +2021-03-26 08:18:41,862 epoch 21 - iter 4/25 - loss 5.56850576 - samples/sec: 132.20 - lr: 0.250000 +2021-03-26 08:18:42,899 epoch 21 - iter 6/25 - loss 5.31449374 - samples/sec: 123.67 - lr: 0.250000 +2021-03-26 08:18:43,865 epoch 21 - iter 8/25 - loss 5.22509336 - samples/sec: 132.62 - lr: 0.250000 +2021-03-26 08:18:44,906 epoch 21 - iter 10/25 - loss 5.08971348 - samples/sec: 123.11 - lr: 0.250000 +2021-03-26 08:18:45,948 epoch 21 - iter 12/25 - loss 5.15612626 - samples/sec: 123.00 - lr: 0.250000 +2021-03-26 08:18:46,941 epoch 21 - iter 14/25 - loss 5.17431855 - samples/sec: 129.13 - lr: 0.250000 +2021-03-26 08:18:47,809 epoch 21 - iter 16/25 - loss 4.99487485 - samples/sec: 147.69 - lr: 0.250000 +2021-03-26 08:18:48,772 epoch 21 - iter 18/25 - loss 5.04701886 - samples/sec: 133.08 - lr: 0.250000 +2021-03-26 08:18:49,714 epoch 21 - iter 20/25 - loss 5.06399451 - samples/sec: 136.13 - lr: 0.250000 +2021-03-26 08:18:50,770 epoch 21 - iter 22/25 - loss 5.08102173 - samples/sec: 121.44 - lr: 0.250000 +2021-03-26 08:18:51,678 epoch 21 - iter 24/25 - loss 4.95142464 - samples/sec: 141.33 - lr: 0.250000 +2021-03-26 08:18:52,070 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:18:52,071 EPOCH 21 done: loss 4.9290 - lr 0.2500000 +2021-03-26 08:18:52,801 DEV : loss 5.545863628387451 - score 0.9061 +2021-03-26 08:18:52,828 BAD EPOCHS (no improvement): 0 +2021-03-26 08:19:02,387 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:19:03,438 epoch 22 - iter 2/25 - loss 4.39438319 - samples/sec: 122.01 - lr: 0.250000 +2021-03-26 08:19:04,390 epoch 22 - iter 4/25 - loss 4.97888792 - samples/sec: 134.67 - lr: 0.250000 +2021-03-26 08:19:05,308 epoch 22 - iter 6/25 - loss 4.67588596 - samples/sec: 139.59 - lr: 0.250000 +2021-03-26 08:19:06,252 epoch 22 - iter 8/25 - loss 4.68256471 - samples/sec: 135.69 - lr: 0.250000 +2021-03-26 08:19:07,180 epoch 22 - iter 10/25 - loss 4.66318874 - samples/sec: 138.33 - lr: 0.250000 +2021-03-26 08:19:08,165 epoch 22 - iter 12/25 - loss 4.74354013 - samples/sec: 130.11 - lr: 0.250000 +2021-03-26 08:19:09,230 epoch 22 - iter 14/25 - loss 4.78080085 - samples/sec: 120.36 - lr: 0.250000 +2021-03-26 08:19:10,157 epoch 22 - iter 16/25 - loss 4.75316647 - samples/sec: 138.46 - lr: 0.250000 +2021-03-26 08:19:11,249 epoch 22 - iter 18/25 - loss 4.82255493 - samples/sec: 117.31 - lr: 0.250000 +2021-03-26 08:19:12,189 epoch 22 - iter 20/25 - loss 4.78287302 - samples/sec: 136.41 - lr: 0.250000 +2021-03-26 08:19:13,247 epoch 22 - iter 22/25 - loss 4.78834465 - samples/sec: 121.13 - lr: 0.250000 +2021-03-26 08:19:14,583 epoch 22 - iter 24/25 - loss 4.79762895 - samples/sec: 95.93 - lr: 0.250000 +2021-03-26 08:19:15,129 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:19:15,130 EPOCH 22 done: loss 4.8014 - lr 0.2500000 +2021-03-26 08:19:15,915 DEV : loss 5.468145370483398 - score 0.9082 +2021-03-26 08:19:15,947 BAD EPOCHS (no improvement): 0 +2021-03-26 08:19:25,505 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:19:26,605 epoch 23 - iter 2/25 - loss 3.57932007 - samples/sec: 116.69 - lr: 0.250000 +2021-03-26 08:19:27,637 epoch 23 - iter 4/25 - loss 4.20975953 - samples/sec: 124.18 - lr: 0.250000 +2021-03-26 08:19:28,685 epoch 23 - iter 6/25 - loss 4.19612292 - samples/sec: 122.23 - lr: 0.250000 +2021-03-26 08:19:29,666 epoch 23 - iter 8/25 - loss 4.45020649 - samples/sec: 130.73 - lr: 0.250000 +2021-03-26 08:19:30,721 epoch 23 - iter 10/25 - loss 4.41350362 - samples/sec: 121.48 - lr: 0.250000 +2021-03-26 08:19:31,691 epoch 23 - iter 12/25 - loss 4.52760651 - samples/sec: 132.20 - lr: 0.250000 +2021-03-26 08:19:32,715 epoch 23 - iter 14/25 - loss 4.49416452 - samples/sec: 125.21 - lr: 0.250000 +2021-03-26 08:19:33,687 epoch 23 - iter 16/25 - loss 4.55967647 - samples/sec: 131.85 - lr: 0.250000 +2021-03-26 08:19:34,644 epoch 23 - iter 18/25 - loss 4.47532535 - samples/sec: 133.90 - lr: 0.250000 +2021-03-26 08:19:35,578 epoch 23 - iter 20/25 - loss 4.42298470 - samples/sec: 137.20 - lr: 0.250000 +2021-03-26 08:19:36,601 epoch 23 - iter 22/25 - loss 4.43825457 - samples/sec: 125.35 - lr: 0.250000 +2021-03-26 08:19:37,737 epoch 23 - iter 24/25 - loss 4.45593623 - samples/sec: 112.76 - lr: 0.250000 +2021-03-26 08:19:38,313 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:19:38,314 EPOCH 23 done: loss 4.4883 - lr 0.2500000 +2021-03-26 08:19:39,090 DEV : loss 5.342250823974609 - score 0.9046 +2021-03-26 08:19:39,115 BAD EPOCHS (no improvement): 1 +2021-03-26 08:19:39,116 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:19:40,413 epoch 24 - iter 2/25 - loss 4.78895926 - samples/sec: 98.89 - lr: 0.250000 +2021-03-26 08:19:41,367 epoch 24 - iter 4/25 - loss 4.32054168 - samples/sec: 134.38 - lr: 0.250000 +2021-03-26 08:19:42,376 epoch 24 - iter 6/25 - loss 4.13959487 - samples/sec: 127.06 - lr: 0.250000 +2021-03-26 08:19:43,311 epoch 24 - iter 8/25 - loss 4.05541480 - samples/sec: 137.20 - lr: 0.250000 +2021-03-26 08:19:44,302 epoch 24 - iter 10/25 - loss 4.31926289 - samples/sec: 129.38 - lr: 0.250000 +2021-03-26 08:19:45,187 epoch 24 - iter 12/25 - loss 4.26717488 - samples/sec: 145.02 - lr: 0.250000 +2021-03-26 08:19:46,212 epoch 24 - iter 14/25 - loss 4.47580140 - samples/sec: 124.99 - lr: 0.250000 +2021-03-26 08:19:47,285 epoch 24 - iter 16/25 - loss 4.44438323 - samples/sec: 119.49 - lr: 0.250000 +2021-03-26 08:19:48,375 epoch 24 - iter 18/25 - loss 4.43706428 - samples/sec: 117.66 - lr: 0.250000 +2021-03-26 08:19:49,363 epoch 24 - iter 20/25 - loss 4.40470082 - samples/sec: 129.78 - lr: 0.250000 +2021-03-26 08:19:50,622 epoch 24 - iter 22/25 - loss 4.38354238 - samples/sec: 101.83 - lr: 0.250000 +2021-03-26 08:19:51,616 epoch 24 - iter 24/25 - loss 4.36244129 - samples/sec: 128.90 - lr: 0.250000 +2021-03-26 08:19:51,988 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:19:51,988 EPOCH 24 done: loss 4.3722 - lr 0.2500000 +2021-03-26 08:19:52,696 DEV : loss 5.464868068695068 - score 0.9065 +2021-03-26 08:19:52,718 BAD EPOCHS (no improvement): 2 +2021-03-26 08:19:52,719 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:19:53,713 epoch 25 - iter 2/25 - loss 4.12841630 - samples/sec: 128.95 - lr: 0.250000 +2021-03-26 08:19:54,650 epoch 25 - iter 4/25 - loss 4.23390102 - samples/sec: 136.91 - lr: 0.250000 +2021-03-26 08:19:55,631 epoch 25 - iter 6/25 - loss 4.19182324 - samples/sec: 130.70 - lr: 0.250000 +2021-03-26 08:19:56,678 epoch 25 - iter 8/25 - loss 4.14069355 - samples/sec: 122.38 - lr: 0.250000 +2021-03-26 08:19:57,614 epoch 25 - iter 10/25 - loss 4.11968174 - samples/sec: 137.05 - lr: 0.250000 +2021-03-26 08:19:58,695 epoch 25 - iter 12/25 - loss 4.03539999 - samples/sec: 118.61 - lr: 0.250000 +2021-03-26 08:19:59,628 epoch 25 - iter 14/25 - loss 4.01701995 - samples/sec: 137.30 - lr: 0.250000 +2021-03-26 08:20:00,655 epoch 25 - iter 16/25 - loss 4.00432700 - samples/sec: 124.91 - lr: 0.250000 +2021-03-26 08:20:01,587 epoch 25 - iter 18/25 - loss 4.02359321 - samples/sec: 137.46 - lr: 0.250000 +2021-03-26 08:20:02,633 epoch 25 - iter 20/25 - loss 4.16669927 - samples/sec: 122.58 - lr: 0.250000 +2021-03-26 08:20:03,623 epoch 25 - iter 22/25 - loss 4.17694152 - samples/sec: 129.43 - lr: 0.250000 +2021-03-26 08:20:04,606 epoch 25 - iter 24/25 - loss 4.19949826 - samples/sec: 130.35 - lr: 0.250000 +2021-03-26 08:20:05,000 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:20:05,001 EPOCH 25 done: loss 4.2124 - lr 0.2500000 +2021-03-26 08:20:05,735 DEV : loss 5.251356601715088 - score 0.9088 +2021-03-26 08:20:05,767 BAD EPOCHS (no improvement): 0 +2021-03-26 08:20:15,151 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:20:16,117 epoch 26 - iter 2/25 - loss 3.64543915 - samples/sec: 132.79 - lr: 0.250000 +2021-03-26 08:20:17,159 epoch 26 - iter 4/25 - loss 3.60610306 - samples/sec: 122.93 - lr: 0.250000 +2021-03-26 08:20:18,177 epoch 26 - iter 6/25 - loss 3.79775095 - samples/sec: 126.03 - lr: 0.250000 +2021-03-26 08:20:19,160 epoch 26 - iter 8/25 - loss 4.02053875 - samples/sec: 130.39 - lr: 0.250000 +2021-03-26 08:20:20,130 epoch 26 - iter 10/25 - loss 4.08083537 - samples/sec: 132.16 - lr: 0.250000 +2021-03-26 08:20:21,211 epoch 26 - iter 12/25 - loss 4.25827795 - samples/sec: 118.56 - lr: 0.250000 +2021-03-26 08:20:22,244 epoch 26 - iter 14/25 - loss 4.30301195 - samples/sec: 124.08 - lr: 0.250000 +2021-03-26 08:20:23,185 epoch 26 - iter 16/25 - loss 4.31946926 - samples/sec: 136.43 - lr: 0.250000 +2021-03-26 08:20:24,115 epoch 26 - iter 18/25 - loss 4.16990701 - samples/sec: 137.79 - lr: 0.250000 +2021-03-26 08:20:25,151 epoch 26 - iter 20/25 - loss 4.20403528 - samples/sec: 123.68 - lr: 0.250000 +2021-03-26 08:20:26,090 epoch 26 - iter 22/25 - loss 4.15471683 - samples/sec: 136.55 - lr: 0.250000 +2021-03-26 08:20:27,107 epoch 26 - iter 24/25 - loss 4.15259549 - samples/sec: 126.08 - lr: 0.250000 +2021-03-26 08:20:27,503 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:20:27,503 EPOCH 26 done: loss 4.1552 - lr 0.2500000 +2021-03-26 08:20:28,206 DEV : loss 5.207316875457764 - score 0.9092 +2021-03-26 08:20:28,229 BAD EPOCHS (no improvement): 0 +2021-03-26 08:20:37,713 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:20:38,743 epoch 27 - iter 2/25 - loss 3.82175541 - samples/sec: 124.54 - lr: 0.250000 +2021-03-26 08:20:39,725 epoch 27 - iter 4/25 - loss 4.01610708 - samples/sec: 130.61 - lr: 0.250000 +2021-03-26 08:20:40,747 epoch 27 - iter 6/25 - loss 3.97728193 - samples/sec: 125.35 - lr: 0.250000 +2021-03-26 08:20:41,683 epoch 27 - iter 8/25 - loss 4.06949121 - samples/sec: 136.92 - lr: 0.250000 +2021-03-26 08:20:42,639 epoch 27 - iter 10/25 - loss 4.05400198 - samples/sec: 134.22 - lr: 0.250000 +2021-03-26 08:20:43,700 epoch 27 - iter 12/25 - loss 3.98850983 - samples/sec: 120.75 - lr: 0.250000 +2021-03-26 08:20:44,594 epoch 27 - iter 14/25 - loss 4.08828683 - samples/sec: 143.38 - lr: 0.250000 +2021-03-26 08:20:45,593 epoch 27 - iter 16/25 - loss 4.22165529 - samples/sec: 128.28 - lr: 0.250000 +2021-03-26 08:20:46,526 epoch 27 - iter 18/25 - loss 4.11700242 - samples/sec: 137.51 - lr: 0.250000 +2021-03-26 08:20:47,568 epoch 27 - iter 20/25 - loss 4.16635238 - samples/sec: 122.95 - lr: 0.250000 +2021-03-26 08:20:48,515 epoch 27 - iter 22/25 - loss 4.14988176 - samples/sec: 135.49 - lr: 0.250000 +2021-03-26 08:20:49,456 epoch 27 - iter 24/25 - loss 4.15121517 - samples/sec: 136.17 - lr: 0.250000 +2021-03-26 08:20:49,927 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:20:49,928 EPOCH 27 done: loss 4.1380 - lr 0.2500000 +2021-03-26 08:20:50,661 DEV : loss 5.364542007446289 - score 0.9118 +2021-03-26 08:20:50,685 BAD EPOCHS (no improvement): 0 +2021-03-26 08:21:00,173 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:21:01,222 epoch 28 - iter 2/25 - loss 3.50919247 - samples/sec: 122.34 - lr: 0.250000 +2021-03-26 08:21:02,106 epoch 28 - iter 4/25 - loss 3.65314603 - samples/sec: 145.06 - lr: 0.250000 +2021-03-26 08:21:03,134 epoch 28 - iter 6/25 - loss 3.99633940 - samples/sec: 124.64 - lr: 0.250000 +2021-03-26 08:21:04,115 epoch 28 - iter 8/25 - loss 3.91186973 - samples/sec: 130.68 - lr: 0.250000 +2021-03-26 08:21:05,034 epoch 28 - iter 10/25 - loss 3.91712847 - samples/sec: 139.47 - lr: 0.250000 +2021-03-26 08:21:06,086 epoch 28 - iter 12/25 - loss 3.98461171 - samples/sec: 121.88 - lr: 0.250000 +2021-03-26 08:21:07,029 epoch 28 - iter 14/25 - loss 3.93786047 - samples/sec: 135.97 - lr: 0.250000 +2021-03-26 08:21:08,164 epoch 28 - iter 16/25 - loss 3.96120444 - samples/sec: 112.95 - lr: 0.250000 +2021-03-26 08:21:09,231 epoch 28 - iter 18/25 - loss 4.01231313 - samples/sec: 120.07 - lr: 0.250000 +2021-03-26 08:21:10,217 epoch 28 - iter 20/25 - loss 4.00550351 - samples/sec: 130.04 - lr: 0.250000 +2021-03-26 08:21:11,216 epoch 28 - iter 22/25 - loss 4.02875296 - samples/sec: 128.34 - lr: 0.250000 +2021-03-26 08:21:12,171 epoch 28 - iter 24/25 - loss 4.04454490 - samples/sec: 134.28 - lr: 0.250000 +2021-03-26 08:21:12,550 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:21:12,551 EPOCH 28 done: loss 4.0430 - lr 0.2500000 +2021-03-26 08:21:13,276 DEV : loss 5.4191999435424805 - score 0.9057 +2021-03-26 08:21:13,293 BAD EPOCHS (no improvement): 1 +2021-03-26 08:21:13,293 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:21:14,274 epoch 29 - iter 2/25 - loss 3.11907506 - samples/sec: 130.75 - lr: 0.250000 +2021-03-26 08:21:15,264 epoch 29 - iter 4/25 - loss 3.22725344 - samples/sec: 129.39 - lr: 0.250000 +2021-03-26 08:21:16,193 epoch 29 - iter 6/25 - loss 3.43090073 - samples/sec: 137.98 - lr: 0.250000 +2021-03-26 08:21:17,157 epoch 29 - iter 8/25 - loss 3.64438808 - samples/sec: 132.98 - lr: 0.250000 +2021-03-26 08:21:18,103 epoch 29 - iter 10/25 - loss 3.61872423 - samples/sec: 135.63 - lr: 0.250000 +2021-03-26 08:21:19,180 epoch 29 - iter 12/25 - loss 3.71516027 - samples/sec: 119.04 - lr: 0.250000 +2021-03-26 08:21:20,179 epoch 29 - iter 14/25 - loss 3.74714305 - samples/sec: 128.19 - lr: 0.250000 +2021-03-26 08:21:21,090 epoch 29 - iter 16/25 - loss 3.79307090 - samples/sec: 140.77 - lr: 0.250000 +2021-03-26 08:21:22,053 epoch 29 - iter 18/25 - loss 3.75355324 - samples/sec: 132.99 - lr: 0.250000 +2021-03-26 08:21:23,062 epoch 29 - iter 20/25 - loss 3.80725433 - samples/sec: 127.15 - lr: 0.250000 +2021-03-26 08:21:24,146 epoch 29 - iter 22/25 - loss 3.83380900 - samples/sec: 118.29 - lr: 0.250000 +2021-03-26 08:21:25,193 epoch 29 - iter 24/25 - loss 3.79775758 - samples/sec: 122.47 - lr: 0.250000 +2021-03-26 08:21:25,636 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:21:25,637 EPOCH 29 done: loss 3.8274 - lr 0.2500000 +2021-03-26 08:21:26,376 DEV : loss 5.257043838500977 - score 0.9126 +2021-03-26 08:21:26,393 BAD EPOCHS (no improvement): 0 +2021-03-26 08:21:36,002 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:21:37,083 epoch 30 - iter 2/25 - loss 3.26837575 - samples/sec: 118.73 - lr: 0.250000 +2021-03-26 08:21:38,126 epoch 30 - iter 4/25 - loss 3.55052143 - samples/sec: 122.93 - lr: 0.250000 +2021-03-26 08:21:39,218 epoch 30 - iter 6/25 - loss 3.51545898 - samples/sec: 117.34 - lr: 0.250000 +2021-03-26 08:21:40,228 epoch 30 - iter 8/25 - loss 3.52578855 - samples/sec: 127.02 - lr: 0.250000 +2021-03-26 08:21:41,116 epoch 30 - iter 10/25 - loss 3.35951045 - samples/sec: 144.52 - lr: 0.250000 +2021-03-26 08:21:42,067 epoch 30 - iter 12/25 - loss 3.37255822 - samples/sec: 134.75 - lr: 0.250000 +2021-03-26 08:21:43,062 epoch 30 - iter 14/25 - loss 3.49844871 - samples/sec: 128.91 - lr: 0.250000 +2021-03-26 08:21:43,959 epoch 30 - iter 16/25 - loss 3.42460684 - samples/sec: 143.03 - lr: 0.250000 +2021-03-26 08:21:44,987 epoch 30 - iter 18/25 - loss 3.54060519 - samples/sec: 124.75 - lr: 0.250000 +2021-03-26 08:21:45,989 epoch 30 - iter 20/25 - loss 3.58279957 - samples/sec: 128.44 - lr: 0.250000 +2021-03-26 08:21:46,909 epoch 30 - iter 22/25 - loss 3.62058116 - samples/sec: 139.40 - lr: 0.250000 +2021-03-26 08:21:47,921 epoch 30 - iter 24/25 - loss 3.57932650 - samples/sec: 126.62 - lr: 0.250000 +2021-03-26 08:21:48,333 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:21:48,334 EPOCH 30 done: loss 3.6379 - lr 0.2500000 +2021-03-26 08:21:49,048 DEV : loss 5.350982189178467 - score 0.9067 +2021-03-26 08:21:49,071 BAD EPOCHS (no improvement): 1 +2021-03-26 08:21:49,071 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:21:50,133 epoch 31 - iter 2/25 - loss 3.56340146 - samples/sec: 120.82 - lr: 0.250000 +2021-03-26 08:21:51,178 epoch 31 - iter 4/25 - loss 3.83946228 - samples/sec: 122.65 - lr: 0.250000 +2021-03-26 08:21:52,149 epoch 31 - iter 6/25 - loss 3.70126661 - samples/sec: 132.12 - lr: 0.250000 +2021-03-26 08:21:53,119 epoch 31 - iter 8/25 - loss 3.72824273 - samples/sec: 132.22 - lr: 0.250000 +2021-03-26 08:21:54,120 epoch 31 - iter 10/25 - loss 3.52071278 - samples/sec: 128.09 - lr: 0.250000 +2021-03-26 08:21:55,105 epoch 31 - iter 12/25 - loss 3.44530173 - samples/sec: 130.22 - lr: 0.250000 +2021-03-26 08:21:56,140 epoch 31 - iter 14/25 - loss 3.47722861 - samples/sec: 123.90 - lr: 0.250000 +2021-03-26 08:21:57,066 epoch 31 - iter 16/25 - loss 3.49996176 - samples/sec: 138.48 - lr: 0.250000 +2021-03-26 08:21:57,977 epoch 31 - iter 18/25 - loss 3.49459041 - samples/sec: 140.59 - lr: 0.250000 +2021-03-26 08:21:58,985 epoch 31 - iter 20/25 - loss 3.48911209 - samples/sec: 127.26 - lr: 0.250000 +2021-03-26 08:22:00,036 epoch 31 - iter 22/25 - loss 3.47431731 - samples/sec: 122.05 - lr: 0.250000 +2021-03-26 08:22:01,062 epoch 31 - iter 24/25 - loss 3.51283441 - samples/sec: 124.85 - lr: 0.250000 +2021-03-26 08:22:01,395 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:22:01,396 EPOCH 31 done: loss 3.4821 - lr 0.2500000 +2021-03-26 08:22:02,143 DEV : loss 5.382730007171631 - score 0.9099 +2021-03-26 08:22:02,165 BAD EPOCHS (no improvement): 2 +2021-03-26 08:22:02,166 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:22:03,083 epoch 32 - iter 2/25 - loss 2.84548354 - samples/sec: 139.94 - lr: 0.250000 +2021-03-26 08:22:04,010 epoch 32 - iter 4/25 - loss 3.25069106 - samples/sec: 138.21 - lr: 0.250000 +2021-03-26 08:22:05,026 epoch 32 - iter 6/25 - loss 3.52776047 - samples/sec: 126.22 - lr: 0.250000 +2021-03-26 08:22:06,089 epoch 32 - iter 8/25 - loss 3.55593899 - samples/sec: 120.54 - lr: 0.250000 +2021-03-26 08:22:07,090 epoch 32 - iter 10/25 - loss 3.46752748 - samples/sec: 128.07 - lr: 0.250000 +2021-03-26 08:22:08,108 epoch 32 - iter 12/25 - loss 3.50795273 - samples/sec: 125.86 - lr: 0.250000 +2021-03-26 08:22:09,054 epoch 32 - iter 14/25 - loss 3.61622231 - samples/sec: 135.40 - lr: 0.250000 +2021-03-26 08:22:10,010 epoch 32 - iter 16/25 - loss 3.55014120 - samples/sec: 134.27 - lr: 0.250000 +2021-03-26 08:22:11,055 epoch 32 - iter 18/25 - loss 3.56227758 - samples/sec: 122.66 - lr: 0.250000 +2021-03-26 08:22:12,012 epoch 32 - iter 20/25 - loss 3.55966748 - samples/sec: 134.00 - lr: 0.250000 +2021-03-26 08:22:12,924 epoch 32 - iter 22/25 - loss 3.50907019 - samples/sec: 140.67 - lr: 0.250000 +2021-03-26 08:22:13,916 epoch 32 - iter 24/25 - loss 3.52688112 - samples/sec: 129.16 - lr: 0.250000 +2021-03-26 08:22:14,305 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:22:14,306 EPOCH 32 done: loss 3.5538 - lr 0.2500000 +2021-03-26 08:22:15,046 DEV : loss 5.270177841186523 - score 0.9133 +2021-03-26 08:22:15,069 BAD EPOCHS (no improvement): 0 +2021-03-26 08:22:24,675 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:22:25,689 epoch 33 - iter 2/25 - loss 3.15487409 - samples/sec: 126.48 - lr: 0.250000 +2021-03-26 08:22:26,603 epoch 33 - iter 4/25 - loss 2.86552042 - samples/sec: 140.28 - lr: 0.250000 +2021-03-26 08:22:27,589 epoch 33 - iter 6/25 - loss 2.91351219 - samples/sec: 129.92 - lr: 0.250000 +2021-03-26 08:22:28,564 epoch 33 - iter 8/25 - loss 3.18267134 - samples/sec: 131.43 - lr: 0.250000 +2021-03-26 08:22:29,661 epoch 33 - iter 10/25 - loss 3.10160365 - samples/sec: 116.84 - lr: 0.250000 +2021-03-26 08:22:30,697 epoch 33 - iter 12/25 - loss 3.12735937 - samples/sec: 123.76 - lr: 0.250000 +2021-03-26 08:22:31,604 epoch 33 - iter 14/25 - loss 3.19382245 - samples/sec: 141.36 - lr: 0.250000 +2021-03-26 08:22:32,676 epoch 33 - iter 16/25 - loss 3.23439336 - samples/sec: 119.56 - lr: 0.250000 +2021-03-26 08:22:33,697 epoch 33 - iter 18/25 - loss 3.21325998 - samples/sec: 125.67 - lr: 0.250000 +2021-03-26 08:22:34,634 epoch 33 - iter 20/25 - loss 3.26282755 - samples/sec: 136.89 - lr: 0.250000 +2021-03-26 08:22:35,681 epoch 33 - iter 22/25 - loss 3.34196002 - samples/sec: 122.34 - lr: 0.250000 +2021-03-26 08:22:36,630 epoch 33 - iter 24/25 - loss 3.33675192 - samples/sec: 135.72 - lr: 0.250000 +2021-03-26 08:22:37,055 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:22:37,056 EPOCH 33 done: loss 3.3575 - lr 0.2500000 +2021-03-26 08:22:37,766 DEV : loss 5.367270469665527 - score 0.9096 +2021-03-26 08:22:37,788 BAD EPOCHS (no improvement): 1 +2021-03-26 08:22:37,788 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:22:38,731 epoch 34 - iter 2/25 - loss 3.58751345 - samples/sec: 135.99 - lr: 0.250000 +2021-03-26 08:22:39,717 epoch 34 - iter 4/25 - loss 3.53140342 - samples/sec: 129.99 - lr: 0.250000 +2021-03-26 08:22:40,936 epoch 34 - iter 6/25 - loss 3.76132822 - samples/sec: 105.13 - lr: 0.250000 +2021-03-26 08:22:42,158 epoch 34 - iter 8/25 - loss 3.62088448 - samples/sec: 104.94 - lr: 0.250000 +2021-03-26 08:22:43,156 epoch 34 - iter 10/25 - loss 3.64314551 - samples/sec: 128.41 - lr: 0.250000 +2021-03-26 08:22:44,166 epoch 34 - iter 12/25 - loss 3.64726754 - samples/sec: 126.97 - lr: 0.250000 +2021-03-26 08:22:45,122 epoch 34 - iter 14/25 - loss 3.60614933 - samples/sec: 134.05 - lr: 0.250000 +2021-03-26 08:22:46,180 epoch 34 - iter 16/25 - loss 3.61291116 - samples/sec: 121.17 - lr: 0.250000 +2021-03-26 08:22:47,315 epoch 34 - iter 18/25 - loss 3.66392313 - samples/sec: 112.94 - lr: 0.250000 +2021-03-26 08:22:48,344 epoch 34 - iter 20/25 - loss 3.59719365 - samples/sec: 124.61 - lr: 0.250000 +2021-03-26 08:22:49,384 epoch 34 - iter 22/25 - loss 3.57430128 - samples/sec: 123.23 - lr: 0.250000 +2021-03-26 08:22:50,325 epoch 34 - iter 24/25 - loss 3.55224803 - samples/sec: 136.42 - lr: 0.250000 +2021-03-26 08:22:50,787 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:22:50,787 EPOCH 34 done: loss 3.5453 - lr 0.2500000 +2021-03-26 08:22:51,512 DEV : loss 5.184334754943848 - score 0.9107 +2021-03-26 08:22:51,535 BAD EPOCHS (no improvement): 2 +2021-03-26 08:22:51,537 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:22:52,422 epoch 35 - iter 2/25 - loss 3.18492174 - samples/sec: 144.84 - lr: 0.250000 +2021-03-26 08:22:53,467 epoch 35 - iter 4/25 - loss 3.13623476 - samples/sec: 122.76 - lr: 0.250000 +2021-03-26 08:22:54,406 epoch 35 - iter 6/25 - loss 3.33380385 - samples/sec: 136.42 - lr: 0.250000 +2021-03-26 08:22:55,431 epoch 35 - iter 8/25 - loss 3.39252409 - samples/sec: 125.11 - lr: 0.250000 +2021-03-26 08:22:56,487 epoch 35 - iter 10/25 - loss 3.46284273 - samples/sec: 121.42 - lr: 0.250000 +2021-03-26 08:22:57,444 epoch 35 - iter 12/25 - loss 3.38363576 - samples/sec: 133.89 - lr: 0.250000 +2021-03-26 08:22:58,488 epoch 35 - iter 14/25 - loss 3.38682234 - samples/sec: 122.86 - lr: 0.250000 +2021-03-26 08:22:59,420 epoch 35 - iter 16/25 - loss 3.45151280 - samples/sec: 137.47 - lr: 0.250000 +2021-03-26 08:23:00,467 epoch 35 - iter 18/25 - loss 3.41133621 - samples/sec: 122.37 - lr: 0.250000 +2021-03-26 08:23:01,445 epoch 35 - iter 20/25 - loss 3.35318577 - samples/sec: 131.19 - lr: 0.250000 +2021-03-26 08:23:02,375 epoch 35 - iter 22/25 - loss 3.32299045 - samples/sec: 137.74 - lr: 0.250000 +2021-03-26 08:23:03,361 epoch 35 - iter 24/25 - loss 3.32118590 - samples/sec: 130.07 - lr: 0.250000 +2021-03-26 08:23:03,791 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:23:03,792 EPOCH 35 done: loss 3.3499 - lr 0.2500000 +2021-03-26 08:23:04,524 DEV : loss 5.07149600982666 - score 0.9118 +2021-03-26 08:23:04,547 BAD EPOCHS (no improvement): 3 +2021-03-26 08:23:04,548 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:23:05,621 epoch 36 - iter 2/25 - loss 2.85354340 - samples/sec: 119.48 - lr: 0.250000 +2021-03-26 08:23:06,611 epoch 36 - iter 4/25 - loss 2.93653178 - samples/sec: 129.47 - lr: 0.250000 +2021-03-26 08:23:07,630 epoch 36 - iter 6/25 - loss 2.91629008 - samples/sec: 125.87 - lr: 0.250000 +2021-03-26 08:23:08,692 epoch 36 - iter 8/25 - loss 3.09478921 - samples/sec: 120.69 - lr: 0.250000 +2021-03-26 08:23:09,798 epoch 36 - iter 10/25 - loss 3.04699903 - samples/sec: 115.81 - lr: 0.250000 +2021-03-26 08:23:10,807 epoch 36 - iter 12/25 - loss 3.05647022 - samples/sec: 127.10 - lr: 0.250000 +2021-03-26 08:23:11,789 epoch 36 - iter 14/25 - loss 2.99877063 - samples/sec: 130.50 - lr: 0.250000 +2021-03-26 08:23:12,815 epoch 36 - iter 16/25 - loss 3.09249909 - samples/sec: 124.87 - lr: 0.250000 +2021-03-26 08:23:13,775 epoch 36 - iter 18/25 - loss 3.12907799 - samples/sec: 133.47 - lr: 0.250000 +2021-03-26 08:23:14,788 epoch 36 - iter 20/25 - loss 3.14141412 - samples/sec: 126.61 - lr: 0.250000 +2021-03-26 08:23:15,757 epoch 36 - iter 22/25 - loss 3.18839633 - samples/sec: 132.35 - lr: 0.250000 +2021-03-26 08:23:16,740 epoch 36 - iter 24/25 - loss 3.19035632 - samples/sec: 130.44 - lr: 0.250000 +2021-03-26 08:23:17,177 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:23:17,178 EPOCH 36 done: loss 3.1967 - lr 0.2500000 +2021-03-26 08:23:17,868 DEV : loss 5.2465972900390625 - score 0.9126 +2021-03-26 08:23:17,888 BAD EPOCHS (no improvement): 4 +2021-03-26 08:23:17,889 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:23:18,843 epoch 37 - iter 2/25 - loss 2.63817966 - samples/sec: 134.29 - lr: 0.125000 +2021-03-26 08:23:19,830 epoch 37 - iter 4/25 - loss 3.05239159 - samples/sec: 129.86 - lr: 0.125000 +2021-03-26 08:23:20,858 epoch 37 - iter 6/25 - loss 3.11489769 - samples/sec: 124.70 - lr: 0.125000 +2021-03-26 08:23:21,785 epoch 37 - iter 8/25 - loss 3.18249029 - samples/sec: 138.38 - lr: 0.125000 +2021-03-26 08:23:22,760 epoch 37 - iter 10/25 - loss 3.03278530 - samples/sec: 131.41 - lr: 0.125000 +2021-03-26 08:23:23,670 epoch 37 - iter 12/25 - loss 3.05815605 - samples/sec: 140.94 - lr: 0.125000 +2021-03-26 08:23:24,612 epoch 37 - iter 14/25 - loss 3.04020321 - samples/sec: 136.11 - lr: 0.125000 +2021-03-26 08:23:25,562 epoch 37 - iter 16/25 - loss 3.10066675 - samples/sec: 134.96 - lr: 0.125000 +2021-03-26 08:23:26,612 epoch 37 - iter 18/25 - loss 3.06573884 - samples/sec: 122.02 - lr: 0.125000 +2021-03-26 08:23:27,624 epoch 37 - iter 20/25 - loss 3.08411411 - samples/sec: 126.79 - lr: 0.125000 +2021-03-26 08:23:28,687 epoch 37 - iter 22/25 - loss 3.08923004 - samples/sec: 120.68 - lr: 0.125000 +2021-03-26 08:23:29,669 epoch 37 - iter 24/25 - loss 3.10842060 - samples/sec: 130.51 - lr: 0.125000 +2021-03-26 08:23:30,109 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:23:30,110 EPOCH 37 done: loss 3.0953 - lr 0.1250000 +2021-03-26 08:23:30,839 DEV : loss 5.055671691894531 - score 0.916 +2021-03-26 08:23:30,862 BAD EPOCHS (no improvement): 0 +2021-03-26 08:23:40,200 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:23:41,278 epoch 38 - iter 2/25 - loss 2.59178650 - samples/sec: 118.89 - lr: 0.125000 +2021-03-26 08:23:42,280 epoch 38 - iter 4/25 - loss 2.56540596 - samples/sec: 128.03 - lr: 0.125000 +2021-03-26 08:23:43,320 epoch 38 - iter 6/25 - loss 2.72324753 - samples/sec: 123.14 - lr: 0.125000 +2021-03-26 08:23:44,333 epoch 38 - iter 8/25 - loss 2.74542779 - samples/sec: 126.62 - lr: 0.125000 +2021-03-26 08:23:45,362 epoch 38 - iter 10/25 - loss 2.80854850 - samples/sec: 124.49 - lr: 0.125000 +2021-03-26 08:23:46,316 epoch 38 - iter 12/25 - loss 2.78986889 - samples/sec: 134.39 - lr: 0.125000 +2021-03-26 08:23:47,337 epoch 38 - iter 14/25 - loss 2.87522815 - samples/sec: 125.58 - lr: 0.125000 +2021-03-26 08:23:48,349 epoch 38 - iter 16/25 - loss 2.80133051 - samples/sec: 126.72 - lr: 0.125000 +2021-03-26 08:23:49,440 epoch 38 - iter 18/25 - loss 2.84491060 - samples/sec: 117.52 - lr: 0.125000 +2021-03-26 08:23:50,451 epoch 38 - iter 20/25 - loss 2.85927484 - samples/sec: 126.83 - lr: 0.125000 +2021-03-26 08:23:51,373 epoch 38 - iter 22/25 - loss 2.87918811 - samples/sec: 139.14 - lr: 0.125000 +2021-03-26 08:23:52,329 epoch 38 - iter 24/25 - loss 2.89003314 - samples/sec: 134.05 - lr: 0.125000 +2021-03-26 08:23:52,707 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:23:52,708 EPOCH 38 done: loss 2.9956 - lr 0.1250000 +2021-03-26 08:23:53,446 DEV : loss 4.992068290710449 - score 0.9194 +2021-03-26 08:23:53,469 BAD EPOCHS (no improvement): 0 +2021-03-26 08:24:03,131 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:24:04,213 epoch 39 - iter 2/25 - loss 3.26761401 - samples/sec: 118.64 - lr: 0.125000 +2021-03-26 08:24:05,228 epoch 39 - iter 4/25 - loss 3.34832674 - samples/sec: 126.24 - lr: 0.125000 +2021-03-26 08:24:06,237 epoch 39 - iter 6/25 - loss 3.22142450 - samples/sec: 127.10 - lr: 0.125000 +2021-03-26 08:24:07,128 epoch 39 - iter 8/25 - loss 3.15339360 - samples/sec: 143.79 - lr: 0.125000 +2021-03-26 08:24:07,991 epoch 39 - iter 10/25 - loss 3.05391240 - samples/sec: 148.65 - lr: 0.125000 +2021-03-26 08:24:08,889 epoch 39 - iter 12/25 - loss 3.06373362 - samples/sec: 142.63 - lr: 0.125000 +2021-03-26 08:24:09,803 epoch 39 - iter 14/25 - loss 2.98908319 - samples/sec: 140.15 - lr: 0.125000 +2021-03-26 08:24:10,765 epoch 39 - iter 16/25 - loss 2.94973241 - samples/sec: 133.37 - lr: 0.125000 +2021-03-26 08:24:11,716 epoch 39 - iter 18/25 - loss 2.95112207 - samples/sec: 134.86 - lr: 0.125000 +2021-03-26 08:24:12,764 epoch 39 - iter 20/25 - loss 2.91271211 - samples/sec: 122.28 - lr: 0.125000 +2021-03-26 08:24:13,775 epoch 39 - iter 22/25 - loss 2.89700104 - samples/sec: 126.79 - lr: 0.125000 +2021-03-26 08:24:14,756 epoch 39 - iter 24/25 - loss 2.84286279 - samples/sec: 130.68 - lr: 0.125000 +2021-03-26 08:24:15,185 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:24:15,186 EPOCH 39 done: loss 2.8496 - lr 0.1250000 +2021-03-26 08:24:15,932 DEV : loss 5.133090972900391 - score 0.9143 +2021-03-26 08:24:15,969 BAD EPOCHS (no improvement): 1 +2021-03-26 08:24:15,970 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:24:16,996 epoch 40 - iter 2/25 - loss 3.13895917 - samples/sec: 124.91 - lr: 0.125000 +2021-03-26 08:24:17,947 epoch 40 - iter 4/25 - loss 2.89032578 - samples/sec: 134.86 - lr: 0.125000 +2021-03-26 08:24:18,982 epoch 40 - iter 6/25 - loss 2.81307308 - samples/sec: 123.74 - lr: 0.125000 +2021-03-26 08:24:19,989 epoch 40 - iter 8/25 - loss 2.90850329 - samples/sec: 127.39 - lr: 0.125000 +2021-03-26 08:24:21,007 epoch 40 - iter 10/25 - loss 2.93947659 - samples/sec: 125.86 - lr: 0.125000 +2021-03-26 08:24:21,987 epoch 40 - iter 12/25 - loss 2.84735465 - samples/sec: 130.83 - lr: 0.125000 +2021-03-26 08:24:23,164 epoch 40 - iter 14/25 - loss 2.84274534 - samples/sec: 108.83 - lr: 0.125000 +2021-03-26 08:24:24,275 epoch 40 - iter 16/25 - loss 2.84609564 - samples/sec: 115.32 - lr: 0.125000 +2021-03-26 08:24:25,291 epoch 40 - iter 18/25 - loss 2.87500971 - samples/sec: 126.18 - lr: 0.125000 +2021-03-26 08:24:26,308 epoch 40 - iter 20/25 - loss 2.83314930 - samples/sec: 125.95 - lr: 0.125000 +2021-03-26 08:24:27,419 epoch 40 - iter 22/25 - loss 2.86485132 - samples/sec: 115.38 - lr: 0.125000 +2021-03-26 08:24:28,389 epoch 40 - iter 24/25 - loss 2.89132403 - samples/sec: 132.13 - lr: 0.125000 +2021-03-26 08:24:28,789 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:24:28,789 EPOCH 40 done: loss 2.9279 - lr 0.1250000 +2021-03-26 08:24:29,536 DEV : loss 5.163059234619141 - score 0.916 +2021-03-26 08:24:29,559 BAD EPOCHS (no improvement): 2 +2021-03-26 08:24:29,560 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:24:30,520 epoch 41 - iter 2/25 - loss 3.34717524 - samples/sec: 133.59 - lr: 0.125000 +2021-03-26 08:24:31,558 epoch 41 - iter 4/25 - loss 3.11228681 - samples/sec: 123.52 - lr: 0.125000 +2021-03-26 08:24:32,512 epoch 41 - iter 6/25 - loss 2.98143824 - samples/sec: 134.40 - lr: 0.125000 +2021-03-26 08:24:33,467 epoch 41 - iter 8/25 - loss 3.10384235 - samples/sec: 134.12 - lr: 0.125000 +2021-03-26 08:24:34,426 epoch 41 - iter 10/25 - loss 2.93525398 - samples/sec: 133.85 - lr: 0.125000 +2021-03-26 08:24:35,417 epoch 41 - iter 12/25 - loss 2.92955963 - samples/sec: 129.33 - lr: 0.125000 +2021-03-26 08:24:36,439 epoch 41 - iter 14/25 - loss 2.94683690 - samples/sec: 125.41 - lr: 0.125000 +2021-03-26 08:24:37,464 epoch 41 - iter 16/25 - loss 2.97896834 - samples/sec: 125.13 - lr: 0.125000 +2021-03-26 08:24:38,461 epoch 41 - iter 18/25 - loss 2.97161441 - samples/sec: 128.58 - lr: 0.125000 +2021-03-26 08:24:39,423 epoch 41 - iter 20/25 - loss 2.96228577 - samples/sec: 133.37 - lr: 0.125000 +2021-03-26 08:24:40,368 epoch 41 - iter 22/25 - loss 2.91309004 - samples/sec: 135.67 - lr: 0.125000 +2021-03-26 08:24:41,327 epoch 41 - iter 24/25 - loss 2.89624717 - samples/sec: 133.65 - lr: 0.125000 +2021-03-26 08:24:41,760 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:24:41,761 EPOCH 41 done: loss 2.9257 - lr 0.1250000 +2021-03-26 08:24:42,479 DEV : loss 5.18552303314209 - score 0.9135 +2021-03-26 08:24:42,494 BAD EPOCHS (no improvement): 3 +2021-03-26 08:24:42,495 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:24:43,540 epoch 42 - iter 2/25 - loss 3.23972845 - samples/sec: 122.69 - lr: 0.125000 +2021-03-26 08:24:44,629 epoch 42 - iter 4/25 - loss 3.31910771 - samples/sec: 117.69 - lr: 0.125000 +2021-03-26 08:24:45,575 epoch 42 - iter 6/25 - loss 3.12779403 - samples/sec: 135.67 - lr: 0.125000 +2021-03-26 08:24:46,580 epoch 42 - iter 8/25 - loss 3.03878088 - samples/sec: 127.50 - lr: 0.125000 +2021-03-26 08:24:47,585 epoch 42 - iter 10/25 - loss 2.97357968 - samples/sec: 127.57 - lr: 0.125000 +2021-03-26 08:24:48,577 epoch 42 - iter 12/25 - loss 2.94139670 - samples/sec: 129.32 - lr: 0.125000 +2021-03-26 08:24:49,464 epoch 42 - iter 14/25 - loss 2.87050148 - samples/sec: 144.50 - lr: 0.125000 +2021-03-26 08:24:50,514 epoch 42 - iter 16/25 - loss 2.81032179 - samples/sec: 122.15 - lr: 0.125000 +2021-03-26 08:24:51,558 epoch 42 - iter 18/25 - loss 2.79874009 - samples/sec: 122.72 - lr: 0.125000 +2021-03-26 08:24:52,550 epoch 42 - iter 20/25 - loss 2.83001460 - samples/sec: 129.39 - lr: 0.125000 +2021-03-26 08:24:53,508 epoch 42 - iter 22/25 - loss 2.81053819 - samples/sec: 133.73 - lr: 0.125000 +2021-03-26 08:24:54,510 epoch 42 - iter 24/25 - loss 2.81009710 - samples/sec: 128.03 - lr: 0.125000 +2021-03-26 08:24:54,943 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:24:54,944 EPOCH 42 done: loss 2.8110 - lr 0.1250000 +2021-03-26 08:24:55,686 DEV : loss 5.17966365814209 - score 0.9143 +2021-03-26 08:24:55,704 BAD EPOCHS (no improvement): 4 +2021-03-26 08:24:55,705 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:24:56,788 epoch 43 - iter 2/25 - loss 2.25671196 - samples/sec: 118.30 - lr: 0.062500 +2021-03-26 08:24:57,782 epoch 43 - iter 4/25 - loss 2.48410356 - samples/sec: 129.00 - lr: 0.062500 +2021-03-26 08:24:58,870 epoch 43 - iter 6/25 - loss 2.70733134 - samples/sec: 117.79 - lr: 0.062500 +2021-03-26 08:24:59,818 epoch 43 - iter 8/25 - loss 2.80536109 - samples/sec: 135.18 - lr: 0.062500 +2021-03-26 08:25:00,859 epoch 43 - iter 10/25 - loss 2.89832189 - samples/sec: 123.16 - lr: 0.062500 +2021-03-26 08:25:01,864 epoch 43 - iter 12/25 - loss 2.77370048 - samples/sec: 127.56 - lr: 0.062500 +2021-03-26 08:25:02,706 epoch 43 - iter 14/25 - loss 2.72336340 - samples/sec: 152.15 - lr: 0.062500 +2021-03-26 08:25:03,721 epoch 43 - iter 16/25 - loss 2.76130933 - samples/sec: 126.35 - lr: 0.062500 +2021-03-26 08:25:04,708 epoch 43 - iter 18/25 - loss 2.74378959 - samples/sec: 129.93 - lr: 0.062500 +2021-03-26 08:25:05,670 epoch 43 - iter 20/25 - loss 2.73068177 - samples/sec: 133.23 - lr: 0.062500 +2021-03-26 08:25:06,588 epoch 43 - iter 22/25 - loss 2.75959666 - samples/sec: 139.61 - lr: 0.062500 +2021-03-26 08:25:07,534 epoch 43 - iter 24/25 - loss 2.79225791 - samples/sec: 135.51 - lr: 0.062500 +2021-03-26 08:25:07,958 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:25:07,959 EPOCH 43 done: loss 2.7573 - lr 0.0625000 +2021-03-26 08:25:08,683 DEV : loss 5.191972255706787 - score 0.9143 +2021-03-26 08:25:08,697 BAD EPOCHS (no improvement): 1 +2021-03-26 08:25:08,698 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:25:09,707 epoch 44 - iter 2/25 - loss 2.51410568 - samples/sec: 127.08 - lr: 0.062500 +2021-03-26 08:25:10,828 epoch 44 - iter 4/25 - loss 2.87437314 - samples/sec: 114.42 - lr: 0.062500 +2021-03-26 08:25:11,853 epoch 44 - iter 6/25 - loss 2.97266304 - samples/sec: 125.03 - lr: 0.062500 +2021-03-26 08:25:12,832 epoch 44 - iter 8/25 - loss 2.98653641 - samples/sec: 131.11 - lr: 0.062500 +2021-03-26 08:25:13,815 epoch 44 - iter 10/25 - loss 2.91581733 - samples/sec: 130.50 - lr: 0.062500 +2021-03-26 08:25:14,819 epoch 44 - iter 12/25 - loss 2.88142518 - samples/sec: 127.70 - lr: 0.062500 +2021-03-26 08:25:15,785 epoch 44 - iter 14/25 - loss 2.80705702 - samples/sec: 132.71 - lr: 0.062500 +2021-03-26 08:25:16,714 epoch 44 - iter 16/25 - loss 2.71279316 - samples/sec: 137.94 - lr: 0.062500 +2021-03-26 08:25:17,711 epoch 44 - iter 18/25 - loss 2.69600504 - samples/sec: 128.74 - lr: 0.062500 +2021-03-26 08:25:18,938 epoch 44 - iter 20/25 - loss 2.71131549 - samples/sec: 104.38 - lr: 0.062500 +2021-03-26 08:25:20,177 epoch 44 - iter 22/25 - loss 2.72818206 - samples/sec: 103.46 - lr: 0.062500 +2021-03-26 08:25:21,206 epoch 44 - iter 24/25 - loss 2.70353389 - samples/sec: 124.55 - lr: 0.062500 +2021-03-26 08:25:21,749 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:25:21,750 EPOCH 44 done: loss 2.6863 - lr 0.0625000 +2021-03-26 08:25:22,577 DEV : loss 5.165666103363037 - score 0.916 +2021-03-26 08:25:22,607 BAD EPOCHS (no improvement): 2 +2021-03-26 08:25:22,608 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:25:24,008 epoch 45 - iter 2/25 - loss 2.86002135 - samples/sec: 91.55 - lr: 0.062500 +2021-03-26 08:25:25,184 epoch 45 - iter 4/25 - loss 2.98223376 - samples/sec: 109.01 - lr: 0.062500 +2021-03-26 08:25:26,187 epoch 45 - iter 6/25 - loss 2.88732882 - samples/sec: 127.78 - lr: 0.062500 +2021-03-26 08:25:27,263 epoch 45 - iter 8/25 - loss 2.85796985 - samples/sec: 119.14 - lr: 0.062500 +2021-03-26 08:25:28,165 epoch 45 - iter 10/25 - loss 2.75072234 - samples/sec: 142.08 - lr: 0.062500 +2021-03-26 08:25:29,082 epoch 45 - iter 12/25 - loss 2.71562586 - samples/sec: 139.99 - lr: 0.062500 +2021-03-26 08:25:30,055 epoch 45 - iter 14/25 - loss 2.65661091 - samples/sec: 131.71 - lr: 0.062500 +2021-03-26 08:25:31,024 epoch 45 - iter 16/25 - loss 2.68994868 - samples/sec: 132.37 - lr: 0.062500 +2021-03-26 08:25:31,946 epoch 45 - iter 18/25 - loss 2.70727919 - samples/sec: 138.91 - lr: 0.062500 +2021-03-26 08:25:33,055 epoch 45 - iter 20/25 - loss 2.72820472 - samples/sec: 115.57 - lr: 0.062500 +2021-03-26 08:25:34,015 epoch 45 - iter 22/25 - loss 2.69422464 - samples/sec: 133.64 - lr: 0.062500 +2021-03-26 08:25:34,973 epoch 45 - iter 24/25 - loss 2.64335350 - samples/sec: 133.73 - lr: 0.062500 +2021-03-26 08:25:35,351 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:25:35,352 EPOCH 45 done: loss 2.7020 - lr 0.0625000 +2021-03-26 08:25:36,079 DEV : loss 5.152807235717773 - score 0.9135 +2021-03-26 08:25:36,094 BAD EPOCHS (no improvement): 3 +2021-03-26 08:25:36,095 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:25:37,133 epoch 46 - iter 2/25 - loss 3.18182039 - samples/sec: 123.48 - lr: 0.062500 +2021-03-26 08:25:38,117 epoch 46 - iter 4/25 - loss 2.93405211 - samples/sec: 130.35 - lr: 0.062500 +2021-03-26 08:25:39,285 epoch 46 - iter 6/25 - loss 2.85977014 - samples/sec: 109.66 - lr: 0.062500 +2021-03-26 08:25:40,307 epoch 46 - iter 8/25 - loss 2.68965745 - samples/sec: 125.43 - lr: 0.062500 +2021-03-26 08:25:41,269 epoch 46 - iter 10/25 - loss 2.54557719 - samples/sec: 133.24 - lr: 0.062500 +2021-03-26 08:25:42,284 epoch 46 - iter 12/25 - loss 2.54885741 - samples/sec: 126.37 - lr: 0.062500 +2021-03-26 08:25:43,231 epoch 46 - iter 14/25 - loss 2.59709317 - samples/sec: 135.37 - lr: 0.062500 +2021-03-26 08:25:44,319 epoch 46 - iter 16/25 - loss 2.56221853 - samples/sec: 117.78 - lr: 0.062500 +2021-03-26 08:25:45,403 epoch 46 - iter 18/25 - loss 2.59396293 - samples/sec: 118.31 - lr: 0.062500 +2021-03-26 08:25:46,429 epoch 46 - iter 20/25 - loss 2.58809073 - samples/sec: 124.90 - lr: 0.062500 +2021-03-26 08:25:47,507 epoch 46 - iter 22/25 - loss 2.63570851 - samples/sec: 118.93 - lr: 0.062500 +2021-03-26 08:25:48,494 epoch 46 - iter 24/25 - loss 2.62493981 - samples/sec: 129.93 - lr: 0.062500 +2021-03-26 08:25:48,846 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:25:48,847 EPOCH 46 done: loss 2.6255 - lr 0.0625000 +2021-03-26 08:25:49,580 DEV : loss 5.130050182342529 - score 0.9175 +2021-03-26 08:25:49,595 BAD EPOCHS (no improvement): 4 +2021-03-26 08:25:49,595 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:25:50,572 epoch 47 - iter 2/25 - loss 2.34120893 - samples/sec: 131.29 - lr: 0.031250 +2021-03-26 08:25:51,518 epoch 47 - iter 4/25 - loss 2.56537151 - samples/sec: 135.59 - lr: 0.031250 +2021-03-26 08:25:52,459 epoch 47 - iter 6/25 - loss 2.50796159 - samples/sec: 136.28 - lr: 0.031250 +2021-03-26 08:25:53,484 epoch 47 - iter 8/25 - loss 2.43816668 - samples/sec: 124.99 - lr: 0.031250 +2021-03-26 08:25:54,419 epoch 47 - iter 10/25 - loss 2.44248054 - samples/sec: 137.16 - lr: 0.031250 +2021-03-26 08:25:55,379 epoch 47 - iter 12/25 - loss 2.43599850 - samples/sec: 133.59 - lr: 0.031250 +2021-03-26 08:25:56,413 epoch 47 - iter 14/25 - loss 2.45250283 - samples/sec: 123.96 - lr: 0.031250 +2021-03-26 08:25:57,389 epoch 47 - iter 16/25 - loss 2.49436748 - samples/sec: 131.28 - lr: 0.031250 +2021-03-26 08:25:58,493 epoch 47 - iter 18/25 - loss 2.55170678 - samples/sec: 116.04 - lr: 0.031250 +2021-03-26 08:25:59,541 epoch 47 - iter 20/25 - loss 2.54269922 - samples/sec: 122.32 - lr: 0.031250 +2021-03-26 08:26:00,420 epoch 47 - iter 22/25 - loss 2.54198781 - samples/sec: 145.87 - lr: 0.031250 +2021-03-26 08:26:01,372 epoch 47 - iter 24/25 - loss 2.59515851 - samples/sec: 134.72 - lr: 0.031250 +2021-03-26 08:26:01,809 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:26:01,809 EPOCH 47 done: loss 2.5737 - lr 0.0312500 +2021-03-26 08:26:02,527 DEV : loss 5.120047569274902 - score 0.9166 +2021-03-26 08:26:02,550 BAD EPOCHS (no improvement): 1 +2021-03-26 08:26:02,551 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:26:03,525 epoch 48 - iter 2/25 - loss 2.52614760 - samples/sec: 131.64 - lr: 0.031250 +2021-03-26 08:26:04,490 epoch 48 - iter 4/25 - loss 2.53924924 - samples/sec: 133.02 - lr: 0.031250 +2021-03-26 08:26:05,527 epoch 48 - iter 6/25 - loss 2.57990440 - samples/sec: 123.51 - lr: 0.031250 +2021-03-26 08:26:06,504 epoch 48 - iter 8/25 - loss 2.52673838 - samples/sec: 131.27 - lr: 0.031250 +2021-03-26 08:26:07,600 epoch 48 - iter 10/25 - loss 2.44827111 - samples/sec: 116.95 - lr: 0.031250 +2021-03-26 08:26:08,585 epoch 48 - iter 12/25 - loss 2.40130955 - samples/sec: 130.11 - lr: 0.031250 +2021-03-26 08:26:09,552 epoch 48 - iter 14/25 - loss 2.42768495 - samples/sec: 132.66 - lr: 0.031250 +2021-03-26 08:26:10,464 epoch 48 - iter 16/25 - loss 2.40806086 - samples/sec: 140.60 - lr: 0.031250 +2021-03-26 08:26:11,491 epoch 48 - iter 18/25 - loss 2.41822616 - samples/sec: 124.78 - lr: 0.031250 +2021-03-26 08:26:12,395 epoch 48 - iter 20/25 - loss 2.46524236 - samples/sec: 141.84 - lr: 0.031250 +2021-03-26 08:26:13,342 epoch 48 - iter 22/25 - loss 2.53471144 - samples/sec: 135.26 - lr: 0.031250 +2021-03-26 08:26:14,413 epoch 48 - iter 24/25 - loss 2.56991556 - samples/sec: 119.74 - lr: 0.031250 +2021-03-26 08:26:14,822 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:26:14,823 EPOCH 48 done: loss 2.5892 - lr 0.0312500 +2021-03-26 08:26:15,515 DEV : loss 5.128034591674805 - score 0.9158 +2021-03-26 08:26:15,537 BAD EPOCHS (no improvement): 2 +2021-03-26 08:26:15,538 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:26:16,442 epoch 49 - iter 2/25 - loss 2.41402364 - samples/sec: 141.76 - lr: 0.031250 +2021-03-26 08:26:17,409 epoch 49 - iter 4/25 - loss 2.56433320 - samples/sec: 132.75 - lr: 0.031250 +2021-03-26 08:26:18,302 epoch 49 - iter 6/25 - loss 2.50378040 - samples/sec: 143.54 - lr: 0.031250 +2021-03-26 08:26:19,275 epoch 49 - iter 8/25 - loss 2.42913982 - samples/sec: 131.70 - lr: 0.031250 +2021-03-26 08:26:20,310 epoch 49 - iter 10/25 - loss 2.62541809 - samples/sec: 123.92 - lr: 0.031250 +2021-03-26 08:26:21,371 epoch 49 - iter 12/25 - loss 2.64117569 - samples/sec: 120.87 - lr: 0.031250 +2021-03-26 08:26:22,456 epoch 49 - iter 14/25 - loss 2.62500211 - samples/sec: 118.20 - lr: 0.031250 +2021-03-26 08:26:23,494 epoch 49 - iter 16/25 - loss 2.64282882 - samples/sec: 123.41 - lr: 0.031250 +2021-03-26 08:26:24,438 epoch 49 - iter 18/25 - loss 2.60936556 - samples/sec: 135.90 - lr: 0.031250 +2021-03-26 08:26:25,389 epoch 49 - iter 20/25 - loss 2.64132748 - samples/sec: 134.74 - lr: 0.031250 +2021-03-26 08:26:26,456 epoch 49 - iter 22/25 - loss 2.62381291 - samples/sec: 120.09 - lr: 0.031250 +2021-03-26 08:26:27,445 epoch 49 - iter 24/25 - loss 2.61296557 - samples/sec: 129.65 - lr: 0.031250 +2021-03-26 08:26:27,895 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:26:27,895 EPOCH 49 done: loss 2.5931 - lr 0.0312500 +2021-03-26 08:26:28,643 DEV : loss 5.125513076782227 - score 0.9158 +2021-03-26 08:26:28,666 BAD EPOCHS (no improvement): 3 +2021-03-26 08:26:28,667 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:26:29,652 epoch 50 - iter 2/25 - loss 2.38679540 - samples/sec: 130.08 - lr: 0.031250 +2021-03-26 08:26:30,732 epoch 50 - iter 4/25 - loss 2.58361644 - samples/sec: 118.71 - lr: 0.031250 +2021-03-26 08:26:31,603 epoch 50 - iter 6/25 - loss 2.50680178 - samples/sec: 147.16 - lr: 0.031250 +2021-03-26 08:26:32,510 epoch 50 - iter 8/25 - loss 2.59342428 - samples/sec: 141.43 - lr: 0.031250 +2021-03-26 08:26:33,516 epoch 50 - iter 10/25 - loss 2.53629299 - samples/sec: 127.36 - lr: 0.031250 +2021-03-26 08:26:34,560 epoch 50 - iter 12/25 - loss 2.62270388 - samples/sec: 122.94 - lr: 0.031250 +2021-03-26 08:26:35,685 epoch 50 - iter 14/25 - loss 2.70660563 - samples/sec: 113.94 - lr: 0.031250 +2021-03-26 08:26:36,688 epoch 50 - iter 16/25 - loss 2.69249507 - samples/sec: 127.75 - lr: 0.031250 +2021-03-26 08:26:37,739 epoch 50 - iter 18/25 - loss 2.66604349 - samples/sec: 121.97 - lr: 0.031250 +2021-03-26 08:26:38,816 epoch 50 - iter 20/25 - loss 2.68338645 - samples/sec: 119.00 - lr: 0.031250 +2021-03-26 08:26:39,829 epoch 50 - iter 22/25 - loss 2.65240399 - samples/sec: 126.52 - lr: 0.031250 +2021-03-26 08:26:40,821 epoch 50 - iter 24/25 - loss 2.65005972 - samples/sec: 129.30 - lr: 0.031250 +2021-03-26 08:26:41,250 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:26:41,251 EPOCH 50 done: loss 2.7365 - lr 0.0312500 +2021-03-26 08:26:41,962 DEV : loss 5.108526706695557 - score 0.9192 +2021-03-26 08:26:41,985 BAD EPOCHS (no improvement): 4 +2021-03-26 08:26:41,986 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:26:43,035 epoch 51 - iter 2/25 - loss 3.02723408 - samples/sec: 122.25 - lr: 0.015625 +2021-03-26 08:26:44,142 epoch 51 - iter 4/25 - loss 2.87706113 - samples/sec: 115.76 - lr: 0.015625 +2021-03-26 08:26:45,109 epoch 51 - iter 6/25 - loss 2.77404606 - samples/sec: 132.55 - lr: 0.015625 +2021-03-26 08:26:46,056 epoch 51 - iter 8/25 - loss 2.63912219 - samples/sec: 135.39 - lr: 0.015625 +2021-03-26 08:26:47,044 epoch 51 - iter 10/25 - loss 2.55065244 - samples/sec: 129.81 - lr: 0.015625 +2021-03-26 08:26:48,022 epoch 51 - iter 12/25 - loss 2.45815141 - samples/sec: 131.01 - lr: 0.015625 +2021-03-26 08:26:48,952 epoch 51 - iter 14/25 - loss 2.44239549 - samples/sec: 137.90 - lr: 0.015625 +2021-03-26 08:26:49,983 epoch 51 - iter 16/25 - loss 2.48094446 - samples/sec: 124.32 - lr: 0.015625 +2021-03-26 08:26:50,892 epoch 51 - iter 18/25 - loss 2.46754158 - samples/sec: 141.04 - lr: 0.015625 +2021-03-26 08:26:51,901 epoch 51 - iter 20/25 - loss 2.51611657 - samples/sec: 127.05 - lr: 0.015625 +2021-03-26 08:26:52,866 epoch 51 - iter 22/25 - loss 2.47020188 - samples/sec: 132.90 - lr: 0.015625 +2021-03-26 08:26:53,750 epoch 51 - iter 24/25 - loss 2.47272858 - samples/sec: 145.09 - lr: 0.015625 +2021-03-26 08:26:54,155 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:26:54,157 EPOCH 51 done: loss 2.4645 - lr 0.0156250 +2021-03-26 08:26:54,882 DEV : loss 5.136854648590088 - score 0.9192 +2021-03-26 08:26:54,897 BAD EPOCHS (no improvement): 1 +2021-03-26 08:26:54,898 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:26:55,863 epoch 52 - iter 2/25 - loss 2.48442078 - samples/sec: 132.85 - lr: 0.015625 +2021-03-26 08:26:56,817 epoch 52 - iter 4/25 - loss 2.43368804 - samples/sec: 134.33 - lr: 0.015625 +2021-03-26 08:26:57,833 epoch 52 - iter 6/25 - loss 2.57455993 - samples/sec: 126.25 - lr: 0.015625 +2021-03-26 08:26:58,839 epoch 52 - iter 8/25 - loss 2.46462786 - samples/sec: 127.35 - lr: 0.015625 +2021-03-26 08:26:59,814 epoch 52 - iter 10/25 - loss 2.44867802 - samples/sec: 131.52 - lr: 0.015625 +2021-03-26 08:27:00,853 epoch 52 - iter 12/25 - loss 2.39227523 - samples/sec: 123.43 - lr: 0.015625 +2021-03-26 08:27:01,798 epoch 52 - iter 14/25 - loss 2.47848124 - samples/sec: 135.69 - lr: 0.015625 +2021-03-26 08:27:02,804 epoch 52 - iter 16/25 - loss 2.57754317 - samples/sec: 127.33 - lr: 0.015625 +2021-03-26 08:27:03,814 epoch 52 - iter 18/25 - loss 2.58900766 - samples/sec: 126.99 - lr: 0.015625 +2021-03-26 08:27:04,879 epoch 52 - iter 20/25 - loss 2.58191739 - samples/sec: 120.40 - lr: 0.015625 +2021-03-26 08:27:05,879 epoch 52 - iter 22/25 - loss 2.58063118 - samples/sec: 128.32 - lr: 0.015625 +2021-03-26 08:27:06,834 epoch 52 - iter 24/25 - loss 2.57483467 - samples/sec: 134.17 - lr: 0.015625 +2021-03-26 08:27:07,393 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:27:07,394 EPOCH 52 done: loss 2.5751 - lr 0.0156250 +2021-03-26 08:27:09,180 DEV : loss 5.142533302307129 - score 0.9175 +2021-03-26 08:27:09,195 BAD EPOCHS (no improvement): 2 +2021-03-26 08:27:09,196 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:27:10,144 epoch 53 - iter 2/25 - loss 2.56813502 - samples/sec: 135.19 - lr: 0.015625 +2021-03-26 08:27:11,149 epoch 53 - iter 4/25 - loss 2.46606696 - samples/sec: 127.61 - lr: 0.015625 +2021-03-26 08:27:12,157 epoch 53 - iter 6/25 - loss 2.41871005 - samples/sec: 127.15 - lr: 0.015625 +2021-03-26 08:27:13,250 epoch 53 - iter 8/25 - loss 2.33384241 - samples/sec: 117.30 - lr: 0.015625 +2021-03-26 08:27:14,257 epoch 53 - iter 10/25 - loss 2.29487942 - samples/sec: 127.19 - lr: 0.015625 +2021-03-26 08:27:15,252 epoch 53 - iter 12/25 - loss 2.33890817 - samples/sec: 128.85 - lr: 0.015625 +2021-03-26 08:27:16,236 epoch 53 - iter 14/25 - loss 2.33417889 - samples/sec: 130.32 - lr: 0.015625 +2021-03-26 08:27:17,230 epoch 53 - iter 16/25 - loss 2.37781139 - samples/sec: 128.96 - lr: 0.015625 +2021-03-26 08:27:18,275 epoch 53 - iter 18/25 - loss 2.44906245 - samples/sec: 122.64 - lr: 0.015625 +2021-03-26 08:27:19,250 epoch 53 - iter 20/25 - loss 2.49812522 - samples/sec: 131.47 - lr: 0.015625 +2021-03-26 08:27:20,167 epoch 53 - iter 22/25 - loss 2.46865853 - samples/sec: 139.92 - lr: 0.015625 +2021-03-26 08:27:21,113 epoch 53 - iter 24/25 - loss 2.51417203 - samples/sec: 135.49 - lr: 0.015625 +2021-03-26 08:27:21,479 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:27:21,480 EPOCH 53 done: loss 2.4904 - lr 0.0156250 +2021-03-26 08:27:22,198 DEV : loss 5.138154029846191 - score 0.9173 +2021-03-26 08:27:22,223 BAD EPOCHS (no improvement): 3 +2021-03-26 08:27:22,224 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:27:23,305 epoch 54 - iter 2/25 - loss 2.55291057 - samples/sec: 118.67 - lr: 0.015625 +2021-03-26 08:27:24,306 epoch 54 - iter 4/25 - loss 2.38039738 - samples/sec: 128.05 - lr: 0.015625 +2021-03-26 08:27:25,219 epoch 54 - iter 6/25 - loss 2.46816170 - samples/sec: 140.38 - lr: 0.015625 +2021-03-26 08:27:26,232 epoch 54 - iter 8/25 - loss 2.57770142 - samples/sec: 126.63 - lr: 0.015625 +2021-03-26 08:27:27,285 epoch 54 - iter 10/25 - loss 2.55315492 - samples/sec: 121.71 - lr: 0.015625 +2021-03-26 08:27:28,352 epoch 54 - iter 12/25 - loss 2.57296405 - samples/sec: 120.11 - lr: 0.015625 +2021-03-26 08:27:29,343 epoch 54 - iter 14/25 - loss 2.53743523 - samples/sec: 129.49 - lr: 0.015625 +2021-03-26 08:27:30,254 epoch 54 - iter 16/25 - loss 2.55962446 - samples/sec: 140.64 - lr: 0.015625 +2021-03-26 08:27:31,290 epoch 54 - iter 18/25 - loss 2.58272949 - samples/sec: 123.80 - lr: 0.015625 +2021-03-26 08:27:32,325 epoch 54 - iter 20/25 - loss 2.56538886 - samples/sec: 123.75 - lr: 0.015625 +2021-03-26 08:27:33,459 epoch 54 - iter 22/25 - loss 2.54301760 - samples/sec: 113.08 - lr: 0.015625 +2021-03-26 08:27:34,489 epoch 54 - iter 24/25 - loss 2.54989679 - samples/sec: 124.46 - lr: 0.015625 +2021-03-26 08:27:34,888 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:27:34,889 EPOCH 54 done: loss 2.5705 - lr 0.0156250 +2021-03-26 08:27:35,606 DEV : loss 5.1629109382629395 - score 0.9149 +2021-03-26 08:27:35,629 BAD EPOCHS (no improvement): 4 +2021-03-26 08:27:35,629 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:27:36,630 epoch 55 - iter 2/25 - loss 3.01063824 - samples/sec: 128.15 - lr: 0.007812 +2021-03-26 08:27:37,619 epoch 55 - iter 4/25 - loss 2.87859678 - samples/sec: 129.70 - lr: 0.007812 +2021-03-26 08:27:38,620 epoch 55 - iter 6/25 - loss 2.72049872 - samples/sec: 128.02 - lr: 0.007812 +2021-03-26 08:27:39,644 epoch 55 - iter 8/25 - loss 2.69411999 - samples/sec: 125.21 - lr: 0.007812 +2021-03-26 08:27:40,542 epoch 55 - iter 10/25 - loss 2.62634434 - samples/sec: 142.75 - lr: 0.007812 +2021-03-26 08:27:41,576 epoch 55 - iter 12/25 - loss 2.62428525 - samples/sec: 123.99 - lr: 0.007812 +2021-03-26 08:27:42,613 epoch 55 - iter 14/25 - loss 2.71168633 - samples/sec: 123.68 - lr: 0.007812 +2021-03-26 08:27:43,515 epoch 55 - iter 16/25 - loss 2.64237929 - samples/sec: 142.18 - lr: 0.007812 +2021-03-26 08:27:44,608 epoch 55 - iter 18/25 - loss 2.62173416 - samples/sec: 117.26 - lr: 0.007812 +2021-03-26 08:27:45,625 epoch 55 - iter 20/25 - loss 2.61856262 - samples/sec: 126.10 - lr: 0.007812 +2021-03-26 08:27:46,520 epoch 55 - iter 22/25 - loss 2.58942200 - samples/sec: 143.26 - lr: 0.007812 +2021-03-26 08:27:47,575 epoch 55 - iter 24/25 - loss 2.52959455 - samples/sec: 121.56 - lr: 0.007812 +2021-03-26 08:27:48,022 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:27:48,023 EPOCH 55 done: loss 2.5290 - lr 0.0078125 +2021-03-26 08:27:48,748 DEV : loss 5.161101341247559 - score 0.9162 +2021-03-26 08:27:48,771 BAD EPOCHS (no improvement): 1 +2021-03-26 08:27:48,772 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:27:49,929 epoch 56 - iter 2/25 - loss 2.89138258 - samples/sec: 110.79 - lr: 0.007812 +2021-03-26 08:27:50,953 epoch 56 - iter 4/25 - loss 2.80591446 - samples/sec: 125.10 - lr: 0.007812 +2021-03-26 08:27:52,010 epoch 56 - iter 6/25 - loss 2.80518850 - samples/sec: 121.30 - lr: 0.007812 +2021-03-26 08:27:53,149 epoch 56 - iter 8/25 - loss 2.69240418 - samples/sec: 112.59 - lr: 0.007812 +2021-03-26 08:27:54,275 epoch 56 - iter 10/25 - loss 2.70520034 - samples/sec: 113.91 - lr: 0.007812 +2021-03-26 08:27:55,224 epoch 56 - iter 12/25 - loss 2.69055593 - samples/sec: 135.07 - lr: 0.007812 +2021-03-26 08:27:56,175 epoch 56 - iter 14/25 - loss 2.69155152 - samples/sec: 134.80 - lr: 0.007812 +2021-03-26 08:27:57,197 epoch 56 - iter 16/25 - loss 2.69912042 - samples/sec: 125.48 - lr: 0.007812 +2021-03-26 08:27:58,337 epoch 56 - iter 18/25 - loss 2.68279980 - samples/sec: 112.50 - lr: 0.007812 +2021-03-26 08:27:59,445 epoch 56 - iter 20/25 - loss 2.70082166 - samples/sec: 115.69 - lr: 0.007812 +2021-03-26 08:28:00,441 epoch 56 - iter 22/25 - loss 2.71463389 - samples/sec: 128.77 - lr: 0.007812 +2021-03-26 08:28:01,455 epoch 56 - iter 24/25 - loss 2.68750911 - samples/sec: 126.45 - lr: 0.007812 +2021-03-26 08:28:01,809 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:28:01,811 EPOCH 56 done: loss 2.6804 - lr 0.0078125 +2021-03-26 08:28:02,535 DEV : loss 5.161330223083496 - score 0.916 +2021-03-26 08:28:02,558 BAD EPOCHS (no improvement): 2 +2021-03-26 08:28:02,559 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:28:03,556 epoch 57 - iter 2/25 - loss 1.81027997 - samples/sec: 128.61 - lr: 0.007812 +2021-03-26 08:28:04,586 epoch 57 - iter 4/25 - loss 1.96821702 - samples/sec: 124.52 - lr: 0.007812 +2021-03-26 08:28:05,528 epoch 57 - iter 6/25 - loss 2.20572285 - samples/sec: 136.05 - lr: 0.007812 +2021-03-26 08:28:06,513 epoch 57 - iter 8/25 - loss 2.21117917 - samples/sec: 130.15 - lr: 0.007812 +2021-03-26 08:28:07,565 epoch 57 - iter 10/25 - loss 2.29010668 - samples/sec: 121.87 - lr: 0.007812 +2021-03-26 08:28:08,619 epoch 57 - iter 12/25 - loss 2.39001824 - samples/sec: 121.53 - lr: 0.007812 +2021-03-26 08:28:09,591 epoch 57 - iter 14/25 - loss 2.41587717 - samples/sec: 131.94 - lr: 0.007812 +2021-03-26 08:28:10,608 epoch 57 - iter 16/25 - loss 2.40491781 - samples/sec: 126.06 - lr: 0.007812 +2021-03-26 08:28:11,610 epoch 57 - iter 18/25 - loss 2.45139188 - samples/sec: 127.92 - lr: 0.007812 +2021-03-26 08:28:12,515 epoch 57 - iter 20/25 - loss 2.46481062 - samples/sec: 141.71 - lr: 0.007812 +2021-03-26 08:28:13,446 epoch 57 - iter 22/25 - loss 2.46444411 - samples/sec: 137.76 - lr: 0.007812 +2021-03-26 08:28:14,378 epoch 57 - iter 24/25 - loss 2.46720476 - samples/sec: 137.55 - lr: 0.007812 +2021-03-26 08:28:14,801 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:28:14,802 EPOCH 57 done: loss 2.4673 - lr 0.0078125 +2021-03-26 08:28:15,513 DEV : loss 5.167983055114746 - score 0.9156 +2021-03-26 08:28:15,536 BAD EPOCHS (no improvement): 3 +2021-03-26 08:28:15,537 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:28:16,584 epoch 58 - iter 2/25 - loss 2.96477020 - samples/sec: 122.47 - lr: 0.007812 +2021-03-26 08:28:17,500 epoch 58 - iter 4/25 - loss 2.81111413 - samples/sec: 140.09 - lr: 0.007812 +2021-03-26 08:28:18,581 epoch 58 - iter 6/25 - loss 2.69619540 - samples/sec: 118.53 - lr: 0.007812 +2021-03-26 08:28:19,589 epoch 58 - iter 8/25 - loss 2.66887558 - samples/sec: 127.16 - lr: 0.007812 +2021-03-26 08:28:20,526 epoch 58 - iter 10/25 - loss 2.64879587 - samples/sec: 136.90 - lr: 0.007812 +2021-03-26 08:28:21,415 epoch 58 - iter 12/25 - loss 2.59787667 - samples/sec: 144.25 - lr: 0.007812 +2021-03-26 08:28:22,428 epoch 58 - iter 14/25 - loss 2.65796592 - samples/sec: 126.51 - lr: 0.007812 +2021-03-26 08:28:23,432 epoch 58 - iter 16/25 - loss 2.66392018 - samples/sec: 127.69 - lr: 0.007812 +2021-03-26 08:28:24,366 epoch 58 - iter 18/25 - loss 2.61717865 - samples/sec: 137.23 - lr: 0.007812 +2021-03-26 08:28:25,447 epoch 58 - iter 20/25 - loss 2.58745240 - samples/sec: 118.49 - lr: 0.007812 +2021-03-26 08:28:26,437 epoch 58 - iter 22/25 - loss 2.57686839 - samples/sec: 129.51 - lr: 0.007812 +2021-03-26 08:28:27,385 epoch 58 - iter 24/25 - loss 2.61233709 - samples/sec: 135.25 - lr: 0.007812 +2021-03-26 08:28:27,794 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:28:27,795 EPOCH 58 done: loss 2.5902 - lr 0.0078125 +2021-03-26 08:28:28,528 DEV : loss 5.153556823730469 - score 0.9166 +2021-03-26 08:28:28,551 BAD EPOCHS (no improvement): 4 +2021-03-26 08:28:28,552 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:28:29,576 epoch 59 - iter 2/25 - loss 2.64156973 - samples/sec: 125.17 - lr: 0.003906 +2021-03-26 08:28:30,523 epoch 59 - iter 4/25 - loss 2.81922227 - samples/sec: 135.44 - lr: 0.003906 +2021-03-26 08:28:31,506 epoch 59 - iter 6/25 - loss 2.72071330 - samples/sec: 130.46 - lr: 0.003906 +2021-03-26 08:28:32,542 epoch 59 - iter 8/25 - loss 2.69526169 - samples/sec: 123.69 - lr: 0.003906 +2021-03-26 08:28:33,514 epoch 59 - iter 10/25 - loss 2.74493732 - samples/sec: 131.84 - lr: 0.003906 +2021-03-26 08:28:34,443 epoch 59 - iter 12/25 - loss 2.66976053 - samples/sec: 138.06 - lr: 0.003906 +2021-03-26 08:28:35,604 epoch 59 - iter 14/25 - loss 2.61479444 - samples/sec: 110.40 - lr: 0.003906 +2021-03-26 08:28:36,573 epoch 59 - iter 16/25 - loss 2.61310622 - samples/sec: 132.22 - lr: 0.003906 +2021-03-26 08:28:37,494 epoch 59 - iter 18/25 - loss 2.58530426 - samples/sec: 139.19 - lr: 0.003906 +2021-03-26 08:28:38,582 epoch 59 - iter 20/25 - loss 2.56393338 - samples/sec: 117.78 - lr: 0.003906 +2021-03-26 08:28:39,580 epoch 59 - iter 22/25 - loss 2.59042783 - samples/sec: 128.40 - lr: 0.003906 +2021-03-26 08:28:40,582 epoch 59 - iter 24/25 - loss 2.55683567 - samples/sec: 127.95 - lr: 0.003906 +2021-03-26 08:28:41,005 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:28:41,006 EPOCH 59 done: loss 2.5670 - lr 0.0039062 +2021-03-26 08:28:41,716 DEV : loss 5.155137538909912 - score 0.9166 +2021-03-26 08:28:41,738 BAD EPOCHS (no improvement): 1 +2021-03-26 08:28:41,739 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:28:42,744 epoch 60 - iter 2/25 - loss 3.35700023 - samples/sec: 127.64 - lr: 0.003906 +2021-03-26 08:28:43,651 epoch 60 - iter 4/25 - loss 2.85628122 - samples/sec: 141.54 - lr: 0.003906 +2021-03-26 08:28:44,655 epoch 60 - iter 6/25 - loss 2.71600425 - samples/sec: 127.60 - lr: 0.003906 +2021-03-26 08:28:45,584 epoch 60 - iter 8/25 - loss 2.66859806 - samples/sec: 138.13 - lr: 0.003906 +2021-03-26 08:28:46,620 epoch 60 - iter 10/25 - loss 2.69276667 - samples/sec: 123.65 - lr: 0.003906 +2021-03-26 08:28:47,548 epoch 60 - iter 12/25 - loss 2.62539530 - samples/sec: 138.32 - lr: 0.003906 +2021-03-26 08:28:48,426 epoch 60 - iter 14/25 - loss 2.65403557 - samples/sec: 146.05 - lr: 0.003906 +2021-03-26 08:28:49,579 epoch 60 - iter 16/25 - loss 2.66666107 - samples/sec: 111.10 - lr: 0.003906 +2021-03-26 08:28:50,516 epoch 60 - iter 18/25 - loss 2.64792878 - samples/sec: 136.92 - lr: 0.003906 +2021-03-26 08:28:51,531 epoch 60 - iter 20/25 - loss 2.60760083 - samples/sec: 126.23 - lr: 0.003906 +2021-03-26 08:28:52,518 epoch 60 - iter 22/25 - loss 2.58448346 - samples/sec: 129.86 - lr: 0.003906 +2021-03-26 08:28:53,512 epoch 60 - iter 24/25 - loss 2.58142538 - samples/sec: 129.00 - lr: 0.003906 +2021-03-26 08:28:53,869 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:28:53,869 EPOCH 60 done: loss 2.5570 - lr 0.0039062 +2021-03-26 08:28:54,605 DEV : loss 5.156240463256836 - score 0.9166 +2021-03-26 08:28:54,628 BAD EPOCHS (no improvement): 2 +2021-03-26 08:28:54,629 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:28:55,572 epoch 61 - iter 2/25 - loss 2.64861023 - samples/sec: 135.91 - lr: 0.003906 +2021-03-26 08:28:56,509 epoch 61 - iter 4/25 - loss 2.33936024 - samples/sec: 136.95 - lr: 0.003906 +2021-03-26 08:28:57,509 epoch 61 - iter 6/25 - loss 2.32444807 - samples/sec: 128.15 - lr: 0.003906 +2021-03-26 08:28:58,539 epoch 61 - iter 8/25 - loss 2.34189090 - samples/sec: 124.46 - lr: 0.003906 +2021-03-26 08:28:59,639 epoch 61 - iter 10/25 - loss 2.37957444 - samples/sec: 116.45 - lr: 0.003906 +2021-03-26 08:29:00,688 epoch 61 - iter 12/25 - loss 2.39998172 - samples/sec: 122.20 - lr: 0.003906 +2021-03-26 08:29:01,645 epoch 61 - iter 14/25 - loss 2.42385345 - samples/sec: 133.87 - lr: 0.003906 +2021-03-26 08:29:02,655 epoch 61 - iter 16/25 - loss 2.39101814 - samples/sec: 126.88 - lr: 0.003906 +2021-03-26 08:29:03,618 epoch 61 - iter 18/25 - loss 2.43700922 - samples/sec: 133.17 - lr: 0.003906 +2021-03-26 08:29:04,541 epoch 61 - iter 20/25 - loss 2.40053543 - samples/sec: 138.82 - lr: 0.003906 +2021-03-26 08:29:05,590 epoch 61 - iter 22/25 - loss 2.42045249 - samples/sec: 122.23 - lr: 0.003906 +2021-03-26 08:29:06,643 epoch 61 - iter 24/25 - loss 2.46234730 - samples/sec: 121.64 - lr: 0.003906 +2021-03-26 08:29:07,077 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:29:07,078 EPOCH 61 done: loss 2.5105 - lr 0.0039062 +2021-03-26 08:29:07,818 DEV : loss 5.1566362380981445 - score 0.9164 +2021-03-26 08:29:07,842 BAD EPOCHS (no improvement): 3 +2021-03-26 08:29:07,843 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:29:08,811 epoch 62 - iter 2/25 - loss 2.52020955 - samples/sec: 132.49 - lr: 0.003906 +2021-03-26 08:29:09,859 epoch 62 - iter 4/25 - loss 2.50968593 - samples/sec: 122.28 - lr: 0.003906 +2021-03-26 08:29:10,840 epoch 62 - iter 6/25 - loss 2.37634569 - samples/sec: 130.69 - lr: 0.003906 +2021-03-26 08:29:11,786 epoch 62 - iter 8/25 - loss 2.36597966 - samples/sec: 135.68 - lr: 0.003906 +2021-03-26 08:29:12,737 epoch 62 - iter 10/25 - loss 2.49966995 - samples/sec: 134.79 - lr: 0.003906 +2021-03-26 08:29:13,878 epoch 62 - iter 12/25 - loss 2.49869683 - samples/sec: 112.32 - lr: 0.003906 +2021-03-26 08:29:14,888 epoch 62 - iter 14/25 - loss 2.52862756 - samples/sec: 126.88 - lr: 0.003906 +2021-03-26 08:29:15,992 epoch 62 - iter 16/25 - loss 2.55252969 - samples/sec: 116.03 - lr: 0.003906 +2021-03-26 08:29:17,034 epoch 62 - iter 18/25 - loss 2.60047205 - samples/sec: 123.04 - lr: 0.003906 +2021-03-26 08:29:17,979 epoch 62 - iter 20/25 - loss 2.59337807 - samples/sec: 135.56 - lr: 0.003906 +2021-03-26 08:29:18,979 epoch 62 - iter 22/25 - loss 2.61051676 - samples/sec: 128.26 - lr: 0.003906 +2021-03-26 08:29:19,983 epoch 62 - iter 24/25 - loss 2.62917062 - samples/sec: 127.69 - lr: 0.003906 +2021-03-26 08:29:20,411 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:29:20,411 EPOCH 62 done: loss 2.6084 - lr 0.0039062 +2021-03-26 08:29:21,224 DEV : loss 5.154888153076172 - score 0.9173 +2021-03-26 08:29:21,255 BAD EPOCHS (no improvement): 4 +2021-03-26 08:29:21,256 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:29:22,282 epoch 63 - iter 2/25 - loss 2.45950496 - samples/sec: 124.96 - lr: 0.001953 +2021-03-26 08:29:23,280 epoch 63 - iter 4/25 - loss 2.58118606 - samples/sec: 128.45 - lr: 0.001953 +2021-03-26 08:29:24,371 epoch 63 - iter 6/25 - loss 2.55522390 - samples/sec: 117.52 - lr: 0.001953 +2021-03-26 08:29:25,436 epoch 63 - iter 8/25 - loss 2.64915141 - samples/sec: 120.42 - lr: 0.001953 +2021-03-26 08:29:26,379 epoch 63 - iter 10/25 - loss 2.60279477 - samples/sec: 135.94 - lr: 0.001953 +2021-03-26 08:29:27,384 epoch 63 - iter 12/25 - loss 2.62300179 - samples/sec: 127.52 - lr: 0.001953 +2021-03-26 08:29:28,390 epoch 63 - iter 14/25 - loss 2.74238583 - samples/sec: 127.43 - lr: 0.001953 +2021-03-26 08:29:29,472 epoch 63 - iter 16/25 - loss 2.72329572 - samples/sec: 118.47 - lr: 0.001953 +2021-03-26 08:29:30,499 epoch 63 - iter 18/25 - loss 2.68225120 - samples/sec: 124.89 - lr: 0.001953 +2021-03-26 08:29:31,480 epoch 63 - iter 20/25 - loss 2.65703490 - samples/sec: 130.69 - lr: 0.001953 +2021-03-26 08:29:32,507 epoch 63 - iter 22/25 - loss 2.67516613 - samples/sec: 124.84 - lr: 0.001953 +2021-03-26 08:29:33,483 epoch 63 - iter 24/25 - loss 2.68743004 - samples/sec: 131.40 - lr: 0.001953 +2021-03-26 08:29:33,891 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:29:33,892 EPOCH 63 done: loss 2.6802 - lr 0.0019531 +2021-03-26 08:29:34,623 DEV : loss 5.15378475189209 - score 0.9175 +2021-03-26 08:29:34,655 BAD EPOCHS (no improvement): 1 +2021-03-26 08:29:34,656 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:29:35,734 epoch 64 - iter 2/25 - loss 3.01414752 - samples/sec: 119.03 - lr: 0.001953 +2021-03-26 08:29:36,775 epoch 64 - iter 4/25 - loss 2.53828612 - samples/sec: 123.06 - lr: 0.001953 +2021-03-26 08:29:37,735 epoch 64 - iter 6/25 - loss 2.57090575 - samples/sec: 133.51 - lr: 0.001953 +2021-03-26 08:29:38,693 epoch 64 - iter 8/25 - loss 2.60916208 - samples/sec: 133.92 - lr: 0.001953 +2021-03-26 08:29:39,637 epoch 64 - iter 10/25 - loss 2.62027467 - samples/sec: 135.80 - lr: 0.001953 +2021-03-26 08:29:40,653 epoch 64 - iter 12/25 - loss 2.55137331 - samples/sec: 126.13 - lr: 0.001953 +2021-03-26 08:29:41,744 epoch 64 - iter 14/25 - loss 2.54606689 - samples/sec: 117.47 - lr: 0.001953 +2021-03-26 08:29:42,749 epoch 64 - iter 16/25 - loss 2.63612720 - samples/sec: 127.60 - lr: 0.001953 +2021-03-26 08:29:43,739 epoch 64 - iter 18/25 - loss 2.65265855 - samples/sec: 129.55 - lr: 0.001953 +2021-03-26 08:29:44,720 epoch 64 - iter 20/25 - loss 2.65310151 - samples/sec: 130.76 - lr: 0.001953 +2021-03-26 08:29:45,689 epoch 64 - iter 22/25 - loss 2.61852049 - samples/sec: 132.17 - lr: 0.001953 +2021-03-26 08:29:46,721 epoch 64 - iter 24/25 - loss 2.56815796 - samples/sec: 124.21 - lr: 0.001953 +2021-03-26 08:29:47,112 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:29:47,113 EPOCH 64 done: loss 2.6146 - lr 0.0019531 +2021-03-26 08:29:47,886 DEV : loss 5.153861999511719 - score 0.9171 +2021-03-26 08:29:47,901 BAD EPOCHS (no improvement): 2 +2021-03-26 08:29:47,901 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:29:48,902 epoch 65 - iter 2/25 - loss 2.75063539 - samples/sec: 128.18 - lr: 0.001953 +2021-03-26 08:29:49,966 epoch 65 - iter 4/25 - loss 2.82495999 - samples/sec: 120.45 - lr: 0.001953 +2021-03-26 08:29:51,020 epoch 65 - iter 6/25 - loss 2.78207270 - samples/sec: 121.65 - lr: 0.001953 +2021-03-26 08:29:52,051 epoch 65 - iter 8/25 - loss 2.78051665 - samples/sec: 124.34 - lr: 0.001953 +2021-03-26 08:29:53,054 epoch 65 - iter 10/25 - loss 2.76111586 - samples/sec: 127.88 - lr: 0.001953 +2021-03-26 08:29:54,088 epoch 65 - iter 12/25 - loss 2.72665600 - samples/sec: 123.91 - lr: 0.001953 +2021-03-26 08:29:55,081 epoch 65 - iter 14/25 - loss 2.69365876 - samples/sec: 129.06 - lr: 0.001953 +2021-03-26 08:29:56,018 epoch 65 - iter 16/25 - loss 2.62989879 - samples/sec: 136.80 - lr: 0.001953 +2021-03-26 08:29:57,029 epoch 65 - iter 18/25 - loss 2.59235953 - samples/sec: 126.74 - lr: 0.001953 +2021-03-26 08:29:57,989 epoch 65 - iter 20/25 - loss 2.57889878 - samples/sec: 133.57 - lr: 0.001953 +2021-03-26 08:29:58,875 epoch 65 - iter 22/25 - loss 2.57340151 - samples/sec: 144.83 - lr: 0.001953 +2021-03-26 08:29:59,849 epoch 65 - iter 24/25 - loss 2.59013875 - samples/sec: 131.54 - lr: 0.001953 +2021-03-26 08:30:00,230 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:30:00,231 EPOCH 65 done: loss 2.5700 - lr 0.0019531 +2021-03-26 08:30:00,995 DEV : loss 5.1544013023376465 - score 0.9175 +2021-03-26 08:30:01,026 BAD EPOCHS (no improvement): 3 +2021-03-26 08:30:01,027 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:30:01,991 epoch 66 - iter 2/25 - loss 2.42480397 - samples/sec: 133.06 - lr: 0.001953 +2021-03-26 08:30:03,005 epoch 66 - iter 4/25 - loss 2.47575986 - samples/sec: 126.34 - lr: 0.001953 +2021-03-26 08:30:04,018 epoch 66 - iter 6/25 - loss 2.42025113 - samples/sec: 126.51 - lr: 0.001953 +2021-03-26 08:30:05,271 epoch 66 - iter 8/25 - loss 2.42866611 - samples/sec: 102.29 - lr: 0.001953 +2021-03-26 08:30:06,456 epoch 66 - iter 10/25 - loss 2.36550404 - samples/sec: 108.13 - lr: 0.001953 +2021-03-26 08:30:07,496 epoch 66 - iter 12/25 - loss 2.32203812 - samples/sec: 123.30 - lr: 0.001953 +2021-03-26 08:30:08,682 epoch 66 - iter 14/25 - loss 2.29170151 - samples/sec: 108.09 - lr: 0.001953 +2021-03-26 08:30:09,671 epoch 66 - iter 16/25 - loss 2.31758191 - samples/sec: 129.56 - lr: 0.001953 +2021-03-26 08:30:10,646 epoch 66 - iter 18/25 - loss 2.36295978 - samples/sec: 131.56 - lr: 0.001953 +2021-03-26 08:30:11,611 epoch 66 - iter 20/25 - loss 2.32609164 - samples/sec: 132.79 - lr: 0.001953 +2021-03-26 08:30:12,534 epoch 66 - iter 22/25 - loss 2.36869703 - samples/sec: 138.88 - lr: 0.001953 +2021-03-26 08:30:13,541 epoch 66 - iter 24/25 - loss 2.39990243 - samples/sec: 127.23 - lr: 0.001953 +2021-03-26 08:30:13,984 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:30:13,986 EPOCH 66 done: loss 2.3938 - lr 0.0019531 +2021-03-26 08:30:14,680 DEV : loss 5.154584884643555 - score 0.9179 +2021-03-26 08:30:14,695 BAD EPOCHS (no improvement): 4 +2021-03-26 08:30:14,696 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:30:15,634 epoch 67 - iter 2/25 - loss 2.01868904 - samples/sec: 136.63 - lr: 0.000977 +2021-03-26 08:30:16,662 epoch 67 - iter 4/25 - loss 2.34448314 - samples/sec: 124.80 - lr: 0.000977 +2021-03-26 08:30:17,615 epoch 67 - iter 6/25 - loss 2.43418237 - samples/sec: 134.41 - lr: 0.000977 +2021-03-26 08:30:18,661 epoch 67 - iter 8/25 - loss 2.47001508 - samples/sec: 122.58 - lr: 0.000977 +2021-03-26 08:30:19,690 epoch 67 - iter 10/25 - loss 2.52531345 - samples/sec: 124.62 - lr: 0.000977 +2021-03-26 08:30:20,662 epoch 67 - iter 12/25 - loss 2.61063679 - samples/sec: 131.93 - lr: 0.000977 +2021-03-26 08:30:21,524 epoch 67 - iter 14/25 - loss 2.57734524 - samples/sec: 148.73 - lr: 0.000977 +2021-03-26 08:30:22,493 epoch 67 - iter 16/25 - loss 2.52899256 - samples/sec: 132.34 - lr: 0.000977 +2021-03-26 08:30:23,473 epoch 67 - iter 18/25 - loss 2.50297968 - samples/sec: 130.84 - lr: 0.000977 +2021-03-26 08:30:24,447 epoch 67 - iter 20/25 - loss 2.49524022 - samples/sec: 131.62 - lr: 0.000977 +2021-03-26 08:30:25,466 epoch 67 - iter 22/25 - loss 2.46065530 - samples/sec: 125.72 - lr: 0.000977 +2021-03-26 08:30:26,358 epoch 67 - iter 24/25 - loss 2.47536785 - samples/sec: 143.77 - lr: 0.000977 +2021-03-26 08:30:26,731 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:30:26,732 EPOCH 67 done: loss 2.4502 - lr 0.0009766 +2021-03-26 08:30:27,441 DEV : loss 5.155641555786133 - score 0.9177 +2021-03-26 08:30:27,463 BAD EPOCHS (no improvement): 1 +2021-03-26 08:30:27,464 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:30:28,450 epoch 68 - iter 2/25 - loss 2.34856331 - samples/sec: 130.14 - lr: 0.000977 +2021-03-26 08:30:29,385 epoch 68 - iter 4/25 - loss 2.51698828 - samples/sec: 137.22 - lr: 0.000977 +2021-03-26 08:30:30,368 epoch 68 - iter 6/25 - loss 2.46542730 - samples/sec: 130.36 - lr: 0.000977 +2021-03-26 08:30:31,309 epoch 68 - iter 8/25 - loss 2.51394932 - samples/sec: 136.28 - lr: 0.000977 +2021-03-26 08:30:32,230 epoch 68 - iter 10/25 - loss 2.52544693 - samples/sec: 139.25 - lr: 0.000977 +2021-03-26 08:30:33,171 epoch 68 - iter 12/25 - loss 2.41423985 - samples/sec: 136.25 - lr: 0.000977 +2021-03-26 08:30:34,115 epoch 68 - iter 14/25 - loss 2.55178968 - samples/sec: 135.74 - lr: 0.000977 +2021-03-26 08:30:35,129 epoch 68 - iter 16/25 - loss 2.56635130 - samples/sec: 126.42 - lr: 0.000977 +2021-03-26 08:30:36,220 epoch 68 - iter 18/25 - loss 2.52978544 - samples/sec: 117.46 - lr: 0.000977 +2021-03-26 08:30:37,361 epoch 68 - iter 20/25 - loss 2.50168654 - samples/sec: 112.36 - lr: 0.000977 +2021-03-26 08:30:38,438 epoch 68 - iter 22/25 - loss 2.47874781 - samples/sec: 119.03 - lr: 0.000977 +2021-03-26 08:30:39,456 epoch 68 - iter 24/25 - loss 2.49064927 - samples/sec: 125.92 - lr: 0.000977 +2021-03-26 08:30:39,857 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:30:39,858 EPOCH 68 done: loss 2.4624 - lr 0.0009766 +2021-03-26 08:30:40,580 DEV : loss 5.155262470245361 - score 0.9179 +2021-03-26 08:30:40,595 BAD EPOCHS (no improvement): 2 +2021-03-26 08:30:40,596 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:30:41,629 epoch 69 - iter 2/25 - loss 3.18188810 - samples/sec: 124.03 - lr: 0.000977 +2021-03-26 08:30:42,601 epoch 69 - iter 4/25 - loss 2.74411315 - samples/sec: 131.96 - lr: 0.000977 +2021-03-26 08:30:43,523 epoch 69 - iter 6/25 - loss 2.70093520 - samples/sec: 139.06 - lr: 0.000977 +2021-03-26 08:30:44,440 epoch 69 - iter 8/25 - loss 2.50844127 - samples/sec: 139.71 - lr: 0.000977 +2021-03-26 08:30:45,597 epoch 69 - iter 10/25 - loss 2.57270017 - samples/sec: 110.85 - lr: 0.000977 +2021-03-26 08:30:46,508 epoch 69 - iter 12/25 - loss 2.53602714 - samples/sec: 140.61 - lr: 0.000977 +2021-03-26 08:30:47,470 epoch 69 - iter 14/25 - loss 2.56787088 - samples/sec: 133.26 - lr: 0.000977 +2021-03-26 08:30:48,447 epoch 69 - iter 16/25 - loss 2.55436224 - samples/sec: 131.27 - lr: 0.000977 +2021-03-26 08:30:49,456 epoch 69 - iter 18/25 - loss 2.53948792 - samples/sec: 126.96 - lr: 0.000977 +2021-03-26 08:30:50,488 epoch 69 - iter 20/25 - loss 2.50969934 - samples/sec: 124.29 - lr: 0.000977 +2021-03-26 08:30:51,515 epoch 69 - iter 22/25 - loss 2.54528459 - samples/sec: 124.70 - lr: 0.000977 +2021-03-26 08:30:52,466 epoch 69 - iter 24/25 - loss 2.50940784 - samples/sec: 134.92 - lr: 0.000977 +2021-03-26 08:30:52,890 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:30:52,890 EPOCH 69 done: loss 2.4829 - lr 0.0009766 +2021-03-26 08:30:53,588 DEV : loss 5.154378890991211 - score 0.9179 +2021-03-26 08:30:53,604 BAD EPOCHS (no improvement): 3 +2021-03-26 08:30:53,605 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:30:54,681 epoch 70 - iter 2/25 - loss 2.56138349 - samples/sec: 119.15 - lr: 0.000977 +2021-03-26 08:30:55,692 epoch 70 - iter 4/25 - loss 2.57898003 - samples/sec: 126.92 - lr: 0.000977 +2021-03-26 08:30:56,590 epoch 70 - iter 6/25 - loss 2.47086718 - samples/sec: 142.71 - lr: 0.000977 +2021-03-26 08:30:57,656 epoch 70 - iter 8/25 - loss 2.35704540 - samples/sec: 120.34 - lr: 0.000977 +2021-03-26 08:30:58,601 epoch 70 - iter 10/25 - loss 2.33217787 - samples/sec: 135.71 - lr: 0.000977 +2021-03-26 08:30:59,560 epoch 70 - iter 12/25 - loss 2.34995758 - samples/sec: 133.71 - lr: 0.000977 +2021-03-26 08:31:00,545 epoch 70 - iter 14/25 - loss 2.37106590 - samples/sec: 130.08 - lr: 0.000977 +2021-03-26 08:31:01,544 epoch 70 - iter 16/25 - loss 2.41983237 - samples/sec: 128.41 - lr: 0.000977 +2021-03-26 08:31:02,613 epoch 70 - iter 18/25 - loss 2.44145462 - samples/sec: 119.88 - lr: 0.000977 +2021-03-26 08:31:03,609 epoch 70 - iter 20/25 - loss 2.45750895 - samples/sec: 128.85 - lr: 0.000977 +2021-03-26 08:31:04,670 epoch 70 - iter 22/25 - loss 2.48850318 - samples/sec: 120.84 - lr: 0.000977 +2021-03-26 08:31:05,761 epoch 70 - iter 24/25 - loss 2.48575143 - samples/sec: 117.54 - lr: 0.000977 +2021-03-26 08:31:06,188 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:31:06,188 EPOCH 70 done: loss 2.5322 - lr 0.0009766 +2021-03-26 08:31:06,896 DEV : loss 5.15534782409668 - score 0.9177 +2021-03-26 08:31:06,918 BAD EPOCHS (no improvement): 4 +2021-03-26 08:31:06,918 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:31:07,908 epoch 71 - iter 2/25 - loss 3.07665741 - samples/sec: 129.56 - lr: 0.000488 +2021-03-26 08:31:08,921 epoch 71 - iter 4/25 - loss 2.93741924 - samples/sec: 126.56 - lr: 0.000488 +2021-03-26 08:31:09,860 epoch 71 - iter 6/25 - loss 2.79518795 - samples/sec: 136.51 - lr: 0.000488 +2021-03-26 08:31:10,819 epoch 71 - iter 8/25 - loss 2.67835298 - samples/sec: 133.65 - lr: 0.000488 +2021-03-26 08:31:11,826 epoch 71 - iter 10/25 - loss 2.65618186 - samples/sec: 127.29 - lr: 0.000488 +2021-03-26 08:31:12,830 epoch 71 - iter 12/25 - loss 2.59280755 - samples/sec: 127.68 - lr: 0.000488 +2021-03-26 08:31:13,994 epoch 71 - iter 14/25 - loss 2.60106427 - samples/sec: 110.11 - lr: 0.000488 +2021-03-26 08:31:14,915 epoch 71 - iter 16/25 - loss 2.61127068 - samples/sec: 139.19 - lr: 0.000488 +2021-03-26 08:31:15,794 epoch 71 - iter 18/25 - loss 2.57161060 - samples/sec: 145.86 - lr: 0.000488 +2021-03-26 08:31:16,874 epoch 71 - iter 20/25 - loss 2.60581434 - samples/sec: 118.63 - lr: 0.000488 +2021-03-26 08:31:17,850 epoch 71 - iter 22/25 - loss 2.62448980 - samples/sec: 131.39 - lr: 0.000488 +2021-03-26 08:31:19,151 epoch 71 - iter 24/25 - loss 2.64342704 - samples/sec: 98.51 - lr: 0.000488 +2021-03-26 08:31:19,549 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:31:19,549 EPOCH 71 done: loss 2.6406 - lr 0.0004883 +2021-03-26 08:31:20,282 DEV : loss 5.154711723327637 - score 0.9173 +2021-03-26 08:31:20,297 BAD EPOCHS (no improvement): 1 +2021-03-26 08:31:20,298 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:31:21,324 epoch 72 - iter 2/25 - loss 1.73620844 - samples/sec: 124.91 - lr: 0.000488 +2021-03-26 08:31:22,405 epoch 72 - iter 4/25 - loss 2.02630025 - samples/sec: 118.68 - lr: 0.000488 +2021-03-26 08:31:23,505 epoch 72 - iter 6/25 - loss 2.08867693 - samples/sec: 116.50 - lr: 0.000488 +2021-03-26 08:31:24,492 epoch 72 - iter 8/25 - loss 2.03881234 - samples/sec: 129.96 - lr: 0.000488 +2021-03-26 08:31:25,515 epoch 72 - iter 10/25 - loss 2.06679652 - samples/sec: 125.26 - lr: 0.000488 +2021-03-26 08:31:26,514 epoch 72 - iter 12/25 - loss 2.07280229 - samples/sec: 128.40 - lr: 0.000488 +2021-03-26 08:31:27,522 epoch 72 - iter 14/25 - loss 2.14705983 - samples/sec: 127.19 - lr: 0.000488 +2021-03-26 08:31:28,485 epoch 72 - iter 16/25 - loss 2.14889824 - samples/sec: 133.16 - lr: 0.000488 +2021-03-26 08:31:29,417 epoch 72 - iter 18/25 - loss 2.22106131 - samples/sec: 137.54 - lr: 0.000488 +2021-03-26 08:31:30,459 epoch 72 - iter 20/25 - loss 2.25088767 - samples/sec: 123.02 - lr: 0.000488 +2021-03-26 08:31:31,467 epoch 72 - iter 22/25 - loss 2.30408206 - samples/sec: 127.17 - lr: 0.000488 +2021-03-26 08:31:32,410 epoch 72 - iter 24/25 - loss 2.30297814 - samples/sec: 136.00 - lr: 0.000488 +2021-03-26 08:31:32,946 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:31:32,947 EPOCH 72 done: loss 2.3326 - lr 0.0004883 +2021-03-26 08:31:33,666 DEV : loss 5.154988765716553 - score 0.9175 +2021-03-26 08:31:33,688 BAD EPOCHS (no improvement): 2 +2021-03-26 08:31:33,689 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:31:34,657 epoch 73 - iter 2/25 - loss 2.83746290 - samples/sec: 132.34 - lr: 0.000488 +2021-03-26 08:31:35,618 epoch 73 - iter 4/25 - loss 2.60548675 - samples/sec: 133.51 - lr: 0.000488 +2021-03-26 08:31:36,587 epoch 73 - iter 6/25 - loss 2.58915744 - samples/sec: 132.35 - lr: 0.000488 +2021-03-26 08:31:37,719 epoch 73 - iter 8/25 - loss 2.56479962 - samples/sec: 113.21 - lr: 0.000488 +2021-03-26 08:31:38,667 epoch 73 - iter 10/25 - loss 2.53289689 - samples/sec: 135.30 - lr: 0.000488 +2021-03-26 08:31:39,705 epoch 73 - iter 12/25 - loss 2.49248828 - samples/sec: 123.57 - lr: 0.000488 +2021-03-26 08:31:40,718 epoch 73 - iter 14/25 - loss 2.49144954 - samples/sec: 126.52 - lr: 0.000488 +2021-03-26 08:31:41,708 epoch 73 - iter 16/25 - loss 2.48652732 - samples/sec: 129.41 - lr: 0.000488 +2021-03-26 08:31:42,780 epoch 73 - iter 18/25 - loss 2.51980366 - samples/sec: 119.58 - lr: 0.000488 +2021-03-26 08:31:43,711 epoch 73 - iter 20/25 - loss 2.47590793 - samples/sec: 137.67 - lr: 0.000488 +2021-03-26 08:31:44,699 epoch 73 - iter 22/25 - loss 2.51211202 - samples/sec: 129.74 - lr: 0.000488 +2021-03-26 08:31:45,691 epoch 73 - iter 24/25 - loss 2.48965444 - samples/sec: 129.27 - lr: 0.000488 +2021-03-26 08:31:46,088 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:31:46,089 EPOCH 73 done: loss 2.4835 - lr 0.0004883 +2021-03-26 08:31:46,805 DEV : loss 5.154832363128662 - score 0.9173 +2021-03-26 08:31:46,832 BAD EPOCHS (no improvement): 3 +2021-03-26 08:31:46,832 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:31:47,814 epoch 74 - iter 2/25 - loss 2.08305854 - samples/sec: 130.62 - lr: 0.000488 +2021-03-26 08:31:48,847 epoch 74 - iter 4/25 - loss 2.23228666 - samples/sec: 124.13 - lr: 0.000488 +2021-03-26 08:31:49,796 epoch 74 - iter 6/25 - loss 2.21459860 - samples/sec: 135.18 - lr: 0.000488 +2021-03-26 08:31:50,763 epoch 74 - iter 8/25 - loss 2.23361726 - samples/sec: 132.49 - lr: 0.000488 +2021-03-26 08:31:51,801 epoch 74 - iter 10/25 - loss 2.32135657 - samples/sec: 123.54 - lr: 0.000488 +2021-03-26 08:31:52,773 epoch 74 - iter 12/25 - loss 2.39057528 - samples/sec: 131.91 - lr: 0.000488 +2021-03-26 08:31:53,747 epoch 74 - iter 14/25 - loss 2.43192735 - samples/sec: 131.63 - lr: 0.000488 +2021-03-26 08:31:54,668 epoch 74 - iter 16/25 - loss 2.44178198 - samples/sec: 139.10 - lr: 0.000488 +2021-03-26 08:31:55,677 epoch 74 - iter 18/25 - loss 2.45810711 - samples/sec: 127.22 - lr: 0.000488 +2021-03-26 08:31:56,646 epoch 74 - iter 20/25 - loss 2.42499514 - samples/sec: 132.22 - lr: 0.000488 +2021-03-26 08:31:57,572 epoch 74 - iter 22/25 - loss 2.40459371 - samples/sec: 138.51 - lr: 0.000488 +2021-03-26 08:31:58,611 epoch 74 - iter 24/25 - loss 2.45315273 - samples/sec: 123.38 - lr: 0.000488 +2021-03-26 08:31:59,096 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:31:59,096 EPOCH 74 done: loss 2.4754 - lr 0.0004883 +2021-03-26 08:31:59,826 DEV : loss 5.15543270111084 - score 0.9173 +2021-03-26 08:31:59,849 BAD EPOCHS (no improvement): 4 +2021-03-26 08:31:59,850 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:32:00,856 epoch 75 - iter 2/25 - loss 2.59288913 - samples/sec: 127.57 - lr: 0.000244 +2021-03-26 08:32:01,774 epoch 75 - iter 4/25 - loss 2.49805722 - samples/sec: 139.62 - lr: 0.000244 +2021-03-26 08:32:02,764 epoch 75 - iter 6/25 - loss 2.51666794 - samples/sec: 129.57 - lr: 0.000244 +2021-03-26 08:32:03,911 epoch 75 - iter 8/25 - loss 2.53042136 - samples/sec: 111.73 - lr: 0.000244 +2021-03-26 08:32:04,831 epoch 75 - iter 10/25 - loss 2.46552075 - samples/sec: 139.32 - lr: 0.000244 +2021-03-26 08:32:05,839 epoch 75 - iter 12/25 - loss 2.49688201 - samples/sec: 127.12 - lr: 0.000244 +2021-03-26 08:32:06,839 epoch 75 - iter 14/25 - loss 2.53646792 - samples/sec: 128.18 - lr: 0.000244 +2021-03-26 08:32:07,855 epoch 75 - iter 16/25 - loss 2.60008014 - samples/sec: 126.24 - lr: 0.000244 +2021-03-26 08:32:08,720 epoch 75 - iter 18/25 - loss 2.56159735 - samples/sec: 148.19 - lr: 0.000244 +2021-03-26 08:32:09,629 epoch 75 - iter 20/25 - loss 2.49216157 - samples/sec: 140.97 - lr: 0.000244 +2021-03-26 08:32:10,686 epoch 75 - iter 22/25 - loss 2.45089780 - samples/sec: 121.23 - lr: 0.000244 +2021-03-26 08:32:11,686 epoch 75 - iter 24/25 - loss 2.48957231 - samples/sec: 128.26 - lr: 0.000244 +2021-03-26 08:32:12,103 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:32:12,104 EPOCH 75 done: loss 2.4927 - lr 0.0002441 +2021-03-26 08:32:12,823 DEV : loss 5.1556715965271 - score 0.9173 +2021-03-26 08:32:12,845 BAD EPOCHS (no improvement): 1 +2021-03-26 08:32:12,846 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:32:13,823 epoch 76 - iter 2/25 - loss 2.92167127 - samples/sec: 131.27 - lr: 0.000244 +2021-03-26 08:32:14,764 epoch 76 - iter 4/25 - loss 2.67908305 - samples/sec: 136.30 - lr: 0.000244 +2021-03-26 08:32:15,679 epoch 76 - iter 6/25 - loss 2.58029282 - samples/sec: 140.02 - lr: 0.000244 +2021-03-26 08:32:16,622 epoch 76 - iter 8/25 - loss 2.72686112 - samples/sec: 135.99 - lr: 0.000244 +2021-03-26 08:32:17,684 epoch 76 - iter 10/25 - loss 2.58344152 - samples/sec: 120.68 - lr: 0.000244 +2021-03-26 08:32:18,622 epoch 76 - iter 12/25 - loss 2.59247253 - samples/sec: 136.82 - lr: 0.000244 +2021-03-26 08:32:19,542 epoch 76 - iter 14/25 - loss 2.57681600 - samples/sec: 139.36 - lr: 0.000244 +2021-03-26 08:32:20,608 epoch 76 - iter 16/25 - loss 2.54443355 - samples/sec: 120.23 - lr: 0.000244 +2021-03-26 08:32:21,703 epoch 76 - iter 18/25 - loss 2.52020005 - samples/sec: 116.98 - lr: 0.000244 +2021-03-26 08:32:22,619 epoch 76 - iter 20/25 - loss 2.50298864 - samples/sec: 140.05 - lr: 0.000244 +2021-03-26 08:32:23,586 epoch 76 - iter 22/25 - loss 2.49885844 - samples/sec: 132.50 - lr: 0.000244 +2021-03-26 08:32:24,474 epoch 76 - iter 24/25 - loss 2.48496979 - samples/sec: 144.49 - lr: 0.000244 +2021-03-26 08:32:24,923 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:32:24,924 EPOCH 76 done: loss 2.5162 - lr 0.0002441 +2021-03-26 08:32:25,615 DEV : loss 5.155824661254883 - score 0.9177 +2021-03-26 08:32:25,638 BAD EPOCHS (no improvement): 2 +2021-03-26 08:32:25,639 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:32:26,561 epoch 77 - iter 2/25 - loss 2.35044253 - samples/sec: 138.98 - lr: 0.000244 +2021-03-26 08:32:27,486 epoch 77 - iter 4/25 - loss 2.85872233 - samples/sec: 138.68 - lr: 0.000244 +2021-03-26 08:32:28,417 epoch 77 - iter 6/25 - loss 2.92169376 - samples/sec: 137.62 - lr: 0.000244 +2021-03-26 08:32:29,416 epoch 77 - iter 8/25 - loss 2.81242767 - samples/sec: 128.31 - lr: 0.000244 +2021-03-26 08:32:30,435 epoch 77 - iter 10/25 - loss 2.79586065 - samples/sec: 125.78 - lr: 0.000244 +2021-03-26 08:32:31,461 epoch 77 - iter 12/25 - loss 2.73711844 - samples/sec: 125.00 - lr: 0.000244 +2021-03-26 08:32:32,630 epoch 77 - iter 14/25 - loss 2.69382988 - samples/sec: 109.67 - lr: 0.000244 +2021-03-26 08:32:33,805 epoch 77 - iter 16/25 - loss 2.66051930 - samples/sec: 109.02 - lr: 0.000244 +2021-03-26 08:32:34,804 epoch 77 - iter 18/25 - loss 2.65610683 - samples/sec: 128.29 - lr: 0.000244 +2021-03-26 08:32:35,816 epoch 77 - iter 20/25 - loss 2.59753153 - samples/sec: 126.66 - lr: 0.000244 +2021-03-26 08:32:36,754 epoch 77 - iter 22/25 - loss 2.58311897 - samples/sec: 136.66 - lr: 0.000244 +2021-03-26 08:32:37,745 epoch 77 - iter 24/25 - loss 2.57920326 - samples/sec: 129.38 - lr: 0.000244 +2021-03-26 08:32:38,188 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:32:38,189 EPOCH 77 done: loss 2.5370 - lr 0.0002441 +2021-03-26 08:32:38,917 DEV : loss 5.156017780303955 - score 0.9175 +2021-03-26 08:32:38,940 BAD EPOCHS (no improvement): 3 +2021-03-26 08:32:38,941 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:32:39,892 epoch 78 - iter 2/25 - loss 2.84578502 - samples/sec: 134.80 - lr: 0.000244 +2021-03-26 08:32:40,911 epoch 78 - iter 4/25 - loss 2.86701858 - samples/sec: 125.90 - lr: 0.000244 +2021-03-26 08:32:41,923 epoch 78 - iter 6/25 - loss 2.53632077 - samples/sec: 126.58 - lr: 0.000244 +2021-03-26 08:32:42,896 epoch 78 - iter 8/25 - loss 2.59809896 - samples/sec: 131.74 - lr: 0.000244 +2021-03-26 08:32:43,830 epoch 78 - iter 10/25 - loss 2.44800783 - samples/sec: 137.33 - lr: 0.000244 +2021-03-26 08:32:44,759 epoch 78 - iter 12/25 - loss 2.47062728 - samples/sec: 137.99 - lr: 0.000244 +2021-03-26 08:32:45,724 epoch 78 - iter 14/25 - loss 2.50483849 - samples/sec: 132.84 - lr: 0.000244 +2021-03-26 08:32:46,797 epoch 78 - iter 16/25 - loss 2.52945174 - samples/sec: 119.51 - lr: 0.000244 +2021-03-26 08:32:47,790 epoch 78 - iter 18/25 - loss 2.58047466 - samples/sec: 128.96 - lr: 0.000244 +2021-03-26 08:32:48,698 epoch 78 - iter 20/25 - loss 2.59792646 - samples/sec: 141.18 - lr: 0.000244 +2021-03-26 08:32:49,749 epoch 78 - iter 22/25 - loss 2.58495852 - samples/sec: 121.97 - lr: 0.000244 +2021-03-26 08:32:50,717 epoch 78 - iter 24/25 - loss 2.54879833 - samples/sec: 132.52 - lr: 0.000244 +2021-03-26 08:32:51,194 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:32:51,194 EPOCH 78 done: loss 2.5141 - lr 0.0002441 +2021-03-26 08:32:51,938 DEV : loss 5.1558074951171875 - score 0.9175 +2021-03-26 08:32:51,969 BAD EPOCHS (no improvement): 4 +2021-03-26 08:32:51,970 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:32:52,992 epoch 79 - iter 2/25 - loss 2.72548425 - samples/sec: 125.42 - lr: 0.000122 +2021-03-26 08:32:53,943 epoch 79 - iter 4/25 - loss 2.43002558 - samples/sec: 134.78 - lr: 0.000122 +2021-03-26 08:32:54,991 epoch 79 - iter 6/25 - loss 2.39026821 - samples/sec: 122.34 - lr: 0.000122 +2021-03-26 08:32:55,965 epoch 79 - iter 8/25 - loss 2.45706752 - samples/sec: 131.58 - lr: 0.000122 +2021-03-26 08:32:56,886 epoch 79 - iter 10/25 - loss 2.42801330 - samples/sec: 139.07 - lr: 0.000122 +2021-03-26 08:32:57,826 epoch 79 - iter 12/25 - loss 2.43927479 - samples/sec: 136.48 - lr: 0.000122 +2021-03-26 08:32:58,830 epoch 79 - iter 14/25 - loss 2.41582305 - samples/sec: 127.68 - lr: 0.000122 +2021-03-26 08:32:59,895 epoch 79 - iter 16/25 - loss 2.52669768 - samples/sec: 120.32 - lr: 0.000122 +2021-03-26 08:33:00,825 epoch 79 - iter 18/25 - loss 2.50340940 - samples/sec: 137.90 - lr: 0.000122 +2021-03-26 08:33:01,861 epoch 79 - iter 20/25 - loss 2.48553985 - samples/sec: 123.78 - lr: 0.000122 +2021-03-26 08:33:02,826 epoch 79 - iter 22/25 - loss 2.50086822 - samples/sec: 132.91 - lr: 0.000122 +2021-03-26 08:33:03,888 epoch 79 - iter 24/25 - loss 2.50031297 - samples/sec: 120.70 - lr: 0.000122 +2021-03-26 08:33:04,279 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:33:04,280 EPOCH 79 done: loss 2.4812 - lr 0.0001221 +2021-03-26 08:33:05,033 DEV : loss 5.15554141998291 - score 0.9175 +2021-03-26 08:33:05,065 BAD EPOCHS (no improvement): 1 +2021-03-26 08:33:05,065 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:33:06,129 epoch 80 - iter 2/25 - loss 1.92892826 - samples/sec: 120.59 - lr: 0.000122 +2021-03-26 08:33:07,054 epoch 80 - iter 4/25 - loss 1.89410821 - samples/sec: 138.55 - lr: 0.000122 +2021-03-26 08:33:08,094 epoch 80 - iter 6/25 - loss 2.19573849 - samples/sec: 123.29 - lr: 0.000122 +2021-03-26 08:33:09,128 epoch 80 - iter 8/25 - loss 2.34732698 - samples/sec: 123.90 - lr: 0.000122 +2021-03-26 08:33:10,045 epoch 80 - iter 10/25 - loss 2.49000527 - samples/sec: 139.87 - lr: 0.000122 +2021-03-26 08:33:11,079 epoch 80 - iter 12/25 - loss 2.46932315 - samples/sec: 123.93 - lr: 0.000122 +2021-03-26 08:33:12,139 epoch 80 - iter 14/25 - loss 2.45208756 - samples/sec: 120.96 - lr: 0.000122 +2021-03-26 08:33:13,168 epoch 80 - iter 16/25 - loss 2.46849288 - samples/sec: 124.61 - lr: 0.000122 +2021-03-26 08:33:14,237 epoch 80 - iter 18/25 - loss 2.46822666 - samples/sec: 119.92 - lr: 0.000122 +2021-03-26 08:33:15,231 epoch 80 - iter 20/25 - loss 2.48660938 - samples/sec: 128.99 - lr: 0.000122 +2021-03-26 08:33:16,195 epoch 80 - iter 22/25 - loss 2.49127837 - samples/sec: 132.96 - lr: 0.000122 +2021-03-26 08:33:17,218 epoch 80 - iter 24/25 - loss 2.49067168 - samples/sec: 125.22 - lr: 0.000122 +2021-03-26 08:33:17,654 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:33:17,654 EPOCH 80 done: loss 2.4916 - lr 0.0001221 +2021-03-26 08:33:18,378 DEV : loss 5.155505180358887 - score 0.9175 +2021-03-26 08:33:18,394 BAD EPOCHS (no improvement): 2 +2021-03-26 08:33:18,395 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:33:19,480 epoch 81 - iter 2/25 - loss 2.51575708 - samples/sec: 118.07 - lr: 0.000122 +2021-03-26 08:33:20,596 epoch 81 - iter 4/25 - loss 2.59854716 - samples/sec: 114.84 - lr: 0.000122 +2021-03-26 08:33:21,514 epoch 81 - iter 6/25 - loss 2.59062302 - samples/sec: 139.63 - lr: 0.000122 +2021-03-26 08:33:22,572 epoch 81 - iter 8/25 - loss 2.50448807 - samples/sec: 121.15 - lr: 0.000122 +2021-03-26 08:33:23,767 epoch 81 - iter 10/25 - loss 2.54231604 - samples/sec: 107.28 - lr: 0.000122 +2021-03-26 08:33:24,618 epoch 81 - iter 12/25 - loss 2.47345619 - samples/sec: 150.66 - lr: 0.000122 +2021-03-26 08:33:25,531 epoch 81 - iter 14/25 - loss 2.43137515 - samples/sec: 140.44 - lr: 0.000122 +2021-03-26 08:33:26,489 epoch 81 - iter 16/25 - loss 2.44498353 - samples/sec: 133.80 - lr: 0.000122 +2021-03-26 08:33:27,389 epoch 81 - iter 18/25 - loss 2.42864709 - samples/sec: 142.35 - lr: 0.000122 +2021-03-26 08:33:28,390 epoch 81 - iter 20/25 - loss 2.48083928 - samples/sec: 128.02 - lr: 0.000122 +2021-03-26 08:33:29,390 epoch 81 - iter 22/25 - loss 2.47235147 - samples/sec: 128.20 - lr: 0.000122 +2021-03-26 08:33:30,381 epoch 81 - iter 24/25 - loss 2.49743044 - samples/sec: 129.25 - lr: 0.000122 +2021-03-26 08:33:30,811 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:33:30,812 EPOCH 81 done: loss 2.4886 - lr 0.0001221 +2021-03-26 08:33:31,536 DEV : loss 5.15550422668457 - score 0.9175 +2021-03-26 08:33:31,559 BAD EPOCHS (no improvement): 3 +2021-03-26 08:33:31,559 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:33:32,573 epoch 82 - iter 2/25 - loss 2.69717109 - samples/sec: 126.55 - lr: 0.000122 +2021-03-26 08:33:33,590 epoch 82 - iter 4/25 - loss 2.38267857 - samples/sec: 126.02 - lr: 0.000122 +2021-03-26 08:33:34,533 epoch 82 - iter 6/25 - loss 2.37339608 - samples/sec: 136.00 - lr: 0.000122 +2021-03-26 08:33:35,502 epoch 82 - iter 8/25 - loss 2.46923965 - samples/sec: 132.37 - lr: 0.000122 +2021-03-26 08:33:36,515 epoch 82 - iter 10/25 - loss 2.51084020 - samples/sec: 126.59 - lr: 0.000122 +2021-03-26 08:33:37,555 epoch 82 - iter 12/25 - loss 2.60109609 - samples/sec: 123.18 - lr: 0.000122 +2021-03-26 08:33:38,522 epoch 82 - iter 14/25 - loss 2.60343652 - samples/sec: 132.67 - lr: 0.000122 +2021-03-26 08:33:39,516 epoch 82 - iter 16/25 - loss 2.63582183 - samples/sec: 128.88 - lr: 0.000122 +2021-03-26 08:33:40,493 epoch 82 - iter 18/25 - loss 2.60362657 - samples/sec: 131.31 - lr: 0.000122 +2021-03-26 08:33:41,508 epoch 82 - iter 20/25 - loss 2.61728383 - samples/sec: 126.16 - lr: 0.000122 +2021-03-26 08:33:42,475 epoch 82 - iter 22/25 - loss 2.59926463 - samples/sec: 132.56 - lr: 0.000122 +2021-03-26 08:33:43,415 epoch 82 - iter 24/25 - loss 2.59879770 - samples/sec: 136.37 - lr: 0.000122 +2021-03-26 08:33:43,790 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:33:43,790 EPOCH 82 done: loss 2.5751 - lr 0.0001221 +2021-03-26 08:33:44,512 DEV : loss 5.1555328369140625 - score 0.9175 +2021-03-26 08:33:44,534 BAD EPOCHS (no improvement): 4 +2021-03-26 08:33:44,535 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:33:44,536 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:33:44,536 learning rate too small - quitting training! +2021-03-26 08:33:44,536 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:33:53,804 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:33:53,804 Testing using best model ... +2021-03-26 08:33:53,805 loading file /home/tmp/megahedm/models/multipos/multipos_UDMADAR_4Diale-LEV_EGY_GLF_MGR__fasttext_flairbwfw__64__0.5_202103260811/best-model.pt +2021-03-26 08:34:00,860 0.9004 +2021-03-26 08:34:00,861 +Results: +- F-score (micro): 0.897 +- F-score (macro): 0.5285 +- Accuracy (incl. no class): 0.9004 + +By class: + precision recall f1-score support + + PRON 0.9706 0.9706 0.9706 204 + NOUN 0.8985 0.9356 0.9167 388 + ADJ 0.8649 0.8136 0.8384 118 + SCONJ 0.8649 1.0000 0.9275 32 + VERB 0.9259 0.9191 0.9225 136 + DET 0.9167 0.9167 0.9167 48 + CCONJ 0.9697 0.8889 0.9275 72 + ADP 0.9541 0.9811 0.9674 106 + ADV 0.8739 0.8509 0.8622 114 + PROPN 0.7619 0.6957 0.7273 23 + NUM 0.8571 0.7059 0.7742 17 + PART 0.9508 0.9775 0.9640 178 + PUNCT 1.0000 1.0000 1.0000 32 + AUX 0.9459 0.9459 0.9459 37 + INTJ 0.9583 0.9583 0.9583 24 + X 1.0000 0.0000 0.0000 1 + HASH 1.0000 1.0000 1.0000 12 + V+PRON 0.8776 0.8113 0.8431 53 + V+PRON+PRON 0.5000 0.5556 0.5263 9 + PREP+PRON 0.8333 0.9615 0.8929 26 + DET+NOUN+NSUFF 0.8750 0.9130 0.8936 23 + PREP 0.8889 0.9552 0.9209 67 + CONJ+PART 0.9000 0.9000 0.9000 10 + V 0.9481 0.8902 0.9182 82 + CONJ+V+PRON 0.5000 0.6250 0.5556 8 + EOS 1.0000 1.0000 1.0000 70 + NOUN+NSUFF 0.8571 0.9231 0.8889 39 + PREP+NOUN+PRON 0.3333 0.5000 0.4000 4 + PUNC 1.0000 1.0000 1.0000 152 + NOUN+PRON 0.7971 0.8209 0.8088 67 + V+PRON+PREP+PRON 0.5000 1.0000 0.6667 1 + NOUN+NSUFF+PRON 0.6500 0.7222 0.6842 18 + CONJ 1.0000 0.8438 0.9153 32 + FOREIGN 1.0000 1.0000 1.0000 3 + PROG_PART+V 0.8235 0.9333 0.8750 30 + PREP+NOUN 0.9048 0.9500 0.9268 20 + PART+PRON 0.9474 0.9000 0.9231 20 + PREP+NOUN+NSUFF 0.6667 0.4000 0.5000 5 + PROG_PART+V+PRON 0.8571 0.7826 0.8182 23 + DET+NOUN 0.9103 0.9595 0.9342 74 + DET+ADJ+NSUFF 0.6000 0.7500 0.6667 4 + CONJ+DET+NOUN 1.0000 0.5000 0.6667 4 + FUT_PART+V+PRON 1.0000 0.7500 0.8571 4 + DET+ADJ 0.9091 0.5882 0.7143 17 + NOUN+CASE+PRON 0.0000 1.0000 0.0000 0 + ADJ+NSUFF 0.7879 0.8966 0.8387 29 + CONJ+NOUN+PRON 0.6667 0.7500 0.7059 8 + CONJ+NOUN+NSUFF+PRON 0.0000 0.0000 0.0000 1 + CONJ+V 0.6000 0.6000 0.6000 5 + CONJ+PREP+PRON 1.0000 0.0000 0.0000 1 + CONJ+NOUN 0.7778 1.0000 0.8750 7 + CONJ+DET+ADJ 0.0000 0.0000 0.0000 1 + CONJ+PREP+DET+NOUN 0.5000 1.0000 0.6667 1 + EMOT 1.0000 1.0000 1.0000 11 + MENTION 1.0000 1.0000 1.0000 16 + CONJ+FUT_PART+V 1.0000 0.0000 0.0000 2 + CONJ+PROG_PART+V 0.0000 1.0000 0.0000 0 + CONJ+NOUN+NSUFF 1.0000 0.5000 0.6667 2 + FUT_PART+V 0.8000 0.8000 0.8000 15 + PRON+DET+NOUN 1.0000 0.0000 0.0000 3 + PREP+DET+NOUN 0.8889 1.0000 0.9412 8 + CONJ+PART+NOUN 1.0000 0.0000 0.0000 1 + V+PREP+PRON 0.3333 0.2500 0.2857 4 + ADJ+CASE 1.0000 0.3333 0.5000 3 + PREP+NOUN+NSUFF+PRON 0.0000 0.0000 0.0000 1 + CONJ+PRON 1.0000 0.6667 0.8000 3 + ADJ+PRON 0.0000 0.0000 0.0000 5 + CONJ+DET+ADJ+NSUFF 1.0000 0.0000 0.0000 1 + CONJ+DET+NOUN+NSUFF 0.0000 0.0000 0.0000 1 + URL 1.0000 1.0000 1.0000 3 + PRON+NOUN 1.0000 0.0000 0.0000 1 + NUM+NSUFF 1.0000 0.0000 0.0000 3 + NOUN+NSUFF+NSUFF 0.0000 1.0000 0.0000 0 + PREP+PART+PRON 1.0000 1.0000 1.0000 2 + NOUN+PREP+PRON 1.0000 0.0000 0.0000 1 +PART+PROG_PART+V+PREP+PRON+NEG_PART 0.0000 1.0000 0.0000 0 + ADV+PRON 1.0000 0.0000 0.0000 1 + FUT_PART+V+PREP+PRON 0.0000 1.0000 0.0000 0 + PREP+ADJ+NSUFF 1.0000 0.0000 0.0000 2 + NOUN+CASE 0.5000 0.7500 0.6000 4 + CONJ+PREP 1.0000 1.0000 1.0000 1 + PART+V+PRON+PRON 1.0000 0.0000 0.0000 1 + PART+V+PRON 0.0000 0.0000 0.0000 1 + PREP+PRON+DET+NOUN+NSUFF 1.0000 0.0000 0.0000 1 + PRON+DET+NOUN+NSUFF 0.0000 1.0000 0.0000 0 + PREP+DET 1.0000 0.0000 0.0000 1 + PART+V 0.5000 1.0000 0.6667 1 + FUT_PART+V+PRON+PRON 1.0000 0.5000 0.6667 2 + PART+FUT_PART 1.0000 0.0000 0.0000 1 + PROG_PART+V+PRON+PRON 0.0000 1.0000 0.0000 0 + CONJ+FUT_PART+V+PREP+PRON 1.0000 0.0000 0.0000 1 + CONJ+PROG_PART+V+PRON 0.5000 1.0000 0.6667 1 + PART+PREP 1.0000 1.0000 1.0000 1 + FUT_PART 1.0000 1.0000 1.0000 1 + CONJ+PART+V+PRON 1.0000 0.0000 0.0000 2 + CONJ+PART+PRON 1.0000 0.0000 0.0000 1 + CONJ+ADV 0.0000 1.0000 0.0000 0 + CONJ+V+PRON+PREP+PRON 1.0000 0.0000 0.0000 1 + CONJ+PREP+DET+NOUN+NSUFF 0.0000 1.0000 0.0000 0 + PART+PROG_PART+V+NEG_PART 1.0000 1.0000 1.0000 1 + PART+NOUN 0.6000 0.7500 0.6667 4 + CONJ+PREP+PART 1.0000 0.0000 0.0000 1 + CONJ+V+PREP+PRON 1.0000 0.0000 0.0000 1 + PART+PREP+NEG_PART 1.0000 1.0000 1.0000 2 + PART+V+NEG_PART 0.3333 0.6667 0.4444 3 + PART+V+PRON+NEG_PART 0.7500 0.4286 0.5455 7 + PART+NOUN+NEG_PART 1.0000 1.0000 1.0000 1 + PREP+DET+ADJ 1.0000 0.0000 0.0000 1 + CONJ+ADJ+NSUFF 1.0000 0.0000 0.0000 1 + CONJ+PART+V+PRON+NEG_PART 0.0000 1.0000 0.0000 0 + CONJ+ADJ 1.0000 1.0000 1.0000 1 + ADV+NSUFF 1.0000 1.0000 1.0000 2 + PART+V+PRON+PRON+NEG_PART 1.0000 0.0000 0.0000 1 + PART+NOUN+NSUFF 1.0000 0.0000 0.0000 1 + + micro avg 0.8971 0.8968 0.8970 2597 + macro avg 0.7573 0.6262 0.5285 2597 + weighted avg 0.9056 0.8968 0.8931 2597 + +2021-03-26 08:34:00,862 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:34:00,862 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:34:06,156 Reading data from ../../Datasets_adhoc/CSCS_corpus-GUC +2021-03-26 08:34:06,157 Train: ../../Datasets_adhoc/CSCS_corpus-GUC/all_participants.conllu +2021-03-26 08:34:06,157 Dev: None +2021-03-26 08:34:06,158 Test: None +2021-03-26 08:34:07,496 Reading data from ../../Datasets_adhoc/UD_MADAR +2021-03-26 08:34:07,496 Train: ../../Datasets_adhoc/UD_MADAR/ajp_madar-ud-test-edit.conllu +2021-03-26 08:34:07,497 Dev: None +2021-03-26 08:34:07,497 Test: None +2021-03-26 08:34:07,544 Reading data from ../../Datasets_adhoc/dialectal_arabic_resources-master +2021-03-26 08:34:07,545 Train: ../../Datasets_adhoc/dialectal_arabic_resources-master/seg_plus_pos_lev.txt +2021-03-26 08:34:07,546 Dev: None +2021-03-26 08:34:07,546 Test: None +2021-03-26 08:34:07,717 Reading data from ../../Datasets_adhoc/dialectal_arabic_resources-master +2021-03-26 08:34:07,718 Train: ../../Datasets_adhoc/dialectal_arabic_resources-master/seg_plus_pos_egy.txt +2021-03-26 08:34:07,718 Dev: None +2021-03-26 08:34:07,719 Test: None +2021-03-26 08:34:07,889 Reading data from ../../Datasets_adhoc/dialectal_arabic_resources-master +2021-03-26 08:34:07,890 Train: ../../Datasets_adhoc/dialectal_arabic_resources-master/seg_plus_pos_glf.txt +2021-03-26 08:34:07,890 Dev: None +2021-03-26 08:34:07,890 Test: None +2021-03-26 08:34:08,033 Reading data from ../../Datasets_adhoc/dialectal_arabic_resources-master +2021-03-26 08:34:08,034 Train: ../../Datasets_adhoc/dialectal_arabic_resources-master/seg_plus_pos_mgr.txt +2021-03-26 08:34:08,034 Dev: None +2021-03-26 08:34:08,034 Test: None +2021-03-26 08:34:08,185 Filtering long sentences +2021-03-26 08:34:08,214 MultiCorpus: 1575 train + 175 dev + 194 test sentences + - ColumnCorpus Corpus: 934 train + 104 dev + 115 test sentences + - ColumnCorpus Corpus: 81 train + 9 dev + 10 test sentences + - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences + - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences +2021-03-26 08:34:08,592 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:34:08,593 Model: "SequenceTagger( + (embeddings): StackedEmbeddings( + (list_embedding_0): WordEmbeddings('ar') + (list_embedding_1): FlairEmbeddings( + (lm): LanguageModel( + (drop): Dropout(p=0.1, inplace=False) + (encoder): Embedding(7125, 100) + (rnn): LSTM(100, 2048) + (decoder): Linear(in_features=2048, out_features=7125, bias=True) + ) + ) + (list_embedding_2): FlairEmbeddings( + (lm): LanguageModel( + (drop): Dropout(p=0.1, inplace=False) + (encoder): Embedding(7125, 100) + (rnn): LSTM(100, 2048) + (decoder): Linear(in_features=2048, out_features=7125, bias=True) + ) + ) + ) + (word_dropout): WordDropout(p=0.05) + (locked_dropout): LockedDropout(p=0.5) + (embedding2nn): Linear(in_features=4396, out_features=4396, bias=True) + (rnn): LSTM(4396, 256, batch_first=True, bidirectional=True) + (linear): Linear(in_features=512, out_features=206, bias=True) + (beta): 1.0 + (weights): None + (weight_tensor) None +)" +2021-03-26 08:34:08,593 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:34:08,594 Corpus: "MultiCorpus: 1575 train + 175 dev + 194 test sentences + - ColumnCorpus Corpus: 934 train + 104 dev + 115 test sentences + - ColumnCorpus Corpus: 81 train + 9 dev + 10 test sentences + - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences + - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences" +2021-03-26 08:34:08,594 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:34:08,594 Parameters: +2021-03-26 08:34:08,594 - learning_rate: "0.5" +2021-03-26 08:34:08,594 - mini_batch_size: "64" +2021-03-26 08:34:08,594 - patience: "3" +2021-03-26 08:34:08,595 - anneal_factor: "0.5" +2021-03-26 08:34:08,595 - max_epochs: "150" +2021-03-26 08:34:08,595 - shuffle: "True" +2021-03-26 08:34:08,595 - train_with_dev: "False" +2021-03-26 08:34:08,595 - batch_growth_annealing: "False" +2021-03-26 08:34:08,596 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:34:08,596 Model training base path: "/home/tmp/megahedm/models/multipos/multipos_UDMADAR_4Diale-LEV_EGY_GLF_MGR__fasttext_flairbwfw__64__0.5_202103260834" +2021-03-26 08:34:08,596 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:34:08,596 Device: cuda:0 +2021-03-26 08:34:08,597 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:34:08,597 Embeddings storage mode: cpu +2021-03-26 08:34:08,598 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:34:10,119 epoch 1 - iter 2/25 - loss 72.56362534 - samples/sec: 84.26 - lr: 0.500000 +2021-03-26 08:34:11,473 epoch 1 - iter 4/25 - loss 79.34148407 - samples/sec: 94.93 - lr: 0.500000 +2021-03-26 08:34:12,754 epoch 1 - iter 6/25 - loss 74.84145292 - samples/sec: 100.03 - lr: 0.500000 +2021-03-26 08:34:14,201 epoch 1 - iter 8/25 - loss 72.96682358 - samples/sec: 88.58 - lr: 0.500000 +2021-03-26 08:34:15,689 epoch 1 - iter 10/25 - loss 72.07293091 - samples/sec: 86.08 - lr: 0.500000 +2021-03-26 08:34:16,909 epoch 1 - iter 12/25 - loss 68.28134028 - samples/sec: 105.00 - lr: 0.500000 +2021-03-26 08:34:18,230 epoch 1 - iter 14/25 - loss 67.68329539 - samples/sec: 97.04 - lr: 0.500000 +2021-03-26 08:34:19,587 epoch 1 - iter 16/25 - loss 66.21412325 - samples/sec: 94.45 - lr: 0.500000 +2021-03-26 08:34:20,906 epoch 1 - iter 18/25 - loss 64.04796452 - samples/sec: 97.11 - lr: 0.500000 +2021-03-26 08:34:22,192 epoch 1 - iter 20/25 - loss 63.26277981 - samples/sec: 99.69 - lr: 0.500000 +2021-03-26 08:34:23,553 epoch 1 - iter 22/25 - loss 62.30557043 - samples/sec: 94.07 - lr: 0.500000 +2021-03-26 08:34:24,854 epoch 1 - iter 24/25 - loss 60.86900187 - samples/sec: 98.54 - lr: 0.500000 +2021-03-26 08:34:25,383 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:34:25,383 EPOCH 1 done: loss 60.2458 - lr 0.5000000 +2021-03-26 08:34:26,609 DEV : loss 47.09407043457031 - score 0.305 +2021-03-26 08:34:26,632 BAD EPOCHS (no improvement): 0 +2021-03-26 08:34:36,005 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:34:37,029 epoch 2 - iter 2/25 - loss 45.07046700 - samples/sec: 125.27 - lr: 0.500000 +2021-03-26 08:34:38,022 epoch 2 - iter 4/25 - loss 44.52553463 - samples/sec: 129.17 - lr: 0.500000 +2021-03-26 08:34:38,979 epoch 2 - iter 6/25 - loss 42.01777840 - samples/sec: 134.09 - lr: 0.500000 +2021-03-26 08:34:39,908 epoch 2 - iter 8/25 - loss 40.97041130 - samples/sec: 138.00 - lr: 0.500000 +2021-03-26 08:34:40,924 epoch 2 - iter 10/25 - loss 39.93429756 - samples/sec: 126.17 - lr: 0.500000 +2021-03-26 08:34:42,012 epoch 2 - iter 12/25 - loss 38.79689900 - samples/sec: 117.86 - lr: 0.500000 +2021-03-26 08:34:42,960 epoch 2 - iter 14/25 - loss 38.76790496 - samples/sec: 135.23 - lr: 0.500000 +2021-03-26 08:34:43,991 epoch 2 - iter 16/25 - loss 38.38748801 - samples/sec: 124.30 - lr: 0.500000 +2021-03-26 08:34:45,022 epoch 2 - iter 18/25 - loss 37.90621662 - samples/sec: 124.28 - lr: 0.500000 +2021-03-26 08:34:46,056 epoch 2 - iter 20/25 - loss 37.44555807 - samples/sec: 124.01 - lr: 0.500000 +2021-03-26 08:34:46,991 epoch 2 - iter 22/25 - loss 36.58613716 - samples/sec: 137.09 - lr: 0.500000 +2021-03-26 08:34:48,099 epoch 2 - iter 24/25 - loss 35.93759561 - samples/sec: 115.56 - lr: 0.500000 +2021-03-26 08:34:48,504 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:34:48,504 EPOCH 2 done: loss 35.7602 - lr 0.5000000 +2021-03-26 08:34:49,276 DEV : loss 32.68856430053711 - score 0.5186 +2021-03-26 08:34:49,294 BAD EPOCHS (no improvement): 0 +2021-03-26 08:34:58,804 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:34:59,920 epoch 3 - iter 2/25 - loss 30.71884441 - samples/sec: 114.93 - lr: 0.500000 +2021-03-26 08:35:00,892 epoch 3 - iter 4/25 - loss 31.72236824 - samples/sec: 132.05 - lr: 0.500000 +2021-03-26 08:35:01,891 epoch 3 - iter 6/25 - loss 29.34960175 - samples/sec: 128.28 - lr: 0.500000 +2021-03-26 08:35:02,983 epoch 3 - iter 8/25 - loss 28.83567262 - samples/sec: 117.29 - lr: 0.500000 +2021-03-26 08:35:03,978 epoch 3 - iter 10/25 - loss 28.65415955 - samples/sec: 128.78 - lr: 0.500000 +2021-03-26 08:35:05,025 epoch 3 - iter 12/25 - loss 28.65188265 - samples/sec: 122.42 - lr: 0.500000 +2021-03-26 08:35:06,054 epoch 3 - iter 14/25 - loss 28.25057289 - samples/sec: 124.63 - lr: 0.500000 +2021-03-26 08:35:06,949 epoch 3 - iter 16/25 - loss 27.42237711 - samples/sec: 143.32 - lr: 0.500000 +2021-03-26 08:35:07,927 epoch 3 - iter 18/25 - loss 26.69088671 - samples/sec: 131.10 - lr: 0.500000 +2021-03-26 08:35:08,866 epoch 3 - iter 20/25 - loss 26.34798946 - samples/sec: 136.50 - lr: 0.500000 +2021-03-26 08:35:09,798 epoch 3 - iter 22/25 - loss 25.97796076 - samples/sec: 137.58 - lr: 0.500000 +2021-03-26 08:35:10,824 epoch 3 - iter 24/25 - loss 25.55988129 - samples/sec: 124.83 - lr: 0.500000 +2021-03-26 08:35:11,192 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:35:11,192 EPOCH 3 done: loss 25.4119 - lr 0.5000000 +2021-03-26 08:35:11,916 DEV : loss 19.450502395629883 - score 0.6724 +2021-03-26 08:35:11,940 BAD EPOCHS (no improvement): 0 +2021-03-26 08:35:21,664 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:35:22,579 epoch 4 - iter 2/25 - loss 20.26455975 - samples/sec: 140.22 - lr: 0.500000 +2021-03-26 08:35:23,488 epoch 4 - iter 4/25 - loss 20.28417206 - samples/sec: 141.15 - lr: 0.500000 +2021-03-26 08:35:24,512 epoch 4 - iter 6/25 - loss 20.85648982 - samples/sec: 125.22 - lr: 0.500000 +2021-03-26 08:35:25,561 epoch 4 - iter 8/25 - loss 20.71336842 - samples/sec: 122.19 - lr: 0.500000 +2021-03-26 08:35:26,627 epoch 4 - iter 10/25 - loss 21.16067410 - samples/sec: 120.26 - lr: 0.500000 +2021-03-26 08:35:27,621 epoch 4 - iter 12/25 - loss 21.02786001 - samples/sec: 128.99 - lr: 0.500000 +2021-03-26 08:35:28,671 epoch 4 - iter 14/25 - loss 20.83588055 - samples/sec: 122.10 - lr: 0.500000 +2021-03-26 08:35:29,687 epoch 4 - iter 16/25 - loss 20.59734356 - samples/sec: 126.18 - lr: 0.500000 +2021-03-26 08:35:30,603 epoch 4 - iter 18/25 - loss 20.37963645 - samples/sec: 140.06 - lr: 0.500000 +2021-03-26 08:35:31,538 epoch 4 - iter 20/25 - loss 19.81514883 - samples/sec: 137.05 - lr: 0.500000 +2021-03-26 08:35:32,553 epoch 4 - iter 22/25 - loss 19.77684385 - samples/sec: 126.27 - lr: 0.500000 +2021-03-26 08:35:33,630 epoch 4 - iter 24/25 - loss 19.64338191 - samples/sec: 119.09 - lr: 0.500000 +2021-03-26 08:35:34,042 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:35:34,042 EPOCH 4 done: loss 19.6755 - lr 0.5000000 +2021-03-26 08:35:34,796 DEV : loss 14.270602226257324 - score 0.7716 +2021-03-26 08:35:34,818 BAD EPOCHS (no improvement): 0 +2021-03-26 08:35:44,420 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:35:45,530 epoch 5 - iter 2/25 - loss 15.16248655 - samples/sec: 115.62 - lr: 0.500000 +2021-03-26 08:35:46,490 epoch 5 - iter 4/25 - loss 15.24637246 - samples/sec: 133.58 - lr: 0.500000 +2021-03-26 08:35:47,652 epoch 5 - iter 6/25 - loss 16.62559589 - samples/sec: 110.23 - lr: 0.500000 +2021-03-26 08:35:48,738 epoch 5 - iter 8/25 - loss 17.10516322 - samples/sec: 118.15 - lr: 0.500000 +2021-03-26 08:35:49,918 epoch 5 - iter 10/25 - loss 17.65315447 - samples/sec: 108.55 - lr: 0.500000 +2021-03-26 08:35:51,103 epoch 5 - iter 12/25 - loss 17.28618590 - samples/sec: 108.15 - lr: 0.500000 +2021-03-26 08:35:51,982 epoch 5 - iter 14/25 - loss 17.00687572 - samples/sec: 145.92 - lr: 0.500000 +2021-03-26 08:35:52,910 epoch 5 - iter 16/25 - loss 16.78228796 - samples/sec: 138.14 - lr: 0.500000 +2021-03-26 08:35:53,891 epoch 5 - iter 18/25 - loss 16.75672356 - samples/sec: 130.69 - lr: 0.500000 +2021-03-26 08:35:54,872 epoch 5 - iter 20/25 - loss 16.76150398 - samples/sec: 130.61 - lr: 0.500000 +2021-03-26 08:35:55,883 epoch 5 - iter 22/25 - loss 16.63435515 - samples/sec: 126.81 - lr: 0.500000 +2021-03-26 08:35:56,836 epoch 5 - iter 24/25 - loss 16.43824724 - samples/sec: 134.48 - lr: 0.500000 +2021-03-26 08:35:57,308 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:35:57,309 EPOCH 5 done: loss 16.2580 - lr 0.5000000 +2021-03-26 08:35:58,069 DEV : loss 12.45005989074707 - score 0.7882 +2021-03-26 08:35:58,091 BAD EPOCHS (no improvement): 0 +2021-03-26 08:36:07,764 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:36:08,767 epoch 6 - iter 2/25 - loss 14.79663944 - samples/sec: 127.90 - lr: 0.500000 +2021-03-26 08:36:09,712 epoch 6 - iter 4/25 - loss 14.06922007 - samples/sec: 135.66 - lr: 0.500000 +2021-03-26 08:36:10,731 epoch 6 - iter 6/25 - loss 13.96056747 - samples/sec: 125.81 - lr: 0.500000 +2021-03-26 08:36:11,665 epoch 6 - iter 8/25 - loss 14.25202882 - samples/sec: 137.14 - lr: 0.500000 +2021-03-26 08:36:12,618 epoch 6 - iter 10/25 - loss 14.02916756 - samples/sec: 134.54 - lr: 0.500000 +2021-03-26 08:36:13,884 epoch 6 - iter 12/25 - loss 13.93397903 - samples/sec: 101.31 - lr: 0.500000 +2021-03-26 08:36:15,122 epoch 6 - iter 14/25 - loss 13.85568230 - samples/sec: 103.48 - lr: 0.500000 +2021-03-26 08:36:16,162 epoch 6 - iter 16/25 - loss 14.21910048 - samples/sec: 123.28 - lr: 0.500000 +2021-03-26 08:36:17,161 epoch 6 - iter 18/25 - loss 14.39588801 - samples/sec: 128.37 - lr: 0.500000 +2021-03-26 08:36:18,257 epoch 6 - iter 20/25 - loss 14.28939829 - samples/sec: 116.94 - lr: 0.500000 +2021-03-26 08:36:19,194 epoch 6 - iter 22/25 - loss 14.12584786 - samples/sec: 136.82 - lr: 0.500000 +2021-03-26 08:36:20,220 epoch 6 - iter 24/25 - loss 14.23289001 - samples/sec: 124.92 - lr: 0.500000 +2021-03-26 08:36:20,553 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:36:20,553 EPOCH 6 done: loss 14.0823 - lr 0.5000000 +2021-03-26 08:36:21,283 DEV : loss 11.072782516479492 - score 0.8198 +2021-03-26 08:36:21,299 BAD EPOCHS (no improvement): 0 +2021-03-26 08:36:30,981 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:36:32,262 epoch 7 - iter 2/25 - loss 12.87572241 - samples/sec: 100.12 - lr: 0.500000 +2021-03-26 08:36:33,586 epoch 7 - iter 4/25 - loss 12.51348233 - samples/sec: 96.81 - lr: 0.500000 +2021-03-26 08:36:34,826 epoch 7 - iter 6/25 - loss 12.69846837 - samples/sec: 103.30 - lr: 0.500000 +2021-03-26 08:36:36,145 epoch 7 - iter 8/25 - loss 12.91704547 - samples/sec: 97.14 - lr: 0.500000 +2021-03-26 08:36:37,117 epoch 7 - iter 10/25 - loss 12.93074894 - samples/sec: 132.01 - lr: 0.500000 +2021-03-26 08:36:38,164 epoch 7 - iter 12/25 - loss 12.55548334 - samples/sec: 122.49 - lr: 0.500000 +2021-03-26 08:36:39,063 epoch 7 - iter 14/25 - loss 12.45593732 - samples/sec: 142.64 - lr: 0.500000 +2021-03-26 08:36:39,987 epoch 7 - iter 16/25 - loss 12.29866284 - samples/sec: 138.81 - lr: 0.500000 +2021-03-26 08:36:40,989 epoch 7 - iter 18/25 - loss 12.40160184 - samples/sec: 127.91 - lr: 0.500000 +2021-03-26 08:36:41,916 epoch 7 - iter 20/25 - loss 12.34104595 - samples/sec: 138.34 - lr: 0.500000 +2021-03-26 08:36:42,949 epoch 7 - iter 22/25 - loss 12.41457592 - samples/sec: 124.03 - lr: 0.500000 +2021-03-26 08:36:43,950 epoch 7 - iter 24/25 - loss 12.49163620 - samples/sec: 128.02 - lr: 0.500000 +2021-03-26 08:36:44,382 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:36:44,383 EPOCH 7 done: loss 12.4664 - lr 0.5000000 +2021-03-26 08:36:45,152 DEV : loss 11.440208435058594 - score 0.8104 +2021-03-26 08:36:45,177 BAD EPOCHS (no improvement): 1 +2021-03-26 08:36:45,178 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:36:46,089 epoch 8 - iter 2/25 - loss 10.71792698 - samples/sec: 140.71 - lr: 0.500000 +2021-03-26 08:36:47,165 epoch 8 - iter 4/25 - loss 11.36736917 - samples/sec: 119.13 - lr: 0.500000 +2021-03-26 08:36:48,124 epoch 8 - iter 6/25 - loss 11.52289375 - samples/sec: 133.76 - lr: 0.500000 +2021-03-26 08:36:49,121 epoch 8 - iter 8/25 - loss 11.59045017 - samples/sec: 128.50 - lr: 0.500000 +2021-03-26 08:36:50,155 epoch 8 - iter 10/25 - loss 11.33041563 - samples/sec: 124.07 - lr: 0.500000 +2021-03-26 08:36:51,129 epoch 8 - iter 12/25 - loss 11.23035900 - samples/sec: 131.66 - lr: 0.500000 +2021-03-26 08:36:52,047 epoch 8 - iter 14/25 - loss 11.34309203 - samples/sec: 139.60 - lr: 0.500000 +2021-03-26 08:36:52,990 epoch 8 - iter 16/25 - loss 11.18513727 - samples/sec: 135.98 - lr: 0.500000 +2021-03-26 08:36:53,996 epoch 8 - iter 18/25 - loss 11.26731078 - samples/sec: 127.39 - lr: 0.500000 +2021-03-26 08:36:54,884 epoch 8 - iter 20/25 - loss 11.19707775 - samples/sec: 144.36 - lr: 0.500000 +2021-03-26 08:36:55,841 epoch 8 - iter 22/25 - loss 11.44174082 - samples/sec: 134.01 - lr: 0.500000 +2021-03-26 08:36:56,828 epoch 8 - iter 24/25 - loss 11.37479019 - samples/sec: 129.85 - lr: 0.500000 +2021-03-26 08:36:57,289 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:36:57,290 EPOCH 8 done: loss 11.3881 - lr 0.5000000 +2021-03-26 08:36:58,035 DEV : loss 9.5735502243042 - score 0.8414 +2021-03-26 08:36:58,059 BAD EPOCHS (no improvement): 0 +2021-03-26 08:37:07,275 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:37:08,220 epoch 9 - iter 2/25 - loss 10.45719576 - samples/sec: 135.78 - lr: 0.500000 +2021-03-26 08:37:09,281 epoch 9 - iter 4/25 - loss 10.45271993 - samples/sec: 121.36 - lr: 0.500000 +2021-03-26 08:37:10,312 epoch 9 - iter 6/25 - loss 10.19409847 - samples/sec: 124.24 - lr: 0.500000 +2021-03-26 08:37:11,327 epoch 9 - iter 8/25 - loss 10.50802386 - samples/sec: 126.36 - lr: 0.500000 +2021-03-26 08:37:12,329 epoch 9 - iter 10/25 - loss 10.60312605 - samples/sec: 127.89 - lr: 0.500000 +2021-03-26 08:37:13,378 epoch 9 - iter 12/25 - loss 10.58466109 - samples/sec: 122.21 - lr: 0.500000 +2021-03-26 08:37:14,361 epoch 9 - iter 14/25 - loss 10.61166177 - samples/sec: 130.49 - lr: 0.500000 +2021-03-26 08:37:15,380 epoch 9 - iter 16/25 - loss 10.69750690 - samples/sec: 125.77 - lr: 0.500000 +2021-03-26 08:37:16,507 epoch 9 - iter 18/25 - loss 10.68582206 - samples/sec: 113.75 - lr: 0.500000 +2021-03-26 08:37:17,456 epoch 9 - iter 20/25 - loss 10.49424787 - samples/sec: 135.16 - lr: 0.500000 +2021-03-26 08:37:18,358 epoch 9 - iter 22/25 - loss 10.47264957 - samples/sec: 142.10 - lr: 0.500000 +2021-03-26 08:37:19,258 epoch 9 - iter 24/25 - loss 10.37763687 - samples/sec: 142.39 - lr: 0.500000 +2021-03-26 08:37:19,632 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:37:19,633 EPOCH 9 done: loss 10.3442 - lr 0.5000000 +2021-03-26 08:37:20,381 DEV : loss 8.980548858642578 - score 0.8552 +2021-03-26 08:37:20,398 BAD EPOCHS (no improvement): 0 +2021-03-26 08:37:30,009 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:37:31,060 epoch 10 - iter 2/25 - loss 9.98690414 - samples/sec: 122.03 - lr: 0.500000 +2021-03-26 08:37:32,118 epoch 10 - iter 4/25 - loss 9.77284789 - samples/sec: 121.14 - lr: 0.500000 +2021-03-26 08:37:33,151 epoch 10 - iter 6/25 - loss 10.12020858 - samples/sec: 124.13 - lr: 0.500000 +2021-03-26 08:37:34,129 epoch 10 - iter 8/25 - loss 9.93451309 - samples/sec: 131.15 - lr: 0.500000 +2021-03-26 08:37:35,079 epoch 10 - iter 10/25 - loss 9.50131526 - samples/sec: 135.08 - lr: 0.500000 +2021-03-26 08:37:36,152 epoch 10 - iter 12/25 - loss 9.33990649 - samples/sec: 119.46 - lr: 0.500000 +2021-03-26 08:37:37,195 epoch 10 - iter 14/25 - loss 9.14538295 - samples/sec: 122.95 - lr: 0.500000 +2021-03-26 08:37:38,228 epoch 10 - iter 16/25 - loss 9.13881999 - samples/sec: 124.03 - lr: 0.500000 +2021-03-26 08:37:39,194 epoch 10 - iter 18/25 - loss 9.19282744 - samples/sec: 132.71 - lr: 0.500000 +2021-03-26 08:37:40,280 epoch 10 - iter 20/25 - loss 9.38936849 - samples/sec: 117.95 - lr: 0.500000 +2021-03-26 08:37:41,269 epoch 10 - iter 22/25 - loss 9.55408716 - samples/sec: 129.67 - lr: 0.500000 +2021-03-26 08:37:42,196 epoch 10 - iter 24/25 - loss 9.56575978 - samples/sec: 138.31 - lr: 0.500000 +2021-03-26 08:37:42,643 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:37:42,644 EPOCH 10 done: loss 9.6068 - lr 0.5000000 +2021-03-26 08:37:43,392 DEV : loss 8.683530807495117 - score 0.8596 +2021-03-26 08:37:43,412 BAD EPOCHS (no improvement): 0 +2021-03-26 08:37:53,041 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:37:54,008 epoch 11 - iter 2/25 - loss 8.94612074 - samples/sec: 132.62 - lr: 0.500000 +2021-03-26 08:37:55,051 epoch 11 - iter 4/25 - loss 9.17710257 - samples/sec: 122.89 - lr: 0.500000 +2021-03-26 08:37:55,985 epoch 11 - iter 6/25 - loss 8.82817014 - samples/sec: 137.20 - lr: 0.500000 +2021-03-26 08:37:56,966 epoch 11 - iter 8/25 - loss 8.97275388 - samples/sec: 130.85 - lr: 0.500000 +2021-03-26 08:37:57,933 epoch 11 - iter 10/25 - loss 9.16564417 - samples/sec: 132.59 - lr: 0.500000 +2021-03-26 08:37:58,894 epoch 11 - iter 12/25 - loss 8.90157481 - samples/sec: 133.41 - lr: 0.500000 +2021-03-26 08:37:59,848 epoch 11 - iter 14/25 - loss 8.76389977 - samples/sec: 134.37 - lr: 0.500000 +2021-03-26 08:38:00,834 epoch 11 - iter 16/25 - loss 8.81811097 - samples/sec: 129.96 - lr: 0.500000 +2021-03-26 08:38:01,823 epoch 11 - iter 18/25 - loss 8.87183176 - samples/sec: 129.77 - lr: 0.500000 +2021-03-26 08:38:02,834 epoch 11 - iter 20/25 - loss 8.87209656 - samples/sec: 126.72 - lr: 0.500000 +2021-03-26 08:38:03,733 epoch 11 - iter 22/25 - loss 8.93154610 - samples/sec: 142.58 - lr: 0.500000 +2021-03-26 08:38:04,627 epoch 11 - iter 24/25 - loss 9.03405192 - samples/sec: 143.50 - lr: 0.500000 +2021-03-26 08:38:04,988 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:38:04,989 EPOCH 11 done: loss 8.9434 - lr 0.5000000 +2021-03-26 08:38:05,711 DEV : loss 7.607541561126709 - score 0.8788 +2021-03-26 08:38:05,734 BAD EPOCHS (no improvement): 0 +2021-03-26 08:38:15,129 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:38:16,130 epoch 12 - iter 2/25 - loss 7.44216204 - samples/sec: 128.18 - lr: 0.500000 +2021-03-26 08:38:17,114 epoch 12 - iter 4/25 - loss 7.56906962 - samples/sec: 130.32 - lr: 0.500000 +2021-03-26 08:38:18,052 epoch 12 - iter 6/25 - loss 7.80390724 - samples/sec: 136.73 - lr: 0.500000 +2021-03-26 08:38:19,138 epoch 12 - iter 8/25 - loss 7.80161536 - samples/sec: 117.99 - lr: 0.500000 +2021-03-26 08:38:20,109 epoch 12 - iter 10/25 - loss 7.69097266 - samples/sec: 132.06 - lr: 0.500000 +2021-03-26 08:38:21,179 epoch 12 - iter 12/25 - loss 7.66236715 - samples/sec: 119.76 - lr: 0.500000 +2021-03-26 08:38:22,187 epoch 12 - iter 14/25 - loss 7.76549241 - samples/sec: 127.18 - lr: 0.500000 +2021-03-26 08:38:23,178 epoch 12 - iter 16/25 - loss 7.78144372 - samples/sec: 129.30 - lr: 0.500000 +2021-03-26 08:38:24,256 epoch 12 - iter 18/25 - loss 7.74660744 - samples/sec: 119.00 - lr: 0.500000 +2021-03-26 08:38:25,177 epoch 12 - iter 20/25 - loss 7.90175493 - samples/sec: 139.07 - lr: 0.500000 +2021-03-26 08:38:26,171 epoch 12 - iter 22/25 - loss 8.10105963 - samples/sec: 129.05 - lr: 0.500000 +2021-03-26 08:38:27,207 epoch 12 - iter 24/25 - loss 8.17068495 - samples/sec: 123.74 - lr: 0.500000 +2021-03-26 08:38:27,625 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:38:27,626 EPOCH 12 done: loss 8.2684 - lr 0.5000000 +2021-03-26 08:38:28,392 DEV : loss 7.355037212371826 - score 0.8781 +2021-03-26 08:38:28,411 BAD EPOCHS (no improvement): 1 +2021-03-26 08:38:28,412 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:38:29,456 epoch 13 - iter 2/25 - loss 6.67936397 - samples/sec: 122.80 - lr: 0.500000 +2021-03-26 08:38:30,404 epoch 13 - iter 4/25 - loss 7.57645524 - samples/sec: 135.18 - lr: 0.500000 +2021-03-26 08:38:31,396 epoch 13 - iter 6/25 - loss 7.58594402 - samples/sec: 129.29 - lr: 0.500000 +2021-03-26 08:38:32,350 epoch 13 - iter 8/25 - loss 7.89992797 - samples/sec: 134.20 - lr: 0.500000 +2021-03-26 08:38:33,347 epoch 13 - iter 10/25 - loss 7.78038921 - samples/sec: 128.74 - lr: 0.500000 +2021-03-26 08:38:34,356 epoch 13 - iter 12/25 - loss 7.66072885 - samples/sec: 126.96 - lr: 0.500000 +2021-03-26 08:38:35,420 epoch 13 - iter 14/25 - loss 7.55210911 - samples/sec: 120.48 - lr: 0.500000 +2021-03-26 08:38:36,577 epoch 13 - iter 16/25 - loss 7.69292659 - samples/sec: 111.17 - lr: 0.500000 +2021-03-26 08:38:37,593 epoch 13 - iter 18/25 - loss 7.75315873 - samples/sec: 126.23 - lr: 0.500000 +2021-03-26 08:38:38,590 epoch 13 - iter 20/25 - loss 7.81584940 - samples/sec: 128.51 - lr: 0.500000 +2021-03-26 08:38:39,520 epoch 13 - iter 22/25 - loss 7.80141653 - samples/sec: 137.87 - lr: 0.500000 +2021-03-26 08:38:40,478 epoch 13 - iter 24/25 - loss 7.83267780 - samples/sec: 133.82 - lr: 0.500000 +2021-03-26 08:38:40,834 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:38:40,835 EPOCH 13 done: loss 7.8342 - lr 0.5000000 +2021-03-26 08:38:41,566 DEV : loss 7.358130931854248 - score 0.8841 +2021-03-26 08:38:41,590 BAD EPOCHS (no improvement): 0 +2021-03-26 08:38:50,806 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:38:51,808 epoch 14 - iter 2/25 - loss 6.54843807 - samples/sec: 128.04 - lr: 0.500000 +2021-03-26 08:38:52,728 epoch 14 - iter 4/25 - loss 6.75311780 - samples/sec: 139.33 - lr: 0.500000 +2021-03-26 08:38:53,728 epoch 14 - iter 6/25 - loss 7.10001183 - samples/sec: 128.21 - lr: 0.500000 +2021-03-26 08:38:54,648 epoch 14 - iter 8/25 - loss 6.90141511 - samples/sec: 139.28 - lr: 0.500000 +2021-03-26 08:38:55,531 epoch 14 - iter 10/25 - loss 7.01782126 - samples/sec: 145.20 - lr: 0.500000 +2021-03-26 08:38:56,475 epoch 14 - iter 12/25 - loss 7.19454340 - samples/sec: 135.86 - lr: 0.500000 +2021-03-26 08:38:57,459 epoch 14 - iter 14/25 - loss 7.26160775 - samples/sec: 130.31 - lr: 0.500000 +2021-03-26 08:38:58,458 epoch 14 - iter 16/25 - loss 7.34082398 - samples/sec: 128.23 - lr: 0.500000 +2021-03-26 08:38:59,442 epoch 14 - iter 18/25 - loss 7.50727646 - samples/sec: 130.36 - lr: 0.500000 +2021-03-26 08:39:00,365 epoch 14 - iter 20/25 - loss 7.40654581 - samples/sec: 138.79 - lr: 0.500000 +2021-03-26 08:39:01,364 epoch 14 - iter 22/25 - loss 7.31519435 - samples/sec: 128.39 - lr: 0.500000 +2021-03-26 08:39:02,317 epoch 14 - iter 24/25 - loss 7.44761574 - samples/sec: 134.51 - lr: 0.500000 +2021-03-26 08:39:02,718 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:39:02,719 EPOCH 14 done: loss 7.4455 - lr 0.5000000 +2021-03-26 08:39:03,491 DEV : loss 7.53242826461792 - score 0.8761 +2021-03-26 08:39:03,508 BAD EPOCHS (no improvement): 1 +2021-03-26 08:39:03,509 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:39:04,559 epoch 15 - iter 2/25 - loss 7.57845902 - samples/sec: 122.11 - lr: 0.500000 +2021-03-26 08:39:05,486 epoch 15 - iter 4/25 - loss 6.90868700 - samples/sec: 138.40 - lr: 0.500000 +2021-03-26 08:39:06,481 epoch 15 - iter 6/25 - loss 7.40500800 - samples/sec: 128.76 - lr: 0.500000 +2021-03-26 08:39:07,504 epoch 15 - iter 8/25 - loss 7.43870729 - samples/sec: 125.43 - lr: 0.500000 +2021-03-26 08:39:08,523 epoch 15 - iter 10/25 - loss 7.39640107 - samples/sec: 125.85 - lr: 0.500000 +2021-03-26 08:39:09,566 epoch 15 - iter 12/25 - loss 7.33035028 - samples/sec: 122.82 - lr: 0.500000 +2021-03-26 08:39:10,417 epoch 15 - iter 14/25 - loss 7.25882561 - samples/sec: 150.82 - lr: 0.500000 +2021-03-26 08:39:11,383 epoch 15 - iter 16/25 - loss 7.28578192 - samples/sec: 132.74 - lr: 0.500000 +2021-03-26 08:39:12,277 epoch 15 - iter 18/25 - loss 7.23205696 - samples/sec: 143.39 - lr: 0.500000 +2021-03-26 08:39:13,271 epoch 15 - iter 20/25 - loss 7.30031002 - samples/sec: 128.93 - lr: 0.500000 +2021-03-26 08:39:14,196 epoch 15 - iter 22/25 - loss 7.25597163 - samples/sec: 138.67 - lr: 0.500000 +2021-03-26 08:39:15,084 epoch 15 - iter 24/25 - loss 7.16761416 - samples/sec: 144.22 - lr: 0.500000 +2021-03-26 08:39:15,503 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:39:15,504 EPOCH 15 done: loss 7.1529 - lr 0.5000000 +2021-03-26 08:39:16,263 DEV : loss 7.279156684875488 - score 0.8884 +2021-03-26 08:39:16,287 BAD EPOCHS (no improvement): 0 +2021-03-26 08:39:25,789 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:39:26,927 epoch 16 - iter 2/25 - loss 6.64370656 - samples/sec: 112.61 - lr: 0.500000 +2021-03-26 08:39:27,832 epoch 16 - iter 4/25 - loss 6.81591594 - samples/sec: 141.76 - lr: 0.500000 +2021-03-26 08:39:28,778 epoch 16 - iter 6/25 - loss 6.51474301 - samples/sec: 135.60 - lr: 0.500000 +2021-03-26 08:39:29,703 epoch 16 - iter 8/25 - loss 6.64946312 - samples/sec: 138.49 - lr: 0.500000 +2021-03-26 08:39:30,728 epoch 16 - iter 10/25 - loss 6.83099728 - samples/sec: 125.05 - lr: 0.500000 +2021-03-26 08:39:31,795 epoch 16 - iter 12/25 - loss 6.86230055 - samples/sec: 120.21 - lr: 0.500000 +2021-03-26 08:39:32,842 epoch 16 - iter 14/25 - loss 6.90279361 - samples/sec: 122.40 - lr: 0.500000 +2021-03-26 08:39:33,857 epoch 16 - iter 16/25 - loss 6.95628285 - samples/sec: 126.31 - lr: 0.500000 +2021-03-26 08:39:34,779 epoch 16 - iter 18/25 - loss 6.91275793 - samples/sec: 138.99 - lr: 0.500000 +2021-03-26 08:39:35,744 epoch 16 - iter 20/25 - loss 6.88684988 - samples/sec: 132.98 - lr: 0.500000 +2021-03-26 08:39:36,721 epoch 16 - iter 22/25 - loss 6.92513863 - samples/sec: 131.28 - lr: 0.500000 +2021-03-26 08:39:37,649 epoch 16 - iter 24/25 - loss 6.90109104 - samples/sec: 138.11 - lr: 0.500000 +2021-03-26 08:39:38,014 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:39:38,015 EPOCH 16 done: loss 6.8474 - lr 0.5000000 +2021-03-26 08:39:38,765 DEV : loss 7.319377422332764 - score 0.8802 +2021-03-26 08:39:38,788 BAD EPOCHS (no improvement): 1 +2021-03-26 08:39:38,789 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:39:39,920 epoch 17 - iter 2/25 - loss 5.65548801 - samples/sec: 113.31 - lr: 0.500000 +2021-03-26 08:39:40,848 epoch 17 - iter 4/25 - loss 5.68925560 - samples/sec: 138.80 - lr: 0.500000 +2021-03-26 08:39:41,815 epoch 17 - iter 6/25 - loss 6.08442839 - samples/sec: 132.59 - lr: 0.500000 +2021-03-26 08:39:42,824 epoch 17 - iter 8/25 - loss 5.97657681 - samples/sec: 127.09 - lr: 0.500000 +2021-03-26 08:39:43,832 epoch 17 - iter 10/25 - loss 6.26011467 - samples/sec: 127.30 - lr: 0.500000 +2021-03-26 08:39:44,733 epoch 17 - iter 12/25 - loss 6.27742251 - samples/sec: 142.41 - lr: 0.500000 +2021-03-26 08:39:45,731 epoch 17 - iter 14/25 - loss 6.18852012 - samples/sec: 128.35 - lr: 0.500000 +2021-03-26 08:39:46,713 epoch 17 - iter 16/25 - loss 6.38471323 - samples/sec: 130.60 - lr: 0.500000 +2021-03-26 08:39:47,739 epoch 17 - iter 18/25 - loss 6.52620485 - samples/sec: 124.86 - lr: 0.500000 +2021-03-26 08:39:48,764 epoch 17 - iter 20/25 - loss 6.55608342 - samples/sec: 125.10 - lr: 0.500000 +2021-03-26 08:39:49,818 epoch 17 - iter 22/25 - loss 6.47405910 - samples/sec: 121.67 - lr: 0.500000 +2021-03-26 08:39:50,766 epoch 17 - iter 24/25 - loss 6.46988698 - samples/sec: 135.37 - lr: 0.500000 +2021-03-26 08:39:51,111 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:39:51,112 EPOCH 17 done: loss 6.4509 - lr 0.5000000 +2021-03-26 08:39:51,853 DEV : loss 6.72586727142334 - score 0.8953 +2021-03-26 08:39:51,878 BAD EPOCHS (no improvement): 0 +2021-03-26 08:40:01,225 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:40:02,219 epoch 18 - iter 2/25 - loss 6.05486703 - samples/sec: 129.05 - lr: 0.500000 +2021-03-26 08:40:03,451 epoch 18 - iter 4/25 - loss 6.04667962 - samples/sec: 104.06 - lr: 0.500000 +2021-03-26 08:40:04,462 epoch 18 - iter 6/25 - loss 5.93956693 - samples/sec: 126.96 - lr: 0.500000 +2021-03-26 08:40:05,487 epoch 18 - iter 8/25 - loss 5.89706308 - samples/sec: 125.07 - lr: 0.500000 +2021-03-26 08:40:06,433 epoch 18 - iter 10/25 - loss 5.82905989 - samples/sec: 135.45 - lr: 0.500000 +2021-03-26 08:40:07,354 epoch 18 - iter 12/25 - loss 5.86640871 - samples/sec: 139.26 - lr: 0.500000 +2021-03-26 08:40:08,333 epoch 18 - iter 14/25 - loss 5.87944187 - samples/sec: 130.84 - lr: 0.500000 +2021-03-26 08:40:09,424 epoch 18 - iter 16/25 - loss 5.91088557 - samples/sec: 117.56 - lr: 0.500000 +2021-03-26 08:40:10,369 epoch 18 - iter 18/25 - loss 5.97162143 - samples/sec: 135.65 - lr: 0.500000 +2021-03-26 08:40:11,404 epoch 18 - iter 20/25 - loss 6.03725386 - samples/sec: 123.85 - lr: 0.500000 +2021-03-26 08:40:12,408 epoch 18 - iter 22/25 - loss 6.11541224 - samples/sec: 127.79 - lr: 0.500000 +2021-03-26 08:40:13,455 epoch 18 - iter 24/25 - loss 6.06696043 - samples/sec: 122.45 - lr: 0.500000 +2021-03-26 08:40:13,913 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:40:13,913 EPOCH 18 done: loss 5.9884 - lr 0.5000000 +2021-03-26 08:40:14,674 DEV : loss 6.871837139129639 - score 0.8914 +2021-03-26 08:40:14,694 BAD EPOCHS (no improvement): 1 +2021-03-26 08:40:14,694 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:40:15,696 epoch 19 - iter 2/25 - loss 5.24985552 - samples/sec: 127.91 - lr: 0.500000 +2021-03-26 08:40:16,722 epoch 19 - iter 4/25 - loss 5.72425938 - samples/sec: 124.91 - lr: 0.500000 +2021-03-26 08:40:17,651 epoch 19 - iter 6/25 - loss 5.87467360 - samples/sec: 138.16 - lr: 0.500000 +2021-03-26 08:40:18,627 epoch 19 - iter 8/25 - loss 5.82207859 - samples/sec: 131.38 - lr: 0.500000 +2021-03-26 08:40:19,646 epoch 19 - iter 10/25 - loss 5.81725163 - samples/sec: 125.76 - lr: 0.500000 +2021-03-26 08:40:20,586 epoch 19 - iter 12/25 - loss 5.86108053 - samples/sec: 136.35 - lr: 0.500000 +2021-03-26 08:40:21,545 epoch 19 - iter 14/25 - loss 5.91433290 - samples/sec: 133.76 - lr: 0.500000 +2021-03-26 08:40:22,509 epoch 19 - iter 16/25 - loss 5.95745441 - samples/sec: 133.02 - lr: 0.500000 +2021-03-26 08:40:23,495 epoch 19 - iter 18/25 - loss 5.95007020 - samples/sec: 130.00 - lr: 0.500000 +2021-03-26 08:40:24,447 epoch 19 - iter 20/25 - loss 5.98423173 - samples/sec: 134.59 - lr: 0.500000 +2021-03-26 08:40:25,521 epoch 19 - iter 22/25 - loss 6.01124003 - samples/sec: 119.38 - lr: 0.500000 +2021-03-26 08:40:26,593 epoch 19 - iter 24/25 - loss 6.01857322 - samples/sec: 119.53 - lr: 0.500000 +2021-03-26 08:40:26,991 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:40:26,992 EPOCH 19 done: loss 6.0306 - lr 0.5000000 +2021-03-26 08:40:27,758 DEV : loss 7.084388732910156 - score 0.8885 +2021-03-26 08:40:27,782 BAD EPOCHS (no improvement): 2 +2021-03-26 08:40:27,783 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:40:28,699 epoch 20 - iter 2/25 - loss 4.97285271 - samples/sec: 139.91 - lr: 0.500000 +2021-03-26 08:40:29,613 epoch 20 - iter 4/25 - loss 5.13549459 - samples/sec: 140.30 - lr: 0.500000 +2021-03-26 08:40:30,616 epoch 20 - iter 6/25 - loss 5.40857466 - samples/sec: 127.93 - lr: 0.500000 +2021-03-26 08:40:31,607 epoch 20 - iter 8/25 - loss 5.32610589 - samples/sec: 129.41 - lr: 0.500000 +2021-03-26 08:40:32,533 epoch 20 - iter 10/25 - loss 5.53772874 - samples/sec: 138.40 - lr: 0.500000 +2021-03-26 08:40:33,543 epoch 20 - iter 12/25 - loss 5.69724166 - samples/sec: 127.09 - lr: 0.500000 +2021-03-26 08:40:34,538 epoch 20 - iter 14/25 - loss 5.63601783 - samples/sec: 128.87 - lr: 0.500000 +2021-03-26 08:40:35,589 epoch 20 - iter 16/25 - loss 5.61971328 - samples/sec: 121.98 - lr: 0.500000 +2021-03-26 08:40:36,594 epoch 20 - iter 18/25 - loss 5.67689334 - samples/sec: 127.52 - lr: 0.500000 +2021-03-26 08:40:39,218 epoch 20 - iter 20/25 - loss 5.62989011 - samples/sec: 48.80 - lr: 0.500000 +2021-03-26 08:40:40,191 epoch 20 - iter 22/25 - loss 5.64217880 - samples/sec: 131.86 - lr: 0.500000 +2021-03-26 08:40:41,153 epoch 20 - iter 24/25 - loss 5.66027576 - samples/sec: 133.33 - lr: 0.500000 +2021-03-26 08:40:41,618 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:40:41,619 EPOCH 20 done: loss 5.6115 - lr 0.5000000 +2021-03-26 08:40:42,388 DEV : loss 6.946372032165527 - score 0.9005 +2021-03-26 08:40:42,405 BAD EPOCHS (no improvement): 0 +2021-03-26 08:40:51,793 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:40:52,833 epoch 21 - iter 2/25 - loss 6.11340022 - samples/sec: 123.25 - lr: 0.500000 +2021-03-26 08:40:53,724 epoch 21 - iter 4/25 - loss 5.99155140 - samples/sec: 143.91 - lr: 0.500000 +2021-03-26 08:40:54,676 epoch 21 - iter 6/25 - loss 5.56770198 - samples/sec: 134.61 - lr: 0.500000 +2021-03-26 08:40:55,651 epoch 21 - iter 8/25 - loss 6.15605402 - samples/sec: 131.62 - lr: 0.500000 +2021-03-26 08:40:56,618 epoch 21 - iter 10/25 - loss 6.06293464 - samples/sec: 132.62 - lr: 0.500000 +2021-03-26 08:40:57,566 epoch 21 - iter 12/25 - loss 5.87442553 - samples/sec: 135.17 - lr: 0.500000 +2021-03-26 08:40:58,667 epoch 21 - iter 14/25 - loss 5.79054798 - samples/sec: 116.49 - lr: 0.500000 +2021-03-26 08:40:59,713 epoch 21 - iter 16/25 - loss 5.82898086 - samples/sec: 122.49 - lr: 0.500000 +2021-03-26 08:41:00,854 epoch 21 - iter 18/25 - loss 5.75326135 - samples/sec: 112.32 - lr: 0.500000 +2021-03-26 08:41:01,798 epoch 21 - iter 20/25 - loss 5.71771746 - samples/sec: 135.88 - lr: 0.500000 +2021-03-26 08:41:02,742 epoch 21 - iter 22/25 - loss 5.65753235 - samples/sec: 135.71 - lr: 0.500000 +2021-03-26 08:41:03,735 epoch 21 - iter 24/25 - loss 5.69550612 - samples/sec: 129.13 - lr: 0.500000 +2021-03-26 08:41:04,153 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:41:04,154 EPOCH 21 done: loss 5.6895 - lr 0.5000000 +2021-03-26 08:41:04,899 DEV : loss 6.666974067687988 - score 0.9 +2021-03-26 08:41:04,922 BAD EPOCHS (no improvement): 1 +2021-03-26 08:41:04,923 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:41:05,879 epoch 22 - iter 2/25 - loss 5.06302881 - samples/sec: 134.18 - lr: 0.500000 +2021-03-26 08:41:06,784 epoch 22 - iter 4/25 - loss 4.83719516 - samples/sec: 141.67 - lr: 0.500000 +2021-03-26 08:41:07,717 epoch 22 - iter 6/25 - loss 4.85809485 - samples/sec: 137.47 - lr: 0.500000 +2021-03-26 08:41:08,704 epoch 22 - iter 8/25 - loss 5.69873339 - samples/sec: 129.93 - lr: 0.500000 +2021-03-26 08:41:09,704 epoch 22 - iter 10/25 - loss 5.46689510 - samples/sec: 128.09 - lr: 0.500000 +2021-03-26 08:41:10,704 epoch 22 - iter 12/25 - loss 5.50562640 - samples/sec: 128.26 - lr: 0.500000 +2021-03-26 08:41:11,666 epoch 22 - iter 14/25 - loss 5.52807069 - samples/sec: 133.21 - lr: 0.500000 +2021-03-26 08:41:12,677 epoch 22 - iter 16/25 - loss 5.50435984 - samples/sec: 126.85 - lr: 0.500000 +2021-03-26 08:41:13,636 epoch 22 - iter 18/25 - loss 5.54933498 - samples/sec: 133.78 - lr: 0.500000 +2021-03-26 08:41:14,618 epoch 22 - iter 20/25 - loss 5.52546136 - samples/sec: 130.45 - lr: 0.500000 +2021-03-26 08:41:15,586 epoch 22 - iter 22/25 - loss 5.46390449 - samples/sec: 132.48 - lr: 0.500000 +2021-03-26 08:41:16,615 epoch 22 - iter 24/25 - loss 5.45882779 - samples/sec: 124.58 - lr: 0.500000 +2021-03-26 08:41:17,010 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:41:17,010 EPOCH 22 done: loss 5.4367 - lr 0.5000000 +2021-03-26 08:41:17,793 DEV : loss 6.977891445159912 - score 0.8973 +2021-03-26 08:41:17,813 BAD EPOCHS (no improvement): 2 +2021-03-26 08:41:17,814 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:41:18,854 epoch 23 - iter 2/25 - loss 4.55232620 - samples/sec: 123.24 - lr: 0.500000 +2021-03-26 08:41:19,816 epoch 23 - iter 4/25 - loss 4.89346027 - samples/sec: 133.31 - lr: 0.500000 +2021-03-26 08:41:20,769 epoch 23 - iter 6/25 - loss 5.26064793 - samples/sec: 134.43 - lr: 0.500000 +2021-03-26 08:41:21,988 epoch 23 - iter 8/25 - loss 5.34301186 - samples/sec: 105.15 - lr: 0.500000 +2021-03-26 08:41:23,162 epoch 23 - iter 10/25 - loss 5.08064032 - samples/sec: 109.15 - lr: 0.500000 +2021-03-26 08:41:24,175 epoch 23 - iter 12/25 - loss 5.16919645 - samples/sec: 126.65 - lr: 0.500000 +2021-03-26 08:41:25,233 epoch 23 - iter 14/25 - loss 5.28559242 - samples/sec: 121.14 - lr: 0.500000 +2021-03-26 08:41:26,195 epoch 23 - iter 16/25 - loss 5.19672565 - samples/sec: 133.33 - lr: 0.500000 +2021-03-26 08:41:27,107 epoch 23 - iter 18/25 - loss 5.19106085 - samples/sec: 140.47 - lr: 0.500000 +2021-03-26 08:41:28,134 epoch 23 - iter 20/25 - loss 5.15162991 - samples/sec: 124.82 - lr: 0.500000 +2021-03-26 08:41:29,156 epoch 23 - iter 22/25 - loss 5.25128710 - samples/sec: 125.50 - lr: 0.500000 +2021-03-26 08:41:30,026 epoch 23 - iter 24/25 - loss 5.23282271 - samples/sec: 147.53 - lr: 0.500000 +2021-03-26 08:41:30,413 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:41:30,413 EPOCH 23 done: loss 5.2229 - lr 0.5000000 +2021-03-26 08:41:31,149 DEV : loss 7.015801429748535 - score 0.8919 +2021-03-26 08:41:31,174 BAD EPOCHS (no improvement): 3 +2021-03-26 08:41:31,175 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:41:32,168 epoch 24 - iter 2/25 - loss 4.33896101 - samples/sec: 129.11 - lr: 0.500000 +2021-03-26 08:41:33,190 epoch 24 - iter 4/25 - loss 4.21707565 - samples/sec: 126.02 - lr: 0.500000 +2021-03-26 08:41:34,198 epoch 24 - iter 6/25 - loss 4.36630698 - samples/sec: 127.20 - lr: 0.500000 +2021-03-26 08:41:35,226 epoch 24 - iter 8/25 - loss 4.65370998 - samples/sec: 125.28 - lr: 0.500000 +2021-03-26 08:41:36,198 epoch 24 - iter 10/25 - loss 4.71825922 - samples/sec: 131.79 - lr: 0.500000 +2021-03-26 08:41:37,268 epoch 24 - iter 12/25 - loss 4.75398221 - samples/sec: 119.84 - lr: 0.500000 +2021-03-26 08:41:38,265 epoch 24 - iter 14/25 - loss 4.70081021 - samples/sec: 128.58 - lr: 0.500000 +2021-03-26 08:41:39,267 epoch 24 - iter 16/25 - loss 4.67429534 - samples/sec: 127.91 - lr: 0.500000 +2021-03-26 08:41:40,232 epoch 24 - iter 18/25 - loss 4.71875207 - samples/sec: 132.88 - lr: 0.500000 +2021-03-26 08:41:41,242 epoch 24 - iter 20/25 - loss 4.69584209 - samples/sec: 126.91 - lr: 0.500000 +2021-03-26 08:41:42,166 epoch 24 - iter 22/25 - loss 4.68513228 - samples/sec: 138.89 - lr: 0.500000 +2021-03-26 08:41:43,180 epoch 24 - iter 24/25 - loss 4.76182438 - samples/sec: 126.46 - lr: 0.500000 +2021-03-26 08:41:43,537 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:41:43,538 EPOCH 24 done: loss 4.7702 - lr 0.5000000 +2021-03-26 08:41:44,271 DEV : loss 6.763913154602051 - score 0.8957 +2021-03-26 08:41:44,292 BAD EPOCHS (no improvement): 4 +2021-03-26 08:41:44,293 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:41:45,271 epoch 25 - iter 2/25 - loss 5.24708247 - samples/sec: 131.03 - lr: 0.250000 +2021-03-26 08:41:46,204 epoch 25 - iter 4/25 - loss 4.78944886 - samples/sec: 137.57 - lr: 0.250000 +2021-03-26 08:41:47,181 epoch 25 - iter 6/25 - loss 4.57319884 - samples/sec: 131.16 - lr: 0.250000 +2021-03-26 08:41:48,176 epoch 25 - iter 8/25 - loss 4.33389711 - samples/sec: 128.94 - lr: 0.250000 +2021-03-26 08:41:49,180 epoch 25 - iter 10/25 - loss 4.39515495 - samples/sec: 127.59 - lr: 0.250000 +2021-03-26 08:41:50,219 epoch 25 - iter 12/25 - loss 4.48107783 - samples/sec: 123.41 - lr: 0.250000 +2021-03-26 08:41:51,291 epoch 25 - iter 14/25 - loss 4.49906680 - samples/sec: 119.64 - lr: 0.250000 +2021-03-26 08:41:52,312 epoch 25 - iter 16/25 - loss 4.50435439 - samples/sec: 125.44 - lr: 0.250000 +2021-03-26 08:41:53,314 epoch 25 - iter 18/25 - loss 4.42300051 - samples/sec: 127.94 - lr: 0.250000 +2021-03-26 08:41:54,340 epoch 25 - iter 20/25 - loss 4.39977831 - samples/sec: 124.96 - lr: 0.250000 +2021-03-26 08:41:55,354 epoch 25 - iter 22/25 - loss 4.37190740 - samples/sec: 126.53 - lr: 0.250000 +2021-03-26 08:41:56,290 epoch 25 - iter 24/25 - loss 4.33572107 - samples/sec: 136.89 - lr: 0.250000 +2021-03-26 08:41:56,643 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:41:56,644 EPOCH 25 done: loss 4.3092 - lr 0.2500000 +2021-03-26 08:41:57,396 DEV : loss 6.576738357543945 - score 0.9022 +2021-03-26 08:41:57,419 BAD EPOCHS (no improvement): 0 +2021-03-26 08:42:06,929 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:42:07,974 epoch 26 - iter 2/25 - loss 4.18935883 - samples/sec: 122.77 - lr: 0.250000 +2021-03-26 08:42:09,067 epoch 26 - iter 4/25 - loss 3.63023371 - samples/sec: 117.31 - lr: 0.250000 +2021-03-26 08:42:10,081 epoch 26 - iter 6/25 - loss 3.64430161 - samples/sec: 126.52 - lr: 0.250000 +2021-03-26 08:42:11,060 epoch 26 - iter 8/25 - loss 3.71407154 - samples/sec: 130.92 - lr: 0.250000 +2021-03-26 08:42:12,106 epoch 26 - iter 10/25 - loss 3.79352601 - samples/sec: 122.62 - lr: 0.250000 +2021-03-26 08:42:13,032 epoch 26 - iter 12/25 - loss 3.97261920 - samples/sec: 138.35 - lr: 0.250000 +2021-03-26 08:42:14,085 epoch 26 - iter 14/25 - loss 4.01314941 - samples/sec: 121.75 - lr: 0.250000 +2021-03-26 08:42:15,077 epoch 26 - iter 16/25 - loss 4.07384159 - samples/sec: 129.18 - lr: 0.250000 +2021-03-26 08:42:15,993 epoch 26 - iter 18/25 - loss 4.06011808 - samples/sec: 140.02 - lr: 0.250000 +2021-03-26 08:42:16,912 epoch 26 - iter 20/25 - loss 4.01558514 - samples/sec: 139.43 - lr: 0.250000 +2021-03-26 08:42:18,042 epoch 26 - iter 22/25 - loss 3.97340371 - samples/sec: 113.44 - lr: 0.250000 +2021-03-26 08:42:19,058 epoch 26 - iter 24/25 - loss 3.99448071 - samples/sec: 126.18 - lr: 0.250000 +2021-03-26 08:42:19,458 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:42:19,459 EPOCH 26 done: loss 3.9627 - lr 0.2500000 +2021-03-26 08:42:20,219 DEV : loss 6.506423473358154 - score 0.9044 +2021-03-26 08:42:20,248 BAD EPOCHS (no improvement): 0 +2021-03-26 08:42:29,762 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:42:30,887 epoch 27 - iter 2/25 - loss 4.31431675 - samples/sec: 114.01 - lr: 0.250000 +2021-03-26 08:42:31,931 epoch 27 - iter 4/25 - loss 3.82997966 - samples/sec: 122.86 - lr: 0.250000 +2021-03-26 08:42:32,994 epoch 27 - iter 6/25 - loss 3.96800526 - samples/sec: 120.56 - lr: 0.250000 +2021-03-26 08:42:33,968 epoch 27 - iter 8/25 - loss 3.94697222 - samples/sec: 131.53 - lr: 0.250000 +2021-03-26 08:42:34,913 epoch 27 - iter 10/25 - loss 3.94078667 - samples/sec: 135.60 - lr: 0.250000 +2021-03-26 08:42:35,891 epoch 27 - iter 12/25 - loss 3.90649138 - samples/sec: 131.07 - lr: 0.250000 +2021-03-26 08:42:36,912 epoch 27 - iter 14/25 - loss 3.93579703 - samples/sec: 125.51 - lr: 0.250000 +2021-03-26 08:42:37,992 epoch 27 - iter 16/25 - loss 3.91942632 - samples/sec: 118.64 - lr: 0.250000 +2021-03-26 08:42:38,971 epoch 27 - iter 18/25 - loss 3.86642596 - samples/sec: 130.98 - lr: 0.250000 +2021-03-26 08:42:39,933 epoch 27 - iter 20/25 - loss 3.94384916 - samples/sec: 133.23 - lr: 0.250000 +2021-03-26 08:42:40,980 epoch 27 - iter 22/25 - loss 3.94357267 - samples/sec: 122.54 - lr: 0.250000 +2021-03-26 08:42:41,898 epoch 27 - iter 24/25 - loss 3.87114109 - samples/sec: 139.72 - lr: 0.250000 +2021-03-26 08:42:42,291 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:42:42,292 EPOCH 27 done: loss 3.8927 - lr 0.2500000 +2021-03-26 08:42:43,092 DEV : loss 6.57416296005249 - score 0.9036 +2021-03-26 08:42:43,111 BAD EPOCHS (no improvement): 1 +2021-03-26 08:42:43,112 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:42:44,137 epoch 28 - iter 2/25 - loss 4.20441771 - samples/sec: 125.10 - lr: 0.250000 +2021-03-26 08:42:45,079 epoch 28 - iter 4/25 - loss 4.04117072 - samples/sec: 136.22 - lr: 0.250000 +2021-03-26 08:42:46,052 epoch 28 - iter 6/25 - loss 3.90217503 - samples/sec: 131.82 - lr: 0.250000 +2021-03-26 08:42:46,957 epoch 28 - iter 8/25 - loss 3.69132367 - samples/sec: 141.73 - lr: 0.250000 +2021-03-26 08:42:47,947 epoch 28 - iter 10/25 - loss 3.78331764 - samples/sec: 129.49 - lr: 0.250000 +2021-03-26 08:42:49,024 epoch 28 - iter 12/25 - loss 3.77474389 - samples/sec: 119.00 - lr: 0.250000 +2021-03-26 08:42:50,025 epoch 28 - iter 14/25 - loss 3.82563947 - samples/sec: 128.03 - lr: 0.250000 +2021-03-26 08:42:51,001 epoch 28 - iter 16/25 - loss 3.82414263 - samples/sec: 131.42 - lr: 0.250000 +2021-03-26 08:42:51,927 epoch 28 - iter 18/25 - loss 3.78158188 - samples/sec: 138.47 - lr: 0.250000 +2021-03-26 08:42:52,916 epoch 28 - iter 20/25 - loss 3.72995106 - samples/sec: 129.54 - lr: 0.250000 +2021-03-26 08:42:53,934 epoch 28 - iter 22/25 - loss 3.69838404 - samples/sec: 125.95 - lr: 0.250000 +2021-03-26 08:42:54,922 epoch 28 - iter 24/25 - loss 3.72896739 - samples/sec: 129.85 - lr: 0.250000 +2021-03-26 08:42:55,362 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:42:55,363 EPOCH 28 done: loss 3.6976 - lr 0.2500000 +2021-03-26 08:42:56,121 DEV : loss 6.681036949157715 - score 0.9016 +2021-03-26 08:42:56,145 BAD EPOCHS (no improvement): 2 +2021-03-26 08:42:56,146 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:42:57,191 epoch 29 - iter 2/25 - loss 3.47235179 - samples/sec: 122.64 - lr: 0.250000 +2021-03-26 08:42:58,292 epoch 29 - iter 4/25 - loss 3.73076713 - samples/sec: 116.35 - lr: 0.250000 +2021-03-26 08:42:59,334 epoch 29 - iter 6/25 - loss 3.74192850 - samples/sec: 123.00 - lr: 0.250000 +2021-03-26 08:43:00,408 epoch 29 - iter 8/25 - loss 3.61097527 - samples/sec: 119.38 - lr: 0.250000 +2021-03-26 08:43:01,420 epoch 29 - iter 10/25 - loss 3.63619182 - samples/sec: 126.66 - lr: 0.250000 +2021-03-26 08:43:02,308 epoch 29 - iter 12/25 - loss 3.61553208 - samples/sec: 144.40 - lr: 0.250000 +2021-03-26 08:43:03,206 epoch 29 - iter 14/25 - loss 3.58807942 - samples/sec: 142.65 - lr: 0.250000 +2021-03-26 08:43:04,131 epoch 29 - iter 16/25 - loss 3.63437870 - samples/sec: 138.64 - lr: 0.250000 +2021-03-26 08:43:05,215 epoch 29 - iter 18/25 - loss 3.66702800 - samples/sec: 118.30 - lr: 0.250000 +2021-03-26 08:43:06,271 epoch 29 - iter 20/25 - loss 3.67497628 - samples/sec: 121.48 - lr: 0.250000 +2021-03-26 08:43:07,225 epoch 29 - iter 22/25 - loss 3.65543382 - samples/sec: 134.30 - lr: 0.250000 +2021-03-26 08:43:08,153 epoch 29 - iter 24/25 - loss 3.64861295 - samples/sec: 138.21 - lr: 0.250000 +2021-03-26 08:43:08,580 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:43:08,581 EPOCH 29 done: loss 3.6659 - lr 0.2500000 +2021-03-26 08:43:09,315 DEV : loss 6.766953945159912 - score 0.9048 +2021-03-26 08:43:09,353 BAD EPOCHS (no improvement): 0 +2021-03-26 08:43:18,858 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:43:19,893 epoch 30 - iter 2/25 - loss 3.69414401 - samples/sec: 123.98 - lr: 0.250000 +2021-03-26 08:43:20,827 epoch 30 - iter 4/25 - loss 3.38807523 - samples/sec: 137.21 - lr: 0.250000 +2021-03-26 08:43:21,764 epoch 30 - iter 6/25 - loss 3.66507101 - samples/sec: 136.75 - lr: 0.250000 +2021-03-26 08:43:22,826 epoch 30 - iter 8/25 - loss 3.57221785 - samples/sec: 120.83 - lr: 0.250000 +2021-03-26 08:43:23,799 epoch 30 - iter 10/25 - loss 3.56492391 - samples/sec: 131.64 - lr: 0.250000 +2021-03-26 08:43:24,769 epoch 30 - iter 12/25 - loss 3.63968432 - samples/sec: 132.21 - lr: 0.250000 +2021-03-26 08:43:25,831 epoch 30 - iter 14/25 - loss 3.68632470 - samples/sec: 120.75 - lr: 0.250000 +2021-03-26 08:43:26,878 epoch 30 - iter 16/25 - loss 3.70914277 - samples/sec: 122.54 - lr: 0.250000 +2021-03-26 08:43:27,792 epoch 30 - iter 18/25 - loss 3.67375978 - samples/sec: 140.37 - lr: 0.250000 +2021-03-26 08:43:28,862 epoch 30 - iter 20/25 - loss 3.65931660 - samples/sec: 119.81 - lr: 0.250000 +2021-03-26 08:43:29,812 epoch 30 - iter 22/25 - loss 3.63659420 - samples/sec: 134.81 - lr: 0.250000 +2021-03-26 08:43:30,741 epoch 30 - iter 24/25 - loss 3.66693341 - samples/sec: 138.06 - lr: 0.250000 +2021-03-26 08:43:31,134 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:43:31,135 EPOCH 30 done: loss 3.6635 - lr 0.2500000 +2021-03-26 08:43:31,887 DEV : loss 6.755105495452881 - score 0.9054 +2021-03-26 08:43:31,904 BAD EPOCHS (no improvement): 0 +2021-03-26 08:43:41,430 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:43:42,332 epoch 31 - iter 2/25 - loss 3.49644756 - samples/sec: 142.28 - lr: 0.250000 +2021-03-26 08:43:43,336 epoch 31 - iter 4/25 - loss 3.28100044 - samples/sec: 127.58 - lr: 0.250000 +2021-03-26 08:43:44,466 epoch 31 - iter 6/25 - loss 3.24940030 - samples/sec: 113.39 - lr: 0.250000 +2021-03-26 08:43:45,417 epoch 31 - iter 8/25 - loss 3.31971303 - samples/sec: 134.89 - lr: 0.250000 +2021-03-26 08:43:46,462 epoch 31 - iter 10/25 - loss 3.30865183 - samples/sec: 122.75 - lr: 0.250000 +2021-03-26 08:43:47,572 epoch 31 - iter 12/25 - loss 3.39676917 - samples/sec: 115.50 - lr: 0.250000 +2021-03-26 08:43:48,662 epoch 31 - iter 14/25 - loss 3.42026200 - samples/sec: 117.61 - lr: 0.250000 +2021-03-26 08:43:49,642 epoch 31 - iter 16/25 - loss 3.39093278 - samples/sec: 130.67 - lr: 0.250000 +2021-03-26 08:43:50,576 epoch 31 - iter 18/25 - loss 3.48906553 - samples/sec: 137.32 - lr: 0.250000 +2021-03-26 08:43:51,526 epoch 31 - iter 20/25 - loss 3.49162394 - samples/sec: 135.07 - lr: 0.250000 +2021-03-26 08:43:52,472 epoch 31 - iter 22/25 - loss 3.50332228 - samples/sec: 135.46 - lr: 0.250000 +2021-03-26 08:43:53,412 epoch 31 - iter 24/25 - loss 3.52942649 - samples/sec: 136.50 - lr: 0.250000 +2021-03-26 08:43:53,818 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:43:53,819 EPOCH 31 done: loss 3.5330 - lr 0.2500000 +2021-03-26 08:43:54,564 DEV : loss 6.835216522216797 - score 0.903 +2021-03-26 08:43:54,589 BAD EPOCHS (no improvement): 1 +2021-03-26 08:43:54,590 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:43:55,572 epoch 32 - iter 2/25 - loss 3.68399692 - samples/sec: 130.45 - lr: 0.250000 +2021-03-26 08:43:56,593 epoch 32 - iter 4/25 - loss 3.42861271 - samples/sec: 125.63 - lr: 0.250000 +2021-03-26 08:43:57,470 epoch 32 - iter 6/25 - loss 3.22910500 - samples/sec: 146.11 - lr: 0.250000 +2021-03-26 08:43:58,473 epoch 32 - iter 8/25 - loss 3.16920099 - samples/sec: 127.81 - lr: 0.250000 +2021-03-26 08:43:59,454 epoch 32 - iter 10/25 - loss 3.19274833 - samples/sec: 130.76 - lr: 0.250000 +2021-03-26 08:44:00,556 epoch 32 - iter 12/25 - loss 3.26887606 - samples/sec: 116.25 - lr: 0.250000 +2021-03-26 08:44:01,556 epoch 32 - iter 14/25 - loss 3.29435839 - samples/sec: 128.21 - lr: 0.250000 +2021-03-26 08:44:02,485 epoch 32 - iter 16/25 - loss 3.30416803 - samples/sec: 138.11 - lr: 0.250000 +2021-03-26 08:44:03,422 epoch 32 - iter 18/25 - loss 3.31257772 - samples/sec: 136.80 - lr: 0.250000 +2021-03-26 08:44:04,377 epoch 32 - iter 20/25 - loss 3.34610075 - samples/sec: 134.33 - lr: 0.250000 +2021-03-26 08:44:05,421 epoch 32 - iter 22/25 - loss 3.36202605 - samples/sec: 122.87 - lr: 0.250000 +2021-03-26 08:44:06,448 epoch 32 - iter 24/25 - loss 3.36099755 - samples/sec: 124.87 - lr: 0.250000 +2021-03-26 08:44:06,875 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:44:06,876 EPOCH 32 done: loss 3.3529 - lr 0.2500000 +2021-03-26 08:44:07,626 DEV : loss 6.639145851135254 - score 0.9066 +2021-03-26 08:44:07,650 BAD EPOCHS (no improvement): 0 +2021-03-26 08:44:16,987 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:44:17,947 epoch 33 - iter 2/25 - loss 3.20583904 - samples/sec: 133.66 - lr: 0.250000 +2021-03-26 08:44:19,016 epoch 33 - iter 4/25 - loss 3.36605328 - samples/sec: 119.94 - lr: 0.250000 +2021-03-26 08:44:19,959 epoch 33 - iter 6/25 - loss 3.21472077 - samples/sec: 136.06 - lr: 0.250000 +2021-03-26 08:44:20,907 epoch 33 - iter 8/25 - loss 3.14647344 - samples/sec: 135.18 - lr: 0.250000 +2021-03-26 08:44:22,049 epoch 33 - iter 10/25 - loss 3.24315312 - samples/sec: 112.26 - lr: 0.250000 +2021-03-26 08:44:23,046 epoch 33 - iter 12/25 - loss 3.17480514 - samples/sec: 128.67 - lr: 0.250000 +2021-03-26 08:44:24,183 epoch 33 - iter 14/25 - loss 3.30357449 - samples/sec: 112.76 - lr: 0.250000 +2021-03-26 08:44:25,182 epoch 33 - iter 16/25 - loss 3.28795835 - samples/sec: 128.35 - lr: 0.250000 +2021-03-26 08:44:26,164 epoch 33 - iter 18/25 - loss 3.24029554 - samples/sec: 130.45 - lr: 0.250000 +2021-03-26 08:44:27,271 epoch 33 - iter 20/25 - loss 3.30793083 - samples/sec: 115.84 - lr: 0.250000 +2021-03-26 08:44:28,326 epoch 33 - iter 22/25 - loss 3.33434795 - samples/sec: 121.51 - lr: 0.250000 +2021-03-26 08:44:29,317 epoch 33 - iter 24/25 - loss 3.32521027 - samples/sec: 129.35 - lr: 0.250000 +2021-03-26 08:44:29,715 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:44:29,716 EPOCH 33 done: loss 3.3160 - lr 0.2500000 +2021-03-26 08:44:30,494 DEV : loss 6.662256240844727 - score 0.9074 +2021-03-26 08:44:30,514 BAD EPOCHS (no improvement): 0 +2021-03-26 08:44:40,210 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:44:41,391 epoch 34 - iter 2/25 - loss 3.49009538 - samples/sec: 108.61 - lr: 0.250000 +2021-03-26 08:44:42,497 epoch 34 - iter 4/25 - loss 3.26404929 - samples/sec: 115.79 - lr: 0.250000 +2021-03-26 08:44:43,500 epoch 34 - iter 6/25 - loss 3.10120742 - samples/sec: 127.74 - lr: 0.250000 +2021-03-26 08:44:44,518 epoch 34 - iter 8/25 - loss 3.02353513 - samples/sec: 126.04 - lr: 0.250000 +2021-03-26 08:44:45,624 epoch 34 - iter 10/25 - loss 3.07420588 - samples/sec: 115.91 - lr: 0.250000 +2021-03-26 08:44:46,712 epoch 34 - iter 12/25 - loss 3.15896785 - samples/sec: 117.81 - lr: 0.250000 +2021-03-26 08:44:47,674 epoch 34 - iter 14/25 - loss 3.18685291 - samples/sec: 133.24 - lr: 0.250000 +2021-03-26 08:44:48,636 epoch 34 - iter 16/25 - loss 3.23354343 - samples/sec: 133.16 - lr: 0.250000 +2021-03-26 08:44:49,626 epoch 34 - iter 18/25 - loss 3.27925718 - samples/sec: 129.59 - lr: 0.250000 +2021-03-26 08:44:50,736 epoch 34 - iter 20/25 - loss 3.25261000 - samples/sec: 115.49 - lr: 0.250000 +2021-03-26 08:44:51,782 epoch 34 - iter 22/25 - loss 3.26143661 - samples/sec: 122.53 - lr: 0.250000 +2021-03-26 08:44:52,785 epoch 34 - iter 24/25 - loss 3.26214166 - samples/sec: 127.80 - lr: 0.250000 +2021-03-26 08:44:53,190 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:44:53,191 EPOCH 34 done: loss 3.2520 - lr 0.2500000 +2021-03-26 08:44:53,926 DEV : loss 6.694727897644043 - score 0.9102 +2021-03-26 08:44:53,950 BAD EPOCHS (no improvement): 0 +2021-03-26 08:45:03,526 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:45:04,592 epoch 35 - iter 2/25 - loss 3.33741724 - samples/sec: 120.37 - lr: 0.250000 +2021-03-26 08:45:05,596 epoch 35 - iter 4/25 - loss 3.01593387 - samples/sec: 127.60 - lr: 0.250000 +2021-03-26 08:45:06,650 epoch 35 - iter 6/25 - loss 3.08290581 - samples/sec: 121.55 - lr: 0.250000 +2021-03-26 08:45:07,707 epoch 35 - iter 8/25 - loss 3.21430799 - samples/sec: 121.29 - lr: 0.250000 +2021-03-26 08:45:08,768 epoch 35 - iter 10/25 - loss 3.20909221 - samples/sec: 120.87 - lr: 0.250000 +2021-03-26 08:45:09,839 epoch 35 - iter 12/25 - loss 3.20400945 - samples/sec: 119.64 - lr: 0.250000 +2021-03-26 08:45:10,877 epoch 35 - iter 14/25 - loss 3.25257616 - samples/sec: 123.50 - lr: 0.250000 +2021-03-26 08:45:11,873 epoch 35 - iter 16/25 - loss 3.26921830 - samples/sec: 128.72 - lr: 0.250000 +2021-03-26 08:45:12,860 epoch 35 - iter 18/25 - loss 3.23871719 - samples/sec: 129.93 - lr: 0.250000 +2021-03-26 08:45:13,846 epoch 35 - iter 20/25 - loss 3.22748883 - samples/sec: 130.03 - lr: 0.250000 +2021-03-26 08:45:14,923 epoch 35 - iter 22/25 - loss 3.21357602 - samples/sec: 119.01 - lr: 0.250000 +2021-03-26 08:45:15,865 epoch 35 - iter 24/25 - loss 3.24208915 - samples/sec: 136.13 - lr: 0.250000 +2021-03-26 08:45:16,274 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:45:16,275 EPOCH 35 done: loss 3.2753 - lr 0.2500000 +2021-03-26 08:45:17,067 DEV : loss 6.7822699546813965 - score 0.9058 +2021-03-26 08:45:17,092 BAD EPOCHS (no improvement): 1 +2021-03-26 08:45:17,092 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:45:18,105 epoch 36 - iter 2/25 - loss 3.46526396 - samples/sec: 126.58 - lr: 0.250000 +2021-03-26 08:45:19,095 epoch 36 - iter 4/25 - loss 3.33731157 - samples/sec: 129.49 - lr: 0.250000 +2021-03-26 08:45:20,066 epoch 36 - iter 6/25 - loss 3.45556358 - samples/sec: 131.92 - lr: 0.250000 +2021-03-26 08:45:21,042 epoch 36 - iter 8/25 - loss 3.26688248 - samples/sec: 131.39 - lr: 0.250000 +2021-03-26 08:45:21,958 epoch 36 - iter 10/25 - loss 3.25652921 - samples/sec: 139.96 - lr: 0.250000 +2021-03-26 08:45:22,991 epoch 36 - iter 12/25 - loss 3.21865408 - samples/sec: 124.14 - lr: 0.250000 +2021-03-26 08:45:23,951 epoch 36 - iter 14/25 - loss 3.16231186 - samples/sec: 133.34 - lr: 0.250000 +2021-03-26 08:45:24,997 epoch 36 - iter 16/25 - loss 3.22655720 - samples/sec: 122.71 - lr: 0.250000 +2021-03-26 08:45:25,978 epoch 36 - iter 18/25 - loss 3.20245018 - samples/sec: 130.57 - lr: 0.250000 +2021-03-26 08:45:26,943 epoch 36 - iter 20/25 - loss 3.18311380 - samples/sec: 132.80 - lr: 0.250000 +2021-03-26 08:45:27,963 epoch 36 - iter 22/25 - loss 3.12007433 - samples/sec: 125.67 - lr: 0.250000 +2021-03-26 08:45:28,930 epoch 36 - iter 24/25 - loss 3.11229009 - samples/sec: 132.60 - lr: 0.250000 +2021-03-26 08:45:29,331 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:45:29,332 EPOCH 36 done: loss 3.1368 - lr 0.2500000 +2021-03-26 08:45:30,069 DEV : loss 6.718545436859131 - score 0.9066 +2021-03-26 08:45:30,092 BAD EPOCHS (no improvement): 2 +2021-03-26 08:45:30,092 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:45:31,019 epoch 37 - iter 2/25 - loss 2.93264663 - samples/sec: 138.34 - lr: 0.250000 +2021-03-26 08:45:32,020 epoch 37 - iter 4/25 - loss 2.95200199 - samples/sec: 127.98 - lr: 0.250000 +2021-03-26 08:45:33,034 epoch 37 - iter 6/25 - loss 3.01997805 - samples/sec: 127.13 - lr: 0.250000 +2021-03-26 08:45:34,028 epoch 37 - iter 8/25 - loss 3.02635363 - samples/sec: 128.90 - lr: 0.250000 +2021-03-26 08:45:35,004 epoch 37 - iter 10/25 - loss 3.00781195 - samples/sec: 131.34 - lr: 0.250000 +2021-03-26 08:45:36,065 epoch 37 - iter 12/25 - loss 2.94086071 - samples/sec: 120.86 - lr: 0.250000 +2021-03-26 08:45:37,011 epoch 37 - iter 14/25 - loss 2.98774382 - samples/sec: 135.50 - lr: 0.250000 +2021-03-26 08:45:38,064 epoch 37 - iter 16/25 - loss 3.00074844 - samples/sec: 121.68 - lr: 0.250000 +2021-03-26 08:45:39,024 epoch 37 - iter 18/25 - loss 2.99614977 - samples/sec: 133.64 - lr: 0.250000 +2021-03-26 08:45:40,053 epoch 37 - iter 20/25 - loss 3.02320306 - samples/sec: 124.50 - lr: 0.250000 +2021-03-26 08:45:41,101 epoch 37 - iter 22/25 - loss 3.06846061 - samples/sec: 122.33 - lr: 0.250000 +2021-03-26 08:45:42,079 epoch 37 - iter 24/25 - loss 3.08361191 - samples/sec: 131.12 - lr: 0.250000 +2021-03-26 08:45:42,504 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:45:42,505 EPOCH 37 done: loss 3.0804 - lr 0.2500000 +2021-03-26 08:45:43,272 DEV : loss 6.583403587341309 - score 0.9052 +2021-03-26 08:45:43,293 BAD EPOCHS (no improvement): 3 +2021-03-26 08:45:43,293 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:45:44,243 epoch 38 - iter 2/25 - loss 3.14396656 - samples/sec: 134.92 - lr: 0.250000 +2021-03-26 08:45:45,198 epoch 38 - iter 4/25 - loss 2.75033486 - samples/sec: 134.22 - lr: 0.250000 +2021-03-26 08:45:46,123 epoch 38 - iter 6/25 - loss 2.67477520 - samples/sec: 138.80 - lr: 0.250000 +2021-03-26 08:45:47,131 epoch 38 - iter 8/25 - loss 2.75512740 - samples/sec: 127.13 - lr: 0.250000 +2021-03-26 08:45:48,154 epoch 38 - iter 10/25 - loss 2.81929321 - samples/sec: 125.38 - lr: 0.250000 +2021-03-26 08:45:49,108 epoch 38 - iter 12/25 - loss 2.80150716 - samples/sec: 134.46 - lr: 0.250000 +2021-03-26 08:45:50,072 epoch 38 - iter 14/25 - loss 2.77966118 - samples/sec: 132.95 - lr: 0.250000 +2021-03-26 08:45:51,035 epoch 38 - iter 16/25 - loss 2.80229428 - samples/sec: 133.17 - lr: 0.250000 +2021-03-26 08:45:52,024 epoch 38 - iter 18/25 - loss 2.84507292 - samples/sec: 129.76 - lr: 0.250000 +2021-03-26 08:45:53,159 epoch 38 - iter 20/25 - loss 2.81335837 - samples/sec: 112.93 - lr: 0.250000 +2021-03-26 08:45:54,243 epoch 38 - iter 22/25 - loss 2.86274156 - samples/sec: 118.16 - lr: 0.250000 +2021-03-26 08:45:55,333 epoch 38 - iter 24/25 - loss 2.85997577 - samples/sec: 117.71 - lr: 0.250000 +2021-03-26 08:45:55,806 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:45:55,807 EPOCH 38 done: loss 2.8757 - lr 0.2500000 +2021-03-26 08:45:56,583 DEV : loss 6.802188396453857 - score 0.9088 +2021-03-26 08:45:56,599 BAD EPOCHS (no improvement): 4 +2021-03-26 08:45:56,600 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:45:57,722 epoch 39 - iter 2/25 - loss 2.83308125 - samples/sec: 114.33 - lr: 0.125000 +2021-03-26 08:45:58,820 epoch 39 - iter 4/25 - loss 2.94828141 - samples/sec: 116.81 - lr: 0.125000 +2021-03-26 08:45:59,832 epoch 39 - iter 6/25 - loss 2.99452492 - samples/sec: 126.68 - lr: 0.125000 +2021-03-26 08:46:00,874 epoch 39 - iter 8/25 - loss 2.94919509 - samples/sec: 122.90 - lr: 0.125000 +2021-03-26 08:46:01,829 epoch 39 - iter 10/25 - loss 2.93447261 - samples/sec: 134.33 - lr: 0.125000 +2021-03-26 08:46:02,818 epoch 39 - iter 12/25 - loss 2.97544058 - samples/sec: 129.57 - lr: 0.125000 +2021-03-26 08:46:03,779 epoch 39 - iter 14/25 - loss 2.88676938 - samples/sec: 133.40 - lr: 0.125000 +2021-03-26 08:46:04,733 epoch 39 - iter 16/25 - loss 2.84791669 - samples/sec: 134.48 - lr: 0.125000 +2021-03-26 08:46:05,713 epoch 39 - iter 18/25 - loss 2.85739271 - samples/sec: 130.81 - lr: 0.125000 +2021-03-26 08:46:06,696 epoch 39 - iter 20/25 - loss 2.85741124 - samples/sec: 130.42 - lr: 0.125000 +2021-03-26 08:46:07,690 epoch 39 - iter 22/25 - loss 2.86451571 - samples/sec: 128.90 - lr: 0.125000 +2021-03-26 08:46:08,722 epoch 39 - iter 24/25 - loss 2.87007837 - samples/sec: 124.15 - lr: 0.125000 +2021-03-26 08:46:09,126 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:46:09,127 EPOCH 39 done: loss 2.8566 - lr 0.1250000 +2021-03-26 08:46:09,880 DEV : loss 6.693580627441406 - score 0.9088 +2021-03-26 08:46:09,897 BAD EPOCHS (no improvement): 1 +2021-03-26 08:46:09,898 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:46:10,909 epoch 40 - iter 2/25 - loss 2.41403103 - samples/sec: 126.79 - lr: 0.125000 +2021-03-26 08:46:11,820 epoch 40 - iter 4/25 - loss 2.40011775 - samples/sec: 140.76 - lr: 0.125000 +2021-03-26 08:46:12,888 epoch 40 - iter 6/25 - loss 2.49695289 - samples/sec: 120.01 - lr: 0.125000 +2021-03-26 08:46:13,834 epoch 40 - iter 8/25 - loss 2.58933467 - samples/sec: 135.58 - lr: 0.125000 +2021-03-26 08:46:14,709 epoch 40 - iter 10/25 - loss 2.58164473 - samples/sec: 146.48 - lr: 0.125000 +2021-03-26 08:46:15,700 epoch 40 - iter 12/25 - loss 2.68910889 - samples/sec: 129.30 - lr: 0.125000 +2021-03-26 08:46:16,653 epoch 40 - iter 14/25 - loss 2.66335935 - samples/sec: 134.51 - lr: 0.125000 +2021-03-26 08:46:17,672 epoch 40 - iter 16/25 - loss 2.73592710 - samples/sec: 125.95 - lr: 0.125000 +2021-03-26 08:46:18,705 epoch 40 - iter 18/25 - loss 2.75818470 - samples/sec: 124.09 - lr: 0.125000 +2021-03-26 08:46:19,674 epoch 40 - iter 20/25 - loss 2.74944264 - samples/sec: 132.24 - lr: 0.125000 +2021-03-26 08:46:20,646 epoch 40 - iter 22/25 - loss 2.75382914 - samples/sec: 131.99 - lr: 0.125000 +2021-03-26 08:46:21,770 epoch 40 - iter 24/25 - loss 2.74564613 - samples/sec: 114.09 - lr: 0.125000 +2021-03-26 08:46:22,241 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:46:22,242 EPOCH 40 done: loss 2.7428 - lr 0.1250000 +2021-03-26 08:46:23,001 DEV : loss 6.799490928649902 - score 0.9076 +2021-03-26 08:46:23,028 BAD EPOCHS (no improvement): 2 +2021-03-26 08:46:23,029 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:46:23,908 epoch 41 - iter 2/25 - loss 2.25075608 - samples/sec: 146.07 - lr: 0.125000 +2021-03-26 08:46:24,887 epoch 41 - iter 4/25 - loss 2.45685551 - samples/sec: 130.86 - lr: 0.125000 +2021-03-26 08:46:25,810 epoch 41 - iter 6/25 - loss 2.61635317 - samples/sec: 139.11 - lr: 0.125000 +2021-03-26 08:46:26,765 epoch 41 - iter 8/25 - loss 2.69645248 - samples/sec: 134.10 - lr: 0.125000 +2021-03-26 08:46:27,914 epoch 41 - iter 10/25 - loss 2.67517320 - samples/sec: 111.59 - lr: 0.125000 +2021-03-26 08:46:28,939 epoch 41 - iter 12/25 - loss 2.65610324 - samples/sec: 125.05 - lr: 0.125000 +2021-03-26 08:46:30,026 epoch 41 - iter 14/25 - loss 2.65179953 - samples/sec: 117.92 - lr: 0.125000 +2021-03-26 08:46:30,989 epoch 41 - iter 16/25 - loss 2.65190751 - samples/sec: 133.10 - lr: 0.125000 +2021-03-26 08:46:31,896 epoch 41 - iter 18/25 - loss 2.66618386 - samples/sec: 141.29 - lr: 0.125000 +2021-03-26 08:46:32,914 epoch 41 - iter 20/25 - loss 2.61516859 - samples/sec: 125.90 - lr: 0.125000 +2021-03-26 08:46:33,812 epoch 41 - iter 22/25 - loss 2.61662399 - samples/sec: 142.79 - lr: 0.125000 +2021-03-26 08:46:34,740 epoch 41 - iter 24/25 - loss 2.66955232 - samples/sec: 138.23 - lr: 0.125000 +2021-03-26 08:46:35,170 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:46:35,171 EPOCH 41 done: loss 2.6905 - lr 0.1250000 +2021-03-26 08:46:35,915 DEV : loss 6.693910121917725 - score 0.9054 +2021-03-26 08:46:35,940 BAD EPOCHS (no improvement): 3 +2021-03-26 08:46:35,940 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:46:36,906 epoch 42 - iter 2/25 - loss 2.75737107 - samples/sec: 132.81 - lr: 0.125000 +2021-03-26 08:46:37,831 epoch 42 - iter 4/25 - loss 2.93559647 - samples/sec: 138.60 - lr: 0.125000 +2021-03-26 08:46:38,895 epoch 42 - iter 6/25 - loss 2.81965216 - samples/sec: 120.46 - lr: 0.125000 +2021-03-26 08:46:39,871 epoch 42 - iter 8/25 - loss 2.80153769 - samples/sec: 131.31 - lr: 0.125000 +2021-03-26 08:46:40,879 epoch 42 - iter 10/25 - loss 2.78234267 - samples/sec: 127.23 - lr: 0.125000 +2021-03-26 08:46:41,917 epoch 42 - iter 12/25 - loss 2.72432474 - samples/sec: 123.45 - lr: 0.125000 +2021-03-26 08:46:42,898 epoch 42 - iter 14/25 - loss 2.62786084 - samples/sec: 130.71 - lr: 0.125000 +2021-03-26 08:46:43,995 epoch 42 - iter 16/25 - loss 2.63186632 - samples/sec: 116.86 - lr: 0.125000 +2021-03-26 08:46:44,915 epoch 42 - iter 18/25 - loss 2.65961860 - samples/sec: 139.22 - lr: 0.125000 +2021-03-26 08:46:45,855 epoch 42 - iter 20/25 - loss 2.66174264 - samples/sec: 136.53 - lr: 0.125000 +2021-03-26 08:46:46,866 epoch 42 - iter 22/25 - loss 2.61873937 - samples/sec: 126.80 - lr: 0.125000 +2021-03-26 08:46:47,855 epoch 42 - iter 24/25 - loss 2.62547451 - samples/sec: 129.64 - lr: 0.125000 +2021-03-26 08:46:48,363 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:46:48,364 EPOCH 42 done: loss 2.6608 - lr 0.1250000 +2021-03-26 08:46:49,132 DEV : loss 6.850523948669434 - score 0.91 +2021-03-26 08:46:49,165 BAD EPOCHS (no improvement): 4 +2021-03-26 08:46:49,166 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:46:50,220 epoch 43 - iter 2/25 - loss 2.49043608 - samples/sec: 121.62 - lr: 0.062500 +2021-03-26 08:46:51,260 epoch 43 - iter 4/25 - loss 2.86309117 - samples/sec: 123.32 - lr: 0.062500 +2021-03-26 08:46:52,181 epoch 43 - iter 6/25 - loss 2.72066963 - samples/sec: 139.08 - lr: 0.062500 +2021-03-26 08:46:53,208 epoch 43 - iter 8/25 - loss 2.77377698 - samples/sec: 124.80 - lr: 0.062500 +2021-03-26 08:46:54,182 epoch 43 - iter 10/25 - loss 2.66388309 - samples/sec: 131.72 - lr: 0.062500 +2021-03-26 08:46:55,219 epoch 43 - iter 12/25 - loss 2.73270496 - samples/sec: 123.65 - lr: 0.062500 +2021-03-26 08:46:56,229 epoch 43 - iter 14/25 - loss 2.76990683 - samples/sec: 127.04 - lr: 0.062500 +2021-03-26 08:46:57,268 epoch 43 - iter 16/25 - loss 2.69852231 - samples/sec: 123.35 - lr: 0.062500 +2021-03-26 08:46:58,290 epoch 43 - iter 18/25 - loss 2.64845743 - samples/sec: 125.34 - lr: 0.062500 +2021-03-26 08:46:59,225 epoch 43 - iter 20/25 - loss 2.66699877 - samples/sec: 137.08 - lr: 0.062500 +2021-03-26 08:47:00,182 epoch 43 - iter 22/25 - loss 2.68268208 - samples/sec: 134.04 - lr: 0.062500 +2021-03-26 08:47:01,230 epoch 43 - iter 24/25 - loss 2.70284759 - samples/sec: 122.36 - lr: 0.062500 +2021-03-26 08:47:01,612 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:47:01,613 EPOCH 43 done: loss 2.7074 - lr 0.0625000 +2021-03-26 08:47:02,369 DEV : loss 6.793069362640381 - score 0.9094 +2021-03-26 08:47:02,393 BAD EPOCHS (no improvement): 1 +2021-03-26 08:47:02,393 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:47:03,265 epoch 44 - iter 2/25 - loss 2.39724141 - samples/sec: 147.13 - lr: 0.062500 +2021-03-26 08:47:04,189 epoch 44 - iter 4/25 - loss 2.43123588 - samples/sec: 138.76 - lr: 0.062500 +2021-03-26 08:47:05,251 epoch 44 - iter 6/25 - loss 2.56454744 - samples/sec: 120.71 - lr: 0.062500 +2021-03-26 08:47:06,418 epoch 44 - iter 8/25 - loss 2.74957548 - samples/sec: 109.77 - lr: 0.062500 +2021-03-26 08:47:07,411 epoch 44 - iter 10/25 - loss 2.68237354 - samples/sec: 129.16 - lr: 0.062500 +2021-03-26 08:47:08,465 epoch 44 - iter 12/25 - loss 2.71276640 - samples/sec: 121.54 - lr: 0.062500 +2021-03-26 08:47:09,413 epoch 44 - iter 14/25 - loss 2.66092078 - samples/sec: 135.29 - lr: 0.062500 +2021-03-26 08:47:10,426 epoch 44 - iter 16/25 - loss 2.64549053 - samples/sec: 126.61 - lr: 0.062500 +2021-03-26 08:47:11,349 epoch 44 - iter 18/25 - loss 2.62857394 - samples/sec: 138.93 - lr: 0.062500 +2021-03-26 08:47:12,361 epoch 44 - iter 20/25 - loss 2.63828043 - samples/sec: 126.59 - lr: 0.062500 +2021-03-26 08:47:13,360 epoch 44 - iter 22/25 - loss 2.67504076 - samples/sec: 128.40 - lr: 0.062500 +2021-03-26 08:47:14,421 epoch 44 - iter 24/25 - loss 2.75515004 - samples/sec: 120.75 - lr: 0.062500 +2021-03-26 08:47:14,809 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:47:14,809 EPOCH 44 done: loss 2.7510 - lr 0.0625000 +2021-03-26 08:47:15,586 DEV : loss 6.782561302185059 - score 0.9082 +2021-03-26 08:47:15,602 BAD EPOCHS (no improvement): 2 +2021-03-26 08:47:15,603 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:47:16,633 epoch 45 - iter 2/25 - loss 2.96598649 - samples/sec: 124.51 - lr: 0.062500 +2021-03-26 08:47:17,596 epoch 45 - iter 4/25 - loss 2.79108897 - samples/sec: 133.20 - lr: 0.062500 +2021-03-26 08:47:18,627 epoch 45 - iter 6/25 - loss 2.74379843 - samples/sec: 124.28 - lr: 0.062500 +2021-03-26 08:47:19,648 epoch 45 - iter 8/25 - loss 2.79540862 - samples/sec: 125.51 - lr: 0.062500 +2021-03-26 08:47:20,662 epoch 45 - iter 10/25 - loss 2.71001374 - samples/sec: 126.40 - lr: 0.062500 +2021-03-26 08:47:21,649 epoch 45 - iter 12/25 - loss 2.65032153 - samples/sec: 129.83 - lr: 0.062500 +2021-03-26 08:47:22,676 epoch 45 - iter 14/25 - loss 2.61791879 - samples/sec: 124.91 - lr: 0.062500 +2021-03-26 08:47:23,624 epoch 45 - iter 16/25 - loss 2.56420042 - samples/sec: 135.43 - lr: 0.062500 +2021-03-26 08:47:24,634 epoch 45 - iter 18/25 - loss 2.52401664 - samples/sec: 126.91 - lr: 0.062500 +2021-03-26 08:47:25,664 epoch 45 - iter 20/25 - loss 2.52811976 - samples/sec: 124.42 - lr: 0.062500 +2021-03-26 08:47:26,637 epoch 45 - iter 22/25 - loss 2.54474491 - samples/sec: 131.81 - lr: 0.062500 +2021-03-26 08:47:27,589 epoch 45 - iter 24/25 - loss 2.54330914 - samples/sec: 134.69 - lr: 0.062500 +2021-03-26 08:47:28,030 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:47:28,031 EPOCH 45 done: loss 2.5855 - lr 0.0625000 +2021-03-26 08:47:28,791 DEV : loss 6.721575736999512 - score 0.9072 +2021-03-26 08:47:28,809 BAD EPOCHS (no improvement): 3 +2021-03-26 08:47:28,810 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:47:29,702 epoch 46 - iter 2/25 - loss 2.40093923 - samples/sec: 143.82 - lr: 0.062500 +2021-03-26 08:47:30,714 epoch 46 - iter 4/25 - loss 2.47187972 - samples/sec: 126.63 - lr: 0.062500 +2021-03-26 08:47:31,763 epoch 46 - iter 6/25 - loss 2.49867698 - samples/sec: 122.12 - lr: 0.062500 +2021-03-26 08:47:32,865 epoch 46 - iter 8/25 - loss 2.45476428 - samples/sec: 116.42 - lr: 0.062500 +2021-03-26 08:47:33,891 epoch 46 - iter 10/25 - loss 2.33877997 - samples/sec: 124.97 - lr: 0.062500 +2021-03-26 08:47:34,873 epoch 46 - iter 12/25 - loss 2.40645109 - samples/sec: 130.41 - lr: 0.062500 +2021-03-26 08:47:35,832 epoch 46 - iter 14/25 - loss 2.41505643 - samples/sec: 133.77 - lr: 0.062500 +2021-03-26 08:47:36,813 epoch 46 - iter 16/25 - loss 2.48525703 - samples/sec: 130.68 - lr: 0.062500 +2021-03-26 08:47:37,962 epoch 46 - iter 18/25 - loss 2.48812330 - samples/sec: 111.48 - lr: 0.062500 +2021-03-26 08:47:38,938 epoch 46 - iter 20/25 - loss 2.46523433 - samples/sec: 131.45 - lr: 0.062500 +2021-03-26 08:47:39,942 epoch 46 - iter 22/25 - loss 2.48067673 - samples/sec: 127.67 - lr: 0.062500 +2021-03-26 08:47:41,269 epoch 46 - iter 24/25 - loss 2.50672932 - samples/sec: 96.60 - lr: 0.062500 +2021-03-26 08:47:41,713 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:47:41,714 EPOCH 46 done: loss 2.5113 - lr 0.0625000 +2021-03-26 08:47:42,595 DEV : loss 6.7464094161987305 - score 0.9072 +2021-03-26 08:47:42,627 BAD EPOCHS (no improvement): 4 +2021-03-26 08:47:42,628 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:47:43,630 epoch 47 - iter 2/25 - loss 2.07991660 - samples/sec: 128.09 - lr: 0.031250 +2021-03-26 08:47:44,593 epoch 47 - iter 4/25 - loss 2.25694829 - samples/sec: 133.15 - lr: 0.031250 +2021-03-26 08:47:45,688 epoch 47 - iter 6/25 - loss 2.24701436 - samples/sec: 116.90 - lr: 0.031250 +2021-03-26 08:47:46,771 epoch 47 - iter 8/25 - loss 2.25035590 - samples/sec: 118.39 - lr: 0.031250 +2021-03-26 08:47:47,743 epoch 47 - iter 10/25 - loss 2.41273284 - samples/sec: 131.90 - lr: 0.031250 +2021-03-26 08:47:48,713 epoch 47 - iter 12/25 - loss 2.48591616 - samples/sec: 132.08 - lr: 0.031250 +2021-03-26 08:47:49,691 epoch 47 - iter 14/25 - loss 2.50090785 - samples/sec: 131.23 - lr: 0.031250 +2021-03-26 08:47:50,781 epoch 47 - iter 16/25 - loss 2.50587636 - samples/sec: 117.51 - lr: 0.031250 +2021-03-26 08:47:51,824 epoch 47 - iter 18/25 - loss 2.53428050 - samples/sec: 122.88 - lr: 0.031250 +2021-03-26 08:47:52,853 epoch 47 - iter 20/25 - loss 2.57194322 - samples/sec: 124.66 - lr: 0.031250 +2021-03-26 08:47:53,977 epoch 47 - iter 22/25 - loss 2.58494453 - samples/sec: 114.04 - lr: 0.031250 +2021-03-26 08:47:55,119 epoch 47 - iter 24/25 - loss 2.59840622 - samples/sec: 112.20 - lr: 0.031250 +2021-03-26 08:47:55,490 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:47:55,490 EPOCH 47 done: loss 2.5958 - lr 0.0312500 +2021-03-26 08:47:56,241 DEV : loss 6.7468581199646 - score 0.9068 +2021-03-26 08:47:56,265 BAD EPOCHS (no improvement): 1 +2021-03-26 08:47:56,266 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:47:57,179 epoch 48 - iter 2/25 - loss 2.16672790 - samples/sec: 140.57 - lr: 0.031250 +2021-03-26 08:47:58,075 epoch 48 - iter 4/25 - loss 2.28760791 - samples/sec: 143.19 - lr: 0.031250 +2021-03-26 08:47:59,117 epoch 48 - iter 6/25 - loss 2.36543711 - samples/sec: 123.08 - lr: 0.031250 +2021-03-26 08:48:00,091 epoch 48 - iter 8/25 - loss 2.53750896 - samples/sec: 131.64 - lr: 0.031250 +2021-03-26 08:48:01,264 epoch 48 - iter 10/25 - loss 2.52344959 - samples/sec: 109.29 - lr: 0.031250 +2021-03-26 08:48:02,245 epoch 48 - iter 12/25 - loss 2.54485043 - samples/sec: 130.63 - lr: 0.031250 +2021-03-26 08:48:03,192 epoch 48 - iter 14/25 - loss 2.59198126 - samples/sec: 135.37 - lr: 0.031250 +2021-03-26 08:48:04,341 epoch 48 - iter 16/25 - loss 2.58411257 - samples/sec: 111.56 - lr: 0.031250 +2021-03-26 08:48:05,302 epoch 48 - iter 18/25 - loss 2.56820940 - samples/sec: 133.31 - lr: 0.031250 +2021-03-26 08:48:06,359 epoch 48 - iter 20/25 - loss 2.56453640 - samples/sec: 121.25 - lr: 0.031250 +2021-03-26 08:48:07,342 epoch 48 - iter 22/25 - loss 2.58118105 - samples/sec: 130.59 - lr: 0.031250 +2021-03-26 08:48:08,366 epoch 48 - iter 24/25 - loss 2.53787378 - samples/sec: 125.17 - lr: 0.031250 +2021-03-26 08:48:08,770 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:48:08,771 EPOCH 48 done: loss 2.5485 - lr 0.0312500 +2021-03-26 08:48:09,505 DEV : loss 6.730559349060059 - score 0.9076 +2021-03-26 08:48:09,529 BAD EPOCHS (no improvement): 2 +2021-03-26 08:48:09,530 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:48:10,517 epoch 49 - iter 2/25 - loss 2.36573863 - samples/sec: 129.86 - lr: 0.031250 +2021-03-26 08:48:11,512 epoch 49 - iter 4/25 - loss 2.27939576 - samples/sec: 128.82 - lr: 0.031250 +2021-03-26 08:48:12,490 epoch 49 - iter 6/25 - loss 2.51892324 - samples/sec: 131.15 - lr: 0.031250 +2021-03-26 08:48:13,565 epoch 49 - iter 8/25 - loss 2.37164113 - samples/sec: 119.16 - lr: 0.031250 +2021-03-26 08:48:14,605 epoch 49 - iter 10/25 - loss 2.39122179 - samples/sec: 123.27 - lr: 0.031250 +2021-03-26 08:48:15,732 epoch 49 - iter 12/25 - loss 2.49826862 - samples/sec: 113.71 - lr: 0.031250 +2021-03-26 08:48:16,856 epoch 49 - iter 14/25 - loss 2.58189925 - samples/sec: 114.06 - lr: 0.031250 +2021-03-26 08:48:17,929 epoch 49 - iter 16/25 - loss 2.60247803 - samples/sec: 119.58 - lr: 0.031250 +2021-03-26 08:48:18,989 epoch 49 - iter 18/25 - loss 2.54768838 - samples/sec: 120.91 - lr: 0.031250 +2021-03-26 08:48:19,981 epoch 49 - iter 20/25 - loss 2.51007439 - samples/sec: 129.09 - lr: 0.031250 +2021-03-26 08:48:20,902 epoch 49 - iter 22/25 - loss 2.48375236 - samples/sec: 139.24 - lr: 0.031250 +2021-03-26 08:48:21,977 epoch 49 - iter 24/25 - loss 2.44927419 - samples/sec: 119.22 - lr: 0.031250 +2021-03-26 08:48:22,382 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:48:22,382 EPOCH 49 done: loss 2.4858 - lr 0.0312500 +2021-03-26 08:48:23,133 DEV : loss 6.6969757080078125 - score 0.9078 +2021-03-26 08:48:23,157 BAD EPOCHS (no improvement): 3 +2021-03-26 08:48:23,158 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:48:24,151 epoch 50 - iter 2/25 - loss 2.61817443 - samples/sec: 129.17 - lr: 0.031250 +2021-03-26 08:48:25,130 epoch 50 - iter 4/25 - loss 2.31582826 - samples/sec: 130.94 - lr: 0.031250 +2021-03-26 08:48:26,083 epoch 50 - iter 6/25 - loss 2.39882831 - samples/sec: 134.41 - lr: 0.031250 +2021-03-26 08:48:27,099 epoch 50 - iter 8/25 - loss 2.64799169 - samples/sec: 126.25 - lr: 0.031250 +2021-03-26 08:48:28,151 epoch 50 - iter 10/25 - loss 2.57482643 - samples/sec: 121.97 - lr: 0.031250 +2021-03-26 08:48:29,159 epoch 50 - iter 12/25 - loss 2.58212765 - samples/sec: 127.22 - lr: 0.031250 +2021-03-26 08:48:30,091 epoch 50 - iter 14/25 - loss 2.61432253 - samples/sec: 137.47 - lr: 0.031250 +2021-03-26 08:48:31,109 epoch 50 - iter 16/25 - loss 2.60016759 - samples/sec: 125.93 - lr: 0.031250 +2021-03-26 08:48:32,090 epoch 50 - iter 18/25 - loss 2.58133325 - samples/sec: 130.66 - lr: 0.031250 +2021-03-26 08:48:33,114 epoch 50 - iter 20/25 - loss 2.60757553 - samples/sec: 125.08 - lr: 0.031250 +2021-03-26 08:48:34,136 epoch 50 - iter 22/25 - loss 2.61094578 - samples/sec: 125.49 - lr: 0.031250 +2021-03-26 08:48:35,100 epoch 50 - iter 24/25 - loss 2.65030560 - samples/sec: 132.91 - lr: 0.031250 +2021-03-26 08:48:35,586 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:48:35,587 EPOCH 50 done: loss 2.6470 - lr 0.0312500 +2021-03-26 08:48:36,342 DEV : loss 6.714729309082031 - score 0.91 +2021-03-26 08:48:36,366 BAD EPOCHS (no improvement): 4 +2021-03-26 08:48:36,367 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:48:37,375 epoch 51 - iter 2/25 - loss 2.76345909 - samples/sec: 127.17 - lr: 0.015625 +2021-03-26 08:48:38,347 epoch 51 - iter 4/25 - loss 2.76501840 - samples/sec: 131.99 - lr: 0.015625 +2021-03-26 08:48:39,522 epoch 51 - iter 6/25 - loss 2.60267341 - samples/sec: 109.16 - lr: 0.015625 +2021-03-26 08:48:40,671 epoch 51 - iter 8/25 - loss 2.78652990 - samples/sec: 111.51 - lr: 0.015625 +2021-03-26 08:48:41,772 epoch 51 - iter 10/25 - loss 2.75791469 - samples/sec: 116.40 - lr: 0.015625 +2021-03-26 08:48:42,724 epoch 51 - iter 12/25 - loss 2.66620205 - samples/sec: 134.69 - lr: 0.015625 +2021-03-26 08:48:43,780 epoch 51 - iter 14/25 - loss 2.66174328 - samples/sec: 121.36 - lr: 0.015625 +2021-03-26 08:48:44,838 epoch 51 - iter 16/25 - loss 2.58969487 - samples/sec: 121.18 - lr: 0.015625 +2021-03-26 08:48:45,873 epoch 51 - iter 18/25 - loss 2.53882750 - samples/sec: 124.01 - lr: 0.015625 +2021-03-26 08:48:46,806 epoch 51 - iter 20/25 - loss 2.53399588 - samples/sec: 137.43 - lr: 0.015625 +2021-03-26 08:48:47,868 epoch 51 - iter 22/25 - loss 2.51981955 - samples/sec: 120.60 - lr: 0.015625 +2021-03-26 08:48:48,863 epoch 51 - iter 24/25 - loss 2.51983155 - samples/sec: 128.95 - lr: 0.015625 +2021-03-26 08:48:49,259 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:48:49,259 EPOCH 51 done: loss 2.5411 - lr 0.0156250 +2021-03-26 08:48:50,025 DEV : loss 6.709239959716797 - score 0.908 +2021-03-26 08:48:50,049 BAD EPOCHS (no improvement): 1 +2021-03-26 08:48:50,050 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:48:51,077 epoch 52 - iter 2/25 - loss 2.54436791 - samples/sec: 124.85 - lr: 0.015625 +2021-03-26 08:48:52,131 epoch 52 - iter 4/25 - loss 2.69134307 - samples/sec: 121.64 - lr: 0.015625 +2021-03-26 08:48:53,119 epoch 52 - iter 6/25 - loss 2.61419793 - samples/sec: 129.66 - lr: 0.015625 +2021-03-26 08:48:54,174 epoch 52 - iter 8/25 - loss 2.58930629 - samples/sec: 121.62 - lr: 0.015625 +2021-03-26 08:48:55,175 epoch 52 - iter 10/25 - loss 2.53519647 - samples/sec: 128.08 - lr: 0.015625 +2021-03-26 08:48:56,147 epoch 52 - iter 12/25 - loss 2.49006718 - samples/sec: 131.99 - lr: 0.015625 +2021-03-26 08:48:57,092 epoch 52 - iter 14/25 - loss 2.45591760 - samples/sec: 135.62 - lr: 0.015625 +2021-03-26 08:48:58,012 epoch 52 - iter 16/25 - loss 2.49299569 - samples/sec: 139.27 - lr: 0.015625 +2021-03-26 08:48:58,930 epoch 52 - iter 18/25 - loss 2.45735917 - samples/sec: 139.65 - lr: 0.015625 +2021-03-26 08:48:59,851 epoch 52 - iter 20/25 - loss 2.55742084 - samples/sec: 139.20 - lr: 0.015625 +2021-03-26 08:49:00,835 epoch 52 - iter 22/25 - loss 2.54598535 - samples/sec: 130.34 - lr: 0.015625 +2021-03-26 08:49:01,797 epoch 52 - iter 24/25 - loss 2.50375176 - samples/sec: 133.30 - lr: 0.015625 +2021-03-26 08:49:02,235 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:49:02,236 EPOCH 52 done: loss 2.5224 - lr 0.0156250 +2021-03-26 08:49:02,989 DEV : loss 6.722785949707031 - score 0.9084 +2021-03-26 08:49:03,006 BAD EPOCHS (no improvement): 2 +2021-03-26 08:49:03,007 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:49:04,049 epoch 53 - iter 2/25 - loss 3.04602754 - samples/sec: 123.05 - lr: 0.015625 +2021-03-26 08:49:05,091 epoch 53 - iter 4/25 - loss 2.71588653 - samples/sec: 122.99 - lr: 0.015625 +2021-03-26 08:49:06,081 epoch 53 - iter 6/25 - loss 2.78629927 - samples/sec: 129.50 - lr: 0.015625 +2021-03-26 08:49:07,034 epoch 53 - iter 8/25 - loss 2.62624462 - samples/sec: 134.40 - lr: 0.015625 +2021-03-26 08:49:07,922 epoch 53 - iter 10/25 - loss 2.63786384 - samples/sec: 144.55 - lr: 0.015625 +2021-03-26 08:49:08,890 epoch 53 - iter 12/25 - loss 2.64222335 - samples/sec: 132.34 - lr: 0.015625 +2021-03-26 08:49:09,854 epoch 53 - iter 14/25 - loss 2.52385678 - samples/sec: 132.92 - lr: 0.015625 +2021-03-26 08:49:10,766 epoch 53 - iter 16/25 - loss 2.48844558 - samples/sec: 140.64 - lr: 0.015625 +2021-03-26 08:49:11,721 epoch 53 - iter 18/25 - loss 2.53972616 - samples/sec: 134.27 - lr: 0.015625 +2021-03-26 08:49:12,706 epoch 53 - iter 20/25 - loss 2.49504396 - samples/sec: 130.12 - lr: 0.015625 +2021-03-26 08:49:13,751 epoch 53 - iter 22/25 - loss 2.45131991 - samples/sec: 122.71 - lr: 0.015625 +2021-03-26 08:49:14,714 epoch 53 - iter 24/25 - loss 2.45370945 - samples/sec: 133.00 - lr: 0.015625 +2021-03-26 08:49:15,148 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:49:15,150 EPOCH 53 done: loss 2.4649 - lr 0.0156250 +2021-03-26 08:49:15,923 DEV : loss 6.713016033172607 - score 0.909 +2021-03-26 08:49:15,947 BAD EPOCHS (no improvement): 3 +2021-03-26 08:49:15,948 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:49:18,013 epoch 54 - iter 2/25 - loss 2.33827412 - samples/sec: 62.05 - lr: 0.015625 +2021-03-26 08:49:18,981 epoch 54 - iter 4/25 - loss 2.44314307 - samples/sec: 132.48 - lr: 0.015625 +2021-03-26 08:49:19,927 epoch 54 - iter 6/25 - loss 2.50450575 - samples/sec: 135.44 - lr: 0.015625 +2021-03-26 08:49:20,884 epoch 54 - iter 8/25 - loss 2.38388234 - samples/sec: 134.06 - lr: 0.015625 +2021-03-26 08:49:21,837 epoch 54 - iter 10/25 - loss 2.39710271 - samples/sec: 134.53 - lr: 0.015625 +2021-03-26 08:49:22,752 epoch 54 - iter 12/25 - loss 2.38589450 - samples/sec: 140.15 - lr: 0.015625 +2021-03-26 08:49:23,626 epoch 54 - iter 14/25 - loss 2.45272618 - samples/sec: 146.57 - lr: 0.015625 +2021-03-26 08:49:24,625 epoch 54 - iter 16/25 - loss 2.43205299 - samples/sec: 128.45 - lr: 0.015625 +2021-03-26 08:49:25,648 epoch 54 - iter 18/25 - loss 2.44015885 - samples/sec: 125.34 - lr: 0.015625 +2021-03-26 08:49:26,657 epoch 54 - iter 20/25 - loss 2.40970163 - samples/sec: 127.10 - lr: 0.015625 +2021-03-26 08:49:27,652 epoch 54 - iter 22/25 - loss 2.42787213 - samples/sec: 128.88 - lr: 0.015625 +2021-03-26 08:49:28,634 epoch 54 - iter 24/25 - loss 2.45196098 - samples/sec: 130.42 - lr: 0.015625 +2021-03-26 08:49:29,053 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:49:29,054 EPOCH 54 done: loss 2.4558 - lr 0.0156250 +2021-03-26 08:49:29,825 DEV : loss 6.716770172119141 - score 0.9088 +2021-03-26 08:49:29,849 BAD EPOCHS (no improvement): 4 +2021-03-26 08:49:29,850 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:49:30,871 epoch 55 - iter 2/25 - loss 2.50495231 - samples/sec: 125.59 - lr: 0.007812 +2021-03-26 08:49:31,880 epoch 55 - iter 4/25 - loss 2.56151509 - samples/sec: 127.10 - lr: 0.007812 +2021-03-26 08:49:32,794 epoch 55 - iter 6/25 - loss 2.51272758 - samples/sec: 140.33 - lr: 0.007812 +2021-03-26 08:49:33,786 epoch 55 - iter 8/25 - loss 2.48402715 - samples/sec: 129.16 - lr: 0.007812 +2021-03-26 08:49:34,691 epoch 55 - iter 10/25 - loss 2.43414333 - samples/sec: 141.76 - lr: 0.007812 +2021-03-26 08:49:35,643 epoch 55 - iter 12/25 - loss 2.41734570 - samples/sec: 134.57 - lr: 0.007812 +2021-03-26 08:49:36,656 epoch 55 - iter 14/25 - loss 2.44474280 - samples/sec: 126.55 - lr: 0.007812 +2021-03-26 08:49:37,660 epoch 55 - iter 16/25 - loss 2.44347511 - samples/sec: 127.73 - lr: 0.007812 +2021-03-26 08:49:38,662 epoch 55 - iter 18/25 - loss 2.40479814 - samples/sec: 128.05 - lr: 0.007812 +2021-03-26 08:49:39,680 epoch 55 - iter 20/25 - loss 2.41784536 - samples/sec: 125.90 - lr: 0.007812 +2021-03-26 08:49:40,728 epoch 55 - iter 22/25 - loss 2.42849638 - samples/sec: 122.33 - lr: 0.007812 +2021-03-26 08:49:41,808 epoch 55 - iter 24/25 - loss 2.42535428 - samples/sec: 118.68 - lr: 0.007812 +2021-03-26 08:49:42,208 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:49:42,209 EPOCH 55 done: loss 2.4129 - lr 0.0078125 +2021-03-26 08:49:42,994 DEV : loss 6.715353965759277 - score 0.9082 +2021-03-26 08:49:43,014 BAD EPOCHS (no improvement): 1 +2021-03-26 08:49:43,015 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:49:44,063 epoch 56 - iter 2/25 - loss 2.41294694 - samples/sec: 122.81 - lr: 0.007812 +2021-03-26 08:49:45,012 epoch 56 - iter 4/25 - loss 2.40887398 - samples/sec: 135.11 - lr: 0.007812 +2021-03-26 08:49:45,883 epoch 56 - iter 6/25 - loss 2.53534293 - samples/sec: 147.28 - lr: 0.007812 +2021-03-26 08:49:46,844 epoch 56 - iter 8/25 - loss 2.63588372 - samples/sec: 133.35 - lr: 0.007812 +2021-03-26 08:49:47,785 epoch 56 - iter 10/25 - loss 2.57245722 - samples/sec: 136.30 - lr: 0.007812 +2021-03-26 08:49:48,681 epoch 56 - iter 12/25 - loss 2.50227737 - samples/sec: 142.97 - lr: 0.007812 +2021-03-26 08:49:49,645 epoch 56 - iter 14/25 - loss 2.51777281 - samples/sec: 133.06 - lr: 0.007812 +2021-03-26 08:49:50,669 epoch 56 - iter 16/25 - loss 2.53424157 - samples/sec: 125.18 - lr: 0.007812 +2021-03-26 08:49:51,625 epoch 56 - iter 18/25 - loss 2.51135084 - samples/sec: 134.01 - lr: 0.007812 +2021-03-26 08:49:52,655 epoch 56 - iter 20/25 - loss 2.54296572 - samples/sec: 124.62 - lr: 0.007812 +2021-03-26 08:49:53,643 epoch 56 - iter 22/25 - loss 2.59556309 - samples/sec: 129.69 - lr: 0.007812 +2021-03-26 08:49:54,662 epoch 56 - iter 24/25 - loss 2.62635016 - samples/sec: 125.76 - lr: 0.007812 +2021-03-26 08:49:55,033 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:49:55,033 EPOCH 56 done: loss 2.5899 - lr 0.0078125 +2021-03-26 08:49:55,798 DEV : loss 6.716400146484375 - score 0.9086 +2021-03-26 08:49:55,826 BAD EPOCHS (no improvement): 2 +2021-03-26 08:49:55,827 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:49:56,862 epoch 57 - iter 2/25 - loss 2.28633606 - samples/sec: 123.85 - lr: 0.007812 +2021-03-26 08:49:57,800 epoch 57 - iter 4/25 - loss 2.49695295 - samples/sec: 136.61 - lr: 0.007812 +2021-03-26 08:49:59,067 epoch 57 - iter 6/25 - loss 2.40712937 - samples/sec: 101.19 - lr: 0.007812 +2021-03-26 08:50:00,242 epoch 57 - iter 8/25 - loss 2.43323019 - samples/sec: 109.06 - lr: 0.007812 +2021-03-26 08:50:01,234 epoch 57 - iter 10/25 - loss 2.32638550 - samples/sec: 129.28 - lr: 0.007812 +2021-03-26 08:50:02,196 epoch 57 - iter 12/25 - loss 2.33791218 - samples/sec: 133.35 - lr: 0.007812 +2021-03-26 08:50:03,223 epoch 57 - iter 14/25 - loss 2.34867431 - samples/sec: 124.76 - lr: 0.007812 +2021-03-26 08:50:04,196 epoch 57 - iter 16/25 - loss 2.39169888 - samples/sec: 131.79 - lr: 0.007812 +2021-03-26 08:50:05,207 epoch 57 - iter 18/25 - loss 2.39465144 - samples/sec: 126.77 - lr: 0.007812 +2021-03-26 08:50:06,121 epoch 57 - iter 20/25 - loss 2.40665802 - samples/sec: 140.32 - lr: 0.007812 +2021-03-26 08:50:07,032 epoch 57 - iter 22/25 - loss 2.43683068 - samples/sec: 140.71 - lr: 0.007812 +2021-03-26 08:50:08,018 epoch 57 - iter 24/25 - loss 2.47992485 - samples/sec: 130.02 - lr: 0.007812 +2021-03-26 08:50:08,401 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:50:08,401 EPOCH 57 done: loss 2.4940 - lr 0.0078125 +2021-03-26 08:50:09,127 DEV : loss 6.717229843139648 - score 0.909 +2021-03-26 08:50:09,151 BAD EPOCHS (no improvement): 3 +2021-03-26 08:50:09,152 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:50:10,028 epoch 58 - iter 2/25 - loss 2.40251827 - samples/sec: 146.45 - lr: 0.007812 +2021-03-26 08:50:10,997 epoch 58 - iter 4/25 - loss 2.45009959 - samples/sec: 132.24 - lr: 0.007812 +2021-03-26 08:50:12,049 epoch 58 - iter 6/25 - loss 2.36056856 - samples/sec: 121.79 - lr: 0.007812 +2021-03-26 08:50:12,988 epoch 58 - iter 8/25 - loss 2.26037887 - samples/sec: 136.63 - lr: 0.007812 +2021-03-26 08:50:13,963 epoch 58 - iter 10/25 - loss 2.39980597 - samples/sec: 131.41 - lr: 0.007812 +2021-03-26 08:50:14,931 epoch 58 - iter 12/25 - loss 2.34398366 - samples/sec: 132.49 - lr: 0.007812 +2021-03-26 08:50:15,908 epoch 58 - iter 14/25 - loss 2.33637424 - samples/sec: 131.18 - lr: 0.007812 +2021-03-26 08:50:17,012 epoch 58 - iter 16/25 - loss 2.41296620 - samples/sec: 116.08 - lr: 0.007812 +2021-03-26 08:50:17,980 epoch 58 - iter 18/25 - loss 2.45545569 - samples/sec: 132.46 - lr: 0.007812 +2021-03-26 08:50:18,835 epoch 58 - iter 20/25 - loss 2.47555558 - samples/sec: 149.95 - lr: 0.007812 +2021-03-26 08:50:19,849 epoch 58 - iter 22/25 - loss 2.50793289 - samples/sec: 126.51 - lr: 0.007812 +2021-03-26 08:50:20,889 epoch 58 - iter 24/25 - loss 2.48611053 - samples/sec: 123.29 - lr: 0.007812 +2021-03-26 08:50:21,350 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:50:21,350 EPOCH 58 done: loss 2.5119 - lr 0.0078125 +2021-03-26 08:50:22,114 DEV : loss 6.7159423828125 - score 0.9094 +2021-03-26 08:50:22,138 BAD EPOCHS (no improvement): 4 +2021-03-26 08:50:22,138 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:50:23,080 epoch 59 - iter 2/25 - loss 2.81640446 - samples/sec: 136.21 - lr: 0.003906 +2021-03-26 08:50:24,095 epoch 59 - iter 4/25 - loss 2.52743006 - samples/sec: 126.39 - lr: 0.003906 +2021-03-26 08:50:25,130 epoch 59 - iter 6/25 - loss 2.78575460 - samples/sec: 123.80 - lr: 0.003906 +2021-03-26 08:50:26,130 epoch 59 - iter 8/25 - loss 2.84091517 - samples/sec: 128.19 - lr: 0.003906 +2021-03-26 08:50:27,019 epoch 59 - iter 10/25 - loss 2.71048800 - samples/sec: 144.21 - lr: 0.003906 +2021-03-26 08:50:28,014 epoch 59 - iter 12/25 - loss 2.75860839 - samples/sec: 128.85 - lr: 0.003906 +2021-03-26 08:50:29,113 epoch 59 - iter 14/25 - loss 2.67045104 - samples/sec: 116.69 - lr: 0.003906 +2021-03-26 08:50:30,088 epoch 59 - iter 16/25 - loss 2.63434067 - samples/sec: 131.50 - lr: 0.003906 +2021-03-26 08:50:31,289 epoch 59 - iter 18/25 - loss 2.56707877 - samples/sec: 106.68 - lr: 0.003906 +2021-03-26 08:50:32,303 epoch 59 - iter 20/25 - loss 2.47917590 - samples/sec: 126.46 - lr: 0.003906 +2021-03-26 08:50:33,245 epoch 59 - iter 22/25 - loss 2.48207686 - samples/sec: 136.08 - lr: 0.003906 +2021-03-26 08:50:34,161 epoch 59 - iter 24/25 - loss 2.48821341 - samples/sec: 139.86 - lr: 0.003906 +2021-03-26 08:50:34,563 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:50:34,563 EPOCH 59 done: loss 2.5272 - lr 0.0039062 +2021-03-26 08:50:35,305 DEV : loss 6.716526985168457 - score 0.9098 +2021-03-26 08:50:35,328 BAD EPOCHS (no improvement): 1 +2021-03-26 08:50:35,329 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:50:36,185 epoch 60 - iter 2/25 - loss 2.27116865 - samples/sec: 149.83 - lr: 0.003906 +2021-03-26 08:50:37,114 epoch 60 - iter 4/25 - loss 2.25008342 - samples/sec: 138.07 - lr: 0.003906 +2021-03-26 08:50:38,045 epoch 60 - iter 6/25 - loss 2.29013628 - samples/sec: 137.75 - lr: 0.003906 +2021-03-26 08:50:39,027 epoch 60 - iter 8/25 - loss 2.38840453 - samples/sec: 130.65 - lr: 0.003906 +2021-03-26 08:50:39,969 epoch 60 - iter 10/25 - loss 2.30501388 - samples/sec: 136.09 - lr: 0.003906 +2021-03-26 08:50:41,052 epoch 60 - iter 12/25 - loss 2.43222015 - samples/sec: 118.32 - lr: 0.003906 +2021-03-26 08:50:42,075 epoch 60 - iter 14/25 - loss 2.48389046 - samples/sec: 125.40 - lr: 0.003906 +2021-03-26 08:50:43,007 epoch 60 - iter 16/25 - loss 2.47347818 - samples/sec: 137.56 - lr: 0.003906 +2021-03-26 08:50:44,112 epoch 60 - iter 18/25 - loss 2.44074412 - samples/sec: 115.92 - lr: 0.003906 +2021-03-26 08:50:45,019 epoch 60 - iter 20/25 - loss 2.43466492 - samples/sec: 141.49 - lr: 0.003906 +2021-03-26 08:50:46,059 epoch 60 - iter 22/25 - loss 2.43043199 - samples/sec: 123.26 - lr: 0.003906 +2021-03-26 08:50:47,074 epoch 60 - iter 24/25 - loss 2.41566441 - samples/sec: 126.19 - lr: 0.003906 +2021-03-26 08:50:47,530 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:50:47,531 EPOCH 60 done: loss 2.4161 - lr 0.0039062 +2021-03-26 08:50:48,287 DEV : loss 6.71820592880249 - score 0.9098 +2021-03-26 08:50:48,303 BAD EPOCHS (no improvement): 2 +2021-03-26 08:50:48,304 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:50:49,290 epoch 61 - iter 2/25 - loss 2.58153760 - samples/sec: 130.06 - lr: 0.003906 +2021-03-26 08:50:50,258 epoch 61 - iter 4/25 - loss 2.38109404 - samples/sec: 132.41 - lr: 0.003906 +2021-03-26 08:50:51,233 epoch 61 - iter 6/25 - loss 2.34258546 - samples/sec: 131.48 - lr: 0.003906 +2021-03-26 08:50:52,241 epoch 61 - iter 8/25 - loss 2.38017832 - samples/sec: 127.10 - lr: 0.003906 +2021-03-26 08:50:53,260 epoch 61 - iter 10/25 - loss 2.41248423 - samples/sec: 125.82 - lr: 0.003906 +2021-03-26 08:50:54,172 epoch 61 - iter 12/25 - loss 2.40937188 - samples/sec: 140.53 - lr: 0.003906 +2021-03-26 08:50:55,172 epoch 61 - iter 14/25 - loss 2.38282467 - samples/sec: 128.25 - lr: 0.003906 +2021-03-26 08:50:56,381 epoch 61 - iter 16/25 - loss 2.39176605 - samples/sec: 106.04 - lr: 0.003906 +2021-03-26 08:50:57,724 epoch 61 - iter 18/25 - loss 2.32950047 - samples/sec: 95.41 - lr: 0.003906 +2021-03-26 08:50:58,859 epoch 61 - iter 20/25 - loss 2.30976757 - samples/sec: 112.95 - lr: 0.003906 +2021-03-26 08:50:59,864 epoch 61 - iter 22/25 - loss 2.32149645 - samples/sec: 127.64 - lr: 0.003906 +2021-03-26 08:51:00,767 epoch 61 - iter 24/25 - loss 2.33463271 - samples/sec: 142.03 - lr: 0.003906 +2021-03-26 08:51:01,175 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:51:01,176 EPOCH 61 done: loss 2.3059 - lr 0.0039062 +2021-03-26 08:51:01,930 DEV : loss 6.716128349304199 - score 0.9098 +2021-03-26 08:51:01,954 BAD EPOCHS (no improvement): 3 +2021-03-26 08:51:01,954 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:51:02,905 epoch 62 - iter 2/25 - loss 2.21142137 - samples/sec: 134.98 - lr: 0.003906 +2021-03-26 08:51:04,081 epoch 62 - iter 4/25 - loss 2.45809269 - samples/sec: 108.99 - lr: 0.003906 +2021-03-26 08:51:05,191 epoch 62 - iter 6/25 - loss 2.34228293 - samples/sec: 115.47 - lr: 0.003906 +2021-03-26 08:51:06,278 epoch 62 - iter 8/25 - loss 2.29517706 - samples/sec: 117.93 - lr: 0.003906 +2021-03-26 08:51:07,228 epoch 62 - iter 10/25 - loss 2.27721502 - samples/sec: 135.01 - lr: 0.003906 +2021-03-26 08:51:08,291 epoch 62 - iter 12/25 - loss 2.28681889 - samples/sec: 120.55 - lr: 0.003906 +2021-03-26 08:51:09,235 epoch 62 - iter 14/25 - loss 2.37437183 - samples/sec: 135.69 - lr: 0.003906 +2021-03-26 08:51:10,159 epoch 62 - iter 16/25 - loss 2.32531175 - samples/sec: 138.77 - lr: 0.003906 +2021-03-26 08:51:11,041 epoch 62 - iter 18/25 - loss 2.32343723 - samples/sec: 145.35 - lr: 0.003906 +2021-03-26 08:51:12,045 epoch 62 - iter 20/25 - loss 2.34342856 - samples/sec: 127.75 - lr: 0.003906 +2021-03-26 08:51:13,083 epoch 62 - iter 22/25 - loss 2.34453206 - samples/sec: 123.48 - lr: 0.003906 +2021-03-26 08:51:14,064 epoch 62 - iter 24/25 - loss 2.34612409 - samples/sec: 130.70 - lr: 0.003906 +2021-03-26 08:51:14,469 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:51:14,470 EPOCH 62 done: loss 2.3247 - lr 0.0039062 +2021-03-26 08:51:15,197 DEV : loss 6.719080924987793 - score 0.9098 +2021-03-26 08:51:15,219 BAD EPOCHS (no improvement): 4 +2021-03-26 08:51:15,220 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:51:16,238 epoch 63 - iter 2/25 - loss 3.00047922 - samples/sec: 125.89 - lr: 0.001953 +2021-03-26 08:51:17,209 epoch 63 - iter 4/25 - loss 2.73255146 - samples/sec: 132.08 - lr: 0.001953 +2021-03-26 08:51:18,118 epoch 63 - iter 6/25 - loss 2.49330461 - samples/sec: 141.03 - lr: 0.001953 +2021-03-26 08:51:19,051 epoch 63 - iter 8/25 - loss 2.45386675 - samples/sec: 137.39 - lr: 0.001953 +2021-03-26 08:51:20,050 epoch 63 - iter 10/25 - loss 2.39355280 - samples/sec: 128.39 - lr: 0.001953 +2021-03-26 08:51:21,009 epoch 63 - iter 12/25 - loss 2.44529299 - samples/sec: 133.62 - lr: 0.001953 +2021-03-26 08:51:22,162 epoch 63 - iter 14/25 - loss 2.46328975 - samples/sec: 111.17 - lr: 0.001953 +2021-03-26 08:51:23,235 epoch 63 - iter 16/25 - loss 2.51815675 - samples/sec: 119.52 - lr: 0.001953 +2021-03-26 08:51:24,479 epoch 63 - iter 18/25 - loss 2.49993791 - samples/sec: 103.00 - lr: 0.001953 +2021-03-26 08:51:25,463 epoch 63 - iter 20/25 - loss 2.52069933 - samples/sec: 130.32 - lr: 0.001953 +2021-03-26 08:51:26,424 epoch 63 - iter 22/25 - loss 2.48497365 - samples/sec: 133.53 - lr: 0.001953 +2021-03-26 08:51:27,384 epoch 63 - iter 24/25 - loss 2.48816618 - samples/sec: 133.46 - lr: 0.001953 +2021-03-26 08:51:27,774 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:51:27,775 EPOCH 63 done: loss 2.5114 - lr 0.0019531 +2021-03-26 08:51:28,546 DEV : loss 6.721194267272949 - score 0.9098 +2021-03-26 08:51:28,570 BAD EPOCHS (no improvement): 1 +2021-03-26 08:51:28,571 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:51:29,485 epoch 64 - iter 2/25 - loss 2.62489450 - samples/sec: 140.44 - lr: 0.001953 +2021-03-26 08:51:30,494 epoch 64 - iter 4/25 - loss 2.66914499 - samples/sec: 127.01 - lr: 0.001953 +2021-03-26 08:51:31,437 epoch 64 - iter 6/25 - loss 2.67395210 - samples/sec: 135.90 - lr: 0.001953 +2021-03-26 08:51:32,494 epoch 64 - iter 8/25 - loss 2.64659414 - samples/sec: 121.28 - lr: 0.001953 +2021-03-26 08:51:33,489 epoch 64 - iter 10/25 - loss 2.68895016 - samples/sec: 128.80 - lr: 0.001953 +2021-03-26 08:51:34,379 epoch 64 - iter 12/25 - loss 2.62112544 - samples/sec: 144.18 - lr: 0.001953 +2021-03-26 08:51:35,342 epoch 64 - iter 14/25 - loss 2.57101262 - samples/sec: 133.02 - lr: 0.001953 +2021-03-26 08:51:36,297 epoch 64 - iter 16/25 - loss 2.49899342 - samples/sec: 134.28 - lr: 0.001953 +2021-03-26 08:51:37,310 epoch 64 - iter 18/25 - loss 2.52965344 - samples/sec: 126.56 - lr: 0.001953 +2021-03-26 08:51:38,334 epoch 64 - iter 20/25 - loss 2.55107920 - samples/sec: 125.16 - lr: 0.001953 +2021-03-26 08:51:39,345 epoch 64 - iter 22/25 - loss 2.56382647 - samples/sec: 126.83 - lr: 0.001953 +2021-03-26 08:51:40,347 epoch 64 - iter 24/25 - loss 2.57456375 - samples/sec: 127.89 - lr: 0.001953 +2021-03-26 08:51:40,832 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:51:40,833 EPOCH 64 done: loss 2.5673 - lr 0.0019531 +2021-03-26 08:51:41,594 DEV : loss 6.721752166748047 - score 0.909 +2021-03-26 08:51:41,613 BAD EPOCHS (no improvement): 2 +2021-03-26 08:51:41,614 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:51:42,619 epoch 65 - iter 2/25 - loss 2.50386107 - samples/sec: 127.54 - lr: 0.001953 +2021-03-26 08:51:43,773 epoch 65 - iter 4/25 - loss 2.74910355 - samples/sec: 111.07 - lr: 0.001953 +2021-03-26 08:51:44,784 epoch 65 - iter 6/25 - loss 2.68734121 - samples/sec: 126.74 - lr: 0.001953 +2021-03-26 08:51:45,784 epoch 65 - iter 8/25 - loss 2.60665485 - samples/sec: 128.36 - lr: 0.001953 +2021-03-26 08:51:46,777 epoch 65 - iter 10/25 - loss 2.51850870 - samples/sec: 129.11 - lr: 0.001953 +2021-03-26 08:51:47,798 epoch 65 - iter 12/25 - loss 2.43202922 - samples/sec: 125.61 - lr: 0.001953 +2021-03-26 08:51:48,717 epoch 65 - iter 14/25 - loss 2.41641241 - samples/sec: 139.37 - lr: 0.001953 +2021-03-26 08:51:49,681 epoch 65 - iter 16/25 - loss 2.39029852 - samples/sec: 133.10 - lr: 0.001953 +2021-03-26 08:51:50,621 epoch 65 - iter 18/25 - loss 2.37561301 - samples/sec: 136.34 - lr: 0.001953 +2021-03-26 08:51:51,591 epoch 65 - iter 20/25 - loss 2.38306705 - samples/sec: 132.05 - lr: 0.001953 +2021-03-26 08:51:52,450 epoch 65 - iter 22/25 - loss 2.36917746 - samples/sec: 149.20 - lr: 0.001953 +2021-03-26 08:51:53,395 epoch 65 - iter 24/25 - loss 2.41181322 - samples/sec: 135.74 - lr: 0.001953 +2021-03-26 08:51:53,816 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:51:53,817 EPOCH 65 done: loss 2.3659 - lr 0.0019531 +2021-03-26 08:51:54,576 DEV : loss 6.72037410736084 - score 0.9102 +2021-03-26 08:51:54,596 BAD EPOCHS (no improvement): 3 +2021-03-26 08:51:54,596 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:51:55,588 epoch 66 - iter 2/25 - loss 2.60044074 - samples/sec: 129.23 - lr: 0.001953 +2021-03-26 08:51:56,727 epoch 66 - iter 4/25 - loss 2.51240951 - samples/sec: 112.51 - lr: 0.001953 +2021-03-26 08:51:57,929 epoch 66 - iter 6/25 - loss 2.61460606 - samples/sec: 106.68 - lr: 0.001953 +2021-03-26 08:51:58,813 epoch 66 - iter 8/25 - loss 2.61795145 - samples/sec: 145.14 - lr: 0.001953 +2021-03-26 08:51:59,833 epoch 66 - iter 10/25 - loss 2.59454665 - samples/sec: 125.66 - lr: 0.001953 +2021-03-26 08:52:00,802 epoch 66 - iter 12/25 - loss 2.51743805 - samples/sec: 132.31 - lr: 0.001953 +2021-03-26 08:52:01,815 epoch 66 - iter 14/25 - loss 2.43958904 - samples/sec: 126.52 - lr: 0.001953 +2021-03-26 08:52:02,774 epoch 66 - iter 16/25 - loss 2.48947333 - samples/sec: 133.79 - lr: 0.001953 +2021-03-26 08:52:03,746 epoch 66 - iter 18/25 - loss 2.50290778 - samples/sec: 131.85 - lr: 0.001953 +2021-03-26 08:52:04,716 epoch 66 - iter 20/25 - loss 2.47735422 - samples/sec: 132.29 - lr: 0.001953 +2021-03-26 08:52:05,793 epoch 66 - iter 22/25 - loss 2.46050443 - samples/sec: 118.94 - lr: 0.001953 +2021-03-26 08:52:06,819 epoch 66 - iter 24/25 - loss 2.50004270 - samples/sec: 125.06 - lr: 0.001953 +2021-03-26 08:52:07,162 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:52:07,163 EPOCH 66 done: loss 2.4471 - lr 0.0019531 +2021-03-26 08:52:07,918 DEV : loss 6.720994472503662 - score 0.9098 +2021-03-26 08:52:07,946 BAD EPOCHS (no improvement): 4 +2021-03-26 08:52:07,946 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:52:08,972 epoch 67 - iter 2/25 - loss 2.64988303 - samples/sec: 125.06 - lr: 0.000977 +2021-03-26 08:52:09,913 epoch 67 - iter 4/25 - loss 2.85384834 - samples/sec: 136.19 - lr: 0.000977 +2021-03-26 08:52:10,838 epoch 67 - iter 6/25 - loss 2.66090802 - samples/sec: 138.57 - lr: 0.000977 +2021-03-26 08:52:11,772 epoch 67 - iter 8/25 - loss 2.55720612 - samples/sec: 137.20 - lr: 0.000977 +2021-03-26 08:52:12,793 epoch 67 - iter 10/25 - loss 2.50371974 - samples/sec: 125.63 - lr: 0.000977 +2021-03-26 08:52:13,719 epoch 67 - iter 12/25 - loss 2.53245260 - samples/sec: 138.44 - lr: 0.000977 +2021-03-26 08:52:14,723 epoch 67 - iter 14/25 - loss 2.57271474 - samples/sec: 127.82 - lr: 0.000977 +2021-03-26 08:52:15,701 epoch 67 - iter 16/25 - loss 2.54835680 - samples/sec: 131.10 - lr: 0.000977 +2021-03-26 08:52:16,666 epoch 67 - iter 18/25 - loss 2.52905520 - samples/sec: 132.85 - lr: 0.000977 +2021-03-26 08:52:17,592 epoch 67 - iter 20/25 - loss 2.48178655 - samples/sec: 138.58 - lr: 0.000977 +2021-03-26 08:52:18,570 epoch 67 - iter 22/25 - loss 2.43028183 - samples/sec: 130.98 - lr: 0.000977 +2021-03-26 08:52:19,685 epoch 67 - iter 24/25 - loss 2.45312720 - samples/sec: 115.01 - lr: 0.000977 +2021-03-26 08:52:20,159 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:52:20,160 EPOCH 67 done: loss 2.4251 - lr 0.0009766 +2021-03-26 08:52:20,941 DEV : loss 6.7206525802612305 - score 0.9098 +2021-03-26 08:52:20,966 BAD EPOCHS (no improvement): 1 +2021-03-26 08:52:20,967 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:52:21,929 epoch 68 - iter 2/25 - loss 2.50237751 - samples/sec: 133.25 - lr: 0.000977 +2021-03-26 08:52:22,910 epoch 68 - iter 4/25 - loss 2.11894467 - samples/sec: 130.79 - lr: 0.000977 +2021-03-26 08:52:23,931 epoch 68 - iter 6/25 - loss 2.22465249 - samples/sec: 125.95 - lr: 0.000977 +2021-03-26 08:52:24,939 epoch 68 - iter 8/25 - loss 2.28452024 - samples/sec: 127.10 - lr: 0.000977 +2021-03-26 08:52:25,963 epoch 68 - iter 10/25 - loss 2.31511095 - samples/sec: 125.17 - lr: 0.000977 +2021-03-26 08:52:26,908 epoch 68 - iter 12/25 - loss 2.33202116 - samples/sec: 135.67 - lr: 0.000977 +2021-03-26 08:52:27,909 epoch 68 - iter 14/25 - loss 2.34010000 - samples/sec: 128.20 - lr: 0.000977 +2021-03-26 08:52:28,933 epoch 68 - iter 16/25 - loss 2.38829292 - samples/sec: 125.21 - lr: 0.000977 +2021-03-26 08:52:29,959 epoch 68 - iter 18/25 - loss 2.34564919 - samples/sec: 124.92 - lr: 0.000977 +2021-03-26 08:52:30,970 epoch 68 - iter 20/25 - loss 2.32275848 - samples/sec: 126.74 - lr: 0.000977 +2021-03-26 08:52:31,947 epoch 68 - iter 22/25 - loss 2.33297563 - samples/sec: 131.31 - lr: 0.000977 +2021-03-26 08:52:32,813 epoch 68 - iter 24/25 - loss 2.30395630 - samples/sec: 148.08 - lr: 0.000977 +2021-03-26 08:52:33,185 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:52:33,185 EPOCH 68 done: loss 2.3152 - lr 0.0009766 +2021-03-26 08:52:33,926 DEV : loss 6.721766471862793 - score 0.9094 +2021-03-26 08:52:33,949 BAD EPOCHS (no improvement): 2 +2021-03-26 08:52:33,950 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:52:34,923 epoch 69 - iter 2/25 - loss 2.32102466 - samples/sec: 131.82 - lr: 0.000977 +2021-03-26 08:52:35,944 epoch 69 - iter 4/25 - loss 2.19986767 - samples/sec: 125.53 - lr: 0.000977 +2021-03-26 08:52:37,137 epoch 69 - iter 6/25 - loss 2.22192717 - samples/sec: 107.50 - lr: 0.000977 +2021-03-26 08:52:38,157 epoch 69 - iter 8/25 - loss 2.37492293 - samples/sec: 125.69 - lr: 0.000977 +2021-03-26 08:52:39,150 epoch 69 - iter 10/25 - loss 2.43023714 - samples/sec: 129.12 - lr: 0.000977 +2021-03-26 08:52:40,154 epoch 69 - iter 12/25 - loss 2.35289692 - samples/sec: 127.75 - lr: 0.000977 +2021-03-26 08:52:41,286 epoch 69 - iter 14/25 - loss 2.35287110 - samples/sec: 113.17 - lr: 0.000977 +2021-03-26 08:52:42,298 epoch 69 - iter 16/25 - loss 2.34908857 - samples/sec: 126.83 - lr: 0.000977 +2021-03-26 08:52:43,292 epoch 69 - iter 18/25 - loss 2.43797569 - samples/sec: 128.98 - lr: 0.000977 +2021-03-26 08:52:44,154 epoch 69 - iter 20/25 - loss 2.44098268 - samples/sec: 148.57 - lr: 0.000977 +2021-03-26 08:52:45,230 epoch 69 - iter 22/25 - loss 2.44536105 - samples/sec: 119.21 - lr: 0.000977 +2021-03-26 08:52:46,184 epoch 69 - iter 24/25 - loss 2.45193523 - samples/sec: 134.34 - lr: 0.000977 +2021-03-26 08:52:46,553 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:52:46,554 EPOCH 69 done: loss 2.4442 - lr 0.0009766 +2021-03-26 08:52:47,314 DEV : loss 6.722614288330078 - score 0.9094 +2021-03-26 08:52:47,341 BAD EPOCHS (no improvement): 3 +2021-03-26 08:52:47,342 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:52:48,216 epoch 70 - iter 2/25 - loss 2.31417513 - samples/sec: 146.87 - lr: 0.000977 +2021-03-26 08:52:49,169 epoch 70 - iter 4/25 - loss 2.40965217 - samples/sec: 134.51 - lr: 0.000977 +2021-03-26 08:52:50,170 epoch 70 - iter 6/25 - loss 2.42630080 - samples/sec: 128.04 - lr: 0.000977 +2021-03-26 08:52:51,162 epoch 70 - iter 8/25 - loss 2.35043907 - samples/sec: 129.32 - lr: 0.000977 +2021-03-26 08:52:52,096 epoch 70 - iter 10/25 - loss 2.41967034 - samples/sec: 137.19 - lr: 0.000977 +2021-03-26 08:52:53,180 epoch 70 - iter 12/25 - loss 2.37920668 - samples/sec: 118.21 - lr: 0.000977 +2021-03-26 08:52:54,122 epoch 70 - iter 14/25 - loss 2.41693105 - samples/sec: 136.07 - lr: 0.000977 +2021-03-26 08:52:55,081 epoch 70 - iter 16/25 - loss 2.46385759 - samples/sec: 133.75 - lr: 0.000977 +2021-03-26 08:52:56,056 epoch 70 - iter 18/25 - loss 2.49344937 - samples/sec: 131.49 - lr: 0.000977 +2021-03-26 08:52:56,987 epoch 70 - iter 20/25 - loss 2.46795544 - samples/sec: 137.77 - lr: 0.000977 +2021-03-26 08:52:57,961 epoch 70 - iter 22/25 - loss 2.43951023 - samples/sec: 131.56 - lr: 0.000977 +2021-03-26 08:52:59,009 epoch 70 - iter 24/25 - loss 2.45959963 - samples/sec: 122.26 - lr: 0.000977 +2021-03-26 08:52:59,439 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:52:59,440 EPOCH 70 done: loss 2.4916 - lr 0.0009766 +2021-03-26 08:53:00,188 DEV : loss 6.721915245056152 - score 0.9094 +2021-03-26 08:53:00,204 BAD EPOCHS (no improvement): 4 +2021-03-26 08:53:00,205 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:53:01,251 epoch 71 - iter 2/25 - loss 2.93990111 - samples/sec: 122.48 - lr: 0.000488 +2021-03-26 08:53:02,230 epoch 71 - iter 4/25 - loss 2.70402431 - samples/sec: 131.10 - lr: 0.000488 +2021-03-26 08:53:03,145 epoch 71 - iter 6/25 - loss 2.52367194 - samples/sec: 140.12 - lr: 0.000488 +2021-03-26 08:53:04,058 epoch 71 - iter 8/25 - loss 2.46390080 - samples/sec: 140.35 - lr: 0.000488 +2021-03-26 08:53:05,079 epoch 71 - iter 10/25 - loss 2.46541066 - samples/sec: 125.51 - lr: 0.000488 +2021-03-26 08:53:06,070 epoch 71 - iter 12/25 - loss 2.41675409 - samples/sec: 129.43 - lr: 0.000488 +2021-03-26 08:53:07,068 epoch 71 - iter 14/25 - loss 2.40485142 - samples/sec: 128.34 - lr: 0.000488 +2021-03-26 08:53:08,182 epoch 71 - iter 16/25 - loss 2.39172129 - samples/sec: 115.13 - lr: 0.000488 +2021-03-26 08:53:09,358 epoch 71 - iter 18/25 - loss 2.42713354 - samples/sec: 108.98 - lr: 0.000488 +2021-03-26 08:53:10,436 epoch 71 - iter 20/25 - loss 2.43891806 - samples/sec: 118.88 - lr: 0.000488 +2021-03-26 08:53:11,471 epoch 71 - iter 22/25 - loss 2.41635902 - samples/sec: 123.89 - lr: 0.000488 +2021-03-26 08:53:12,392 epoch 71 - iter 24/25 - loss 2.41604218 - samples/sec: 139.07 - lr: 0.000488 +2021-03-26 08:53:12,808 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:53:12,808 EPOCH 71 done: loss 2.4220 - lr 0.0004883 +2021-03-26 08:53:13,571 DEV : loss 6.722039699554443 - score 0.9098 +2021-03-26 08:53:13,594 BAD EPOCHS (no improvement): 1 +2021-03-26 08:53:13,594 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:53:14,545 epoch 72 - iter 2/25 - loss 2.27463222 - samples/sec: 134.91 - lr: 0.000488 +2021-03-26 08:53:15,464 epoch 72 - iter 4/25 - loss 2.16546875 - samples/sec: 139.50 - lr: 0.000488 +2021-03-26 08:53:16,499 epoch 72 - iter 6/25 - loss 2.27853771 - samples/sec: 123.95 - lr: 0.000488 +2021-03-26 08:53:17,515 epoch 72 - iter 8/25 - loss 2.27974728 - samples/sec: 126.14 - lr: 0.000488 +2021-03-26 08:53:18,556 epoch 72 - iter 10/25 - loss 2.32490218 - samples/sec: 123.11 - lr: 0.000488 +2021-03-26 08:53:19,579 epoch 72 - iter 12/25 - loss 2.34221594 - samples/sec: 125.28 - lr: 0.000488 +2021-03-26 08:53:20,592 epoch 72 - iter 14/25 - loss 2.33135453 - samples/sec: 126.62 - lr: 0.000488 +2021-03-26 08:53:21,578 epoch 72 - iter 16/25 - loss 2.36180919 - samples/sec: 129.96 - lr: 0.000488 +2021-03-26 08:53:22,646 epoch 72 - iter 18/25 - loss 2.36696060 - samples/sec: 119.98 - lr: 0.000488 +2021-03-26 08:53:23,788 epoch 72 - iter 20/25 - loss 2.34693076 - samples/sec: 112.30 - lr: 0.000488 +2021-03-26 08:53:24,843 epoch 72 - iter 22/25 - loss 2.38968690 - samples/sec: 121.54 - lr: 0.000488 +2021-03-26 08:53:25,993 epoch 72 - iter 24/25 - loss 2.42801057 - samples/sec: 111.46 - lr: 0.000488 +2021-03-26 08:53:26,411 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:53:26,412 EPOCH 72 done: loss 2.4201 - lr 0.0004883 +2021-03-26 08:53:27,138 DEV : loss 6.722582817077637 - score 0.9094 +2021-03-26 08:53:27,161 BAD EPOCHS (no improvement): 2 +2021-03-26 08:53:27,162 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:53:28,166 epoch 73 - iter 2/25 - loss 2.58827686 - samples/sec: 127.78 - lr: 0.000488 +2021-03-26 08:53:29,233 epoch 73 - iter 4/25 - loss 2.46295500 - samples/sec: 120.18 - lr: 0.000488 +2021-03-26 08:53:30,238 epoch 73 - iter 6/25 - loss 2.42940970 - samples/sec: 127.65 - lr: 0.000488 +2021-03-26 08:53:31,226 epoch 73 - iter 8/25 - loss 2.37589297 - samples/sec: 129.86 - lr: 0.000488 +2021-03-26 08:53:32,190 epoch 73 - iter 10/25 - loss 2.33973050 - samples/sec: 133.05 - lr: 0.000488 +2021-03-26 08:53:33,204 epoch 73 - iter 12/25 - loss 2.38512383 - samples/sec: 126.40 - lr: 0.000488 +2021-03-26 08:53:34,377 epoch 73 - iter 14/25 - loss 2.35500358 - samples/sec: 109.17 - lr: 0.000488 +2021-03-26 08:53:35,339 epoch 73 - iter 16/25 - loss 2.44799797 - samples/sec: 133.37 - lr: 0.000488 +2021-03-26 08:53:36,293 epoch 73 - iter 18/25 - loss 2.42572118 - samples/sec: 134.30 - lr: 0.000488 +2021-03-26 08:53:37,246 epoch 73 - iter 20/25 - loss 2.40007474 - samples/sec: 134.59 - lr: 0.000488 +2021-03-26 08:53:38,217 epoch 73 - iter 22/25 - loss 2.39740131 - samples/sec: 132.04 - lr: 0.000488 +2021-03-26 08:53:39,237 epoch 73 - iter 24/25 - loss 2.40759586 - samples/sec: 125.56 - lr: 0.000488 +2021-03-26 08:53:39,595 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:53:39,596 EPOCH 73 done: loss 2.4429 - lr 0.0004883 +2021-03-26 08:53:40,362 DEV : loss 6.72260856628418 - score 0.9094 +2021-03-26 08:53:40,387 BAD EPOCHS (no improvement): 3 +2021-03-26 08:53:40,388 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:53:41,387 epoch 74 - iter 2/25 - loss 2.16657722 - samples/sec: 128.27 - lr: 0.000488 +2021-03-26 08:53:42,381 epoch 74 - iter 4/25 - loss 2.09330899 - samples/sec: 129.02 - lr: 0.000488 +2021-03-26 08:53:43,306 epoch 74 - iter 6/25 - loss 2.12494469 - samples/sec: 138.70 - lr: 0.000488 +2021-03-26 08:53:44,271 epoch 74 - iter 8/25 - loss 2.17297184 - samples/sec: 132.92 - lr: 0.000488 +2021-03-26 08:53:45,324 epoch 74 - iter 10/25 - loss 2.23438246 - samples/sec: 121.75 - lr: 0.000488 +2021-03-26 08:53:46,306 epoch 74 - iter 12/25 - loss 2.32831556 - samples/sec: 130.51 - lr: 0.000488 +2021-03-26 08:53:47,318 epoch 74 - iter 14/25 - loss 2.38299196 - samples/sec: 126.63 - lr: 0.000488 +2021-03-26 08:53:48,329 epoch 74 - iter 16/25 - loss 2.35490888 - samples/sec: 127.34 - lr: 0.000488 +2021-03-26 08:53:49,330 epoch 74 - iter 18/25 - loss 2.33497572 - samples/sec: 128.13 - lr: 0.000488 +2021-03-26 08:53:50,309 epoch 74 - iter 20/25 - loss 2.32606256 - samples/sec: 130.97 - lr: 0.000488 +2021-03-26 08:53:51,387 epoch 74 - iter 22/25 - loss 2.38837590 - samples/sec: 118.87 - lr: 0.000488 +2021-03-26 08:53:52,300 epoch 74 - iter 24/25 - loss 2.42492236 - samples/sec: 140.46 - lr: 0.000488 +2021-03-26 08:53:52,709 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:53:52,710 EPOCH 74 done: loss 2.4280 - lr 0.0004883 +2021-03-26 08:53:53,450 DEV : loss 6.722847938537598 - score 0.9094 +2021-03-26 08:53:53,474 BAD EPOCHS (no improvement): 4 +2021-03-26 08:53:53,475 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:53:54,521 epoch 75 - iter 2/25 - loss 2.46175408 - samples/sec: 122.60 - lr: 0.000244 +2021-03-26 08:53:55,421 epoch 75 - iter 4/25 - loss 2.41136295 - samples/sec: 142.55 - lr: 0.000244 +2021-03-26 08:53:56,511 epoch 75 - iter 6/25 - loss 2.28692702 - samples/sec: 117.55 - lr: 0.000244 +2021-03-26 08:53:57,471 epoch 75 - iter 8/25 - loss 2.30924901 - samples/sec: 133.50 - lr: 0.000244 +2021-03-26 08:53:58,455 epoch 75 - iter 10/25 - loss 2.30625987 - samples/sec: 130.32 - lr: 0.000244 +2021-03-26 08:53:59,578 epoch 75 - iter 12/25 - loss 2.36866377 - samples/sec: 114.07 - lr: 0.000244 +2021-03-26 08:54:00,636 epoch 75 - iter 14/25 - loss 2.39279367 - samples/sec: 121.22 - lr: 0.000244 +2021-03-26 08:54:01,651 epoch 75 - iter 16/25 - loss 2.32599847 - samples/sec: 126.27 - lr: 0.000244 +2021-03-26 08:54:02,626 epoch 75 - iter 18/25 - loss 2.35100826 - samples/sec: 131.52 - lr: 0.000244 +2021-03-26 08:54:03,615 epoch 75 - iter 20/25 - loss 2.42343712 - samples/sec: 129.99 - lr: 0.000244 +2021-03-26 08:54:04,589 epoch 75 - iter 22/25 - loss 2.39739620 - samples/sec: 131.60 - lr: 0.000244 +2021-03-26 08:54:05,609 epoch 75 - iter 24/25 - loss 2.42359165 - samples/sec: 125.70 - lr: 0.000244 +2021-03-26 08:54:06,013 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:54:06,014 EPOCH 75 done: loss 2.4093 - lr 0.0002441 +2021-03-26 08:54:06,779 DEV : loss 6.722868919372559 - score 0.9094 +2021-03-26 08:54:06,797 BAD EPOCHS (no improvement): 1 +2021-03-26 08:54:06,797 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:54:07,740 epoch 76 - iter 2/25 - loss 1.97292817 - samples/sec: 136.00 - lr: 0.000244 +2021-03-26 08:54:08,727 epoch 76 - iter 4/25 - loss 2.30610174 - samples/sec: 129.98 - lr: 0.000244 +2021-03-26 08:54:09,685 epoch 76 - iter 6/25 - loss 2.43610533 - samples/sec: 133.72 - lr: 0.000244 +2021-03-26 08:54:10,579 epoch 76 - iter 8/25 - loss 2.45422328 - samples/sec: 143.49 - lr: 0.000244 +2021-03-26 08:54:11,519 epoch 76 - iter 10/25 - loss 2.53655019 - samples/sec: 136.48 - lr: 0.000244 +2021-03-26 08:54:12,621 epoch 76 - iter 12/25 - loss 2.52940021 - samples/sec: 116.63 - lr: 0.000244 +2021-03-26 08:54:13,782 epoch 76 - iter 14/25 - loss 2.54521218 - samples/sec: 110.44 - lr: 0.000244 +2021-03-26 08:54:14,836 epoch 76 - iter 16/25 - loss 2.51659589 - samples/sec: 121.67 - lr: 0.000244 +2021-03-26 08:54:15,836 epoch 76 - iter 18/25 - loss 2.49805910 - samples/sec: 128.13 - lr: 0.000244 +2021-03-26 08:54:16,857 epoch 76 - iter 20/25 - loss 2.47484744 - samples/sec: 125.62 - lr: 0.000244 +2021-03-26 08:54:18,039 epoch 76 - iter 22/25 - loss 2.50383779 - samples/sec: 108.43 - lr: 0.000244 +2021-03-26 08:54:19,048 epoch 76 - iter 24/25 - loss 2.48512718 - samples/sec: 126.98 - lr: 0.000244 +2021-03-26 08:54:19,599 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:54:19,599 EPOCH 76 done: loss 2.4889 - lr 0.0002441 +2021-03-26 08:54:20,401 DEV : loss 6.722756862640381 - score 0.9094 +2021-03-26 08:54:20,428 BAD EPOCHS (no improvement): 2 +2021-03-26 08:54:20,429 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:54:21,570 epoch 77 - iter 2/25 - loss 2.60748661 - samples/sec: 112.30 - lr: 0.000244 +2021-03-26 08:54:22,597 epoch 77 - iter 4/25 - loss 2.72534341 - samples/sec: 124.91 - lr: 0.000244 +2021-03-26 08:54:23,694 epoch 77 - iter 6/25 - loss 2.57711697 - samples/sec: 116.75 - lr: 0.000244 +2021-03-26 08:54:24,640 epoch 77 - iter 8/25 - loss 2.57871893 - samples/sec: 135.53 - lr: 0.000244 +2021-03-26 08:54:25,687 epoch 77 - iter 10/25 - loss 2.60255415 - samples/sec: 122.44 - lr: 0.000244 +2021-03-26 08:54:26,692 epoch 77 - iter 12/25 - loss 2.64264709 - samples/sec: 127.58 - lr: 0.000244 +2021-03-26 08:54:27,695 epoch 77 - iter 14/25 - loss 2.64700404 - samples/sec: 127.73 - lr: 0.000244 +2021-03-26 08:54:28,648 epoch 77 - iter 16/25 - loss 2.62158404 - samples/sec: 134.46 - lr: 0.000244 +2021-03-26 08:54:29,513 epoch 77 - iter 18/25 - loss 2.54049926 - samples/sec: 148.20 - lr: 0.000244 +2021-03-26 08:54:30,385 epoch 77 - iter 20/25 - loss 2.55079408 - samples/sec: 146.97 - lr: 0.000244 +2021-03-26 08:54:31,391 epoch 77 - iter 22/25 - loss 2.53056429 - samples/sec: 127.47 - lr: 0.000244 +2021-03-26 08:54:32,377 epoch 77 - iter 24/25 - loss 2.48710426 - samples/sec: 129.97 - lr: 0.000244 +2021-03-26 08:54:32,814 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:54:32,814 EPOCH 77 done: loss 2.4795 - lr 0.0002441 +2021-03-26 08:54:33,578 DEV : loss 6.7229108810424805 - score 0.9094 +2021-03-26 08:54:33,596 BAD EPOCHS (no improvement): 3 +2021-03-26 08:54:33,597 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:54:34,543 epoch 78 - iter 2/25 - loss 2.53820717 - samples/sec: 135.43 - lr: 0.000244 +2021-03-26 08:54:35,576 epoch 78 - iter 4/25 - loss 2.44104677 - samples/sec: 124.18 - lr: 0.000244 +2021-03-26 08:54:36,596 epoch 78 - iter 6/25 - loss 2.45857209 - samples/sec: 125.67 - lr: 0.000244 +2021-03-26 08:54:37,609 epoch 78 - iter 8/25 - loss 2.51672958 - samples/sec: 126.60 - lr: 0.000244 +2021-03-26 08:54:38,658 epoch 78 - iter 10/25 - loss 2.58631538 - samples/sec: 122.16 - lr: 0.000244 +2021-03-26 08:54:39,592 epoch 78 - iter 12/25 - loss 2.54622651 - samples/sec: 137.35 - lr: 0.000244 +2021-03-26 08:54:40,594 epoch 78 - iter 14/25 - loss 2.48726221 - samples/sec: 127.97 - lr: 0.000244 +2021-03-26 08:54:41,568 epoch 78 - iter 16/25 - loss 2.41305912 - samples/sec: 131.63 - lr: 0.000244 +2021-03-26 08:54:42,555 epoch 78 - iter 18/25 - loss 2.45293010 - samples/sec: 129.87 - lr: 0.000244 +2021-03-26 08:54:43,512 epoch 78 - iter 20/25 - loss 2.43574464 - samples/sec: 133.87 - lr: 0.000244 +2021-03-26 08:54:44,494 epoch 78 - iter 22/25 - loss 2.46249009 - samples/sec: 130.58 - lr: 0.000244 +2021-03-26 08:54:45,578 epoch 78 - iter 24/25 - loss 2.45119586 - samples/sec: 118.29 - lr: 0.000244 +2021-03-26 08:54:46,007 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:54:46,007 EPOCH 78 done: loss 2.4746 - lr 0.0002441 +2021-03-26 08:54:46,815 DEV : loss 6.723141670227051 - score 0.9094 +2021-03-26 08:54:46,840 BAD EPOCHS (no improvement): 4 +2021-03-26 08:54:46,841 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:54:47,790 epoch 79 - iter 2/25 - loss 2.38345897 - samples/sec: 135.18 - lr: 0.000122 +2021-03-26 08:54:48,784 epoch 79 - iter 4/25 - loss 2.43685853 - samples/sec: 128.89 - lr: 0.000122 +2021-03-26 08:54:49,760 epoch 79 - iter 6/25 - loss 2.47693690 - samples/sec: 131.40 - lr: 0.000122 +2021-03-26 08:54:50,768 epoch 79 - iter 8/25 - loss 2.40356702 - samples/sec: 127.29 - lr: 0.000122 +2021-03-26 08:54:51,693 epoch 79 - iter 10/25 - loss 2.47259574 - samples/sec: 138.51 - lr: 0.000122 +2021-03-26 08:54:52,575 epoch 79 - iter 12/25 - loss 2.50643518 - samples/sec: 145.33 - lr: 0.000122 +2021-03-26 08:54:53,494 epoch 79 - iter 14/25 - loss 2.56891007 - samples/sec: 139.52 - lr: 0.000122 +2021-03-26 08:54:54,511 epoch 79 - iter 16/25 - loss 2.45421075 - samples/sec: 126.04 - lr: 0.000122 +2021-03-26 08:54:55,512 epoch 79 - iter 18/25 - loss 2.46415308 - samples/sec: 127.97 - lr: 0.000122 +2021-03-26 08:54:56,580 epoch 79 - iter 20/25 - loss 2.48957347 - samples/sec: 120.02 - lr: 0.000122 +2021-03-26 08:54:57,569 epoch 79 - iter 22/25 - loss 2.53055824 - samples/sec: 129.73 - lr: 0.000122 +2021-03-26 08:54:58,509 epoch 79 - iter 24/25 - loss 2.53782905 - samples/sec: 136.41 - lr: 0.000122 +2021-03-26 08:54:58,882 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:54:58,882 EPOCH 79 done: loss 2.5240 - lr 0.0001221 +2021-03-26 08:54:59,619 DEV : loss 6.723060131072998 - score 0.9094 +2021-03-26 08:54:59,643 BAD EPOCHS (no improvement): 1 +2021-03-26 08:54:59,644 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:55:00,590 epoch 80 - iter 2/25 - loss 2.24184740 - samples/sec: 135.56 - lr: 0.000122 +2021-03-26 08:55:01,578 epoch 80 - iter 4/25 - loss 2.30228198 - samples/sec: 129.73 - lr: 0.000122 +2021-03-26 08:55:02,567 epoch 80 - iter 6/25 - loss 2.19478816 - samples/sec: 129.71 - lr: 0.000122 +2021-03-26 08:55:03,494 epoch 80 - iter 8/25 - loss 2.23243473 - samples/sec: 138.34 - lr: 0.000122 +2021-03-26 08:55:04,476 epoch 80 - iter 10/25 - loss 2.29499611 - samples/sec: 130.42 - lr: 0.000122 +2021-03-26 08:55:05,464 epoch 80 - iter 12/25 - loss 2.33867406 - samples/sec: 129.75 - lr: 0.000122 +2021-03-26 08:55:06,444 epoch 80 - iter 14/25 - loss 2.35571365 - samples/sec: 130.84 - lr: 0.000122 +2021-03-26 08:55:07,476 epoch 80 - iter 16/25 - loss 2.43696856 - samples/sec: 124.27 - lr: 0.000122 +2021-03-26 08:55:08,400 epoch 80 - iter 18/25 - loss 2.44001334 - samples/sec: 138.73 - lr: 0.000122 +2021-03-26 08:55:09,364 epoch 80 - iter 20/25 - loss 2.48000899 - samples/sec: 133.03 - lr: 0.000122 +2021-03-26 08:55:10,349 epoch 80 - iter 22/25 - loss 2.50344019 - samples/sec: 130.23 - lr: 0.000122 +2021-03-26 08:55:11,333 epoch 80 - iter 24/25 - loss 2.47424016 - samples/sec: 130.27 - lr: 0.000122 +2021-03-26 08:55:11,724 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:55:11,725 EPOCH 80 done: loss 2.4586 - lr 0.0001221 +2021-03-26 08:55:12,493 DEV : loss 6.722960472106934 - score 0.9094 +2021-03-26 08:55:12,511 BAD EPOCHS (no improvement): 2 +2021-03-26 08:55:12,512 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:55:13,442 epoch 81 - iter 2/25 - loss 2.23566240 - samples/sec: 137.85 - lr: 0.000122 +2021-03-26 08:55:14,385 epoch 81 - iter 4/25 - loss 2.49205181 - samples/sec: 136.14 - lr: 0.000122 +2021-03-26 08:55:15,306 epoch 81 - iter 6/25 - loss 2.55151534 - samples/sec: 139.09 - lr: 0.000122 +2021-03-26 08:55:16,308 epoch 81 - iter 8/25 - loss 2.50896537 - samples/sec: 128.08 - lr: 0.000122 +2021-03-26 08:55:17,513 epoch 81 - iter 10/25 - loss 2.48709247 - samples/sec: 106.37 - lr: 0.000122 +2021-03-26 08:55:18,463 epoch 81 - iter 12/25 - loss 2.54523623 - samples/sec: 134.97 - lr: 0.000122 +2021-03-26 08:55:19,496 epoch 81 - iter 14/25 - loss 2.48191902 - samples/sec: 124.10 - lr: 0.000122 +2021-03-26 08:55:20,493 epoch 81 - iter 16/25 - loss 2.55448468 - samples/sec: 128.46 - lr: 0.000122 +2021-03-26 08:55:21,492 epoch 81 - iter 18/25 - loss 2.46937637 - samples/sec: 128.27 - lr: 0.000122 +2021-03-26 08:55:22,468 epoch 81 - iter 20/25 - loss 2.45314644 - samples/sec: 131.29 - lr: 0.000122 +2021-03-26 08:55:23,382 epoch 81 - iter 22/25 - loss 2.46111201 - samples/sec: 140.31 - lr: 0.000122 +2021-03-26 08:55:24,310 epoch 81 - iter 24/25 - loss 2.49231903 - samples/sec: 138.22 - lr: 0.000122 +2021-03-26 08:55:24,770 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:55:24,771 EPOCH 81 done: loss 2.4980 - lr 0.0001221 +2021-03-26 08:55:25,498 DEV : loss 6.722949981689453 - score 0.9094 +2021-03-26 08:55:25,520 BAD EPOCHS (no improvement): 3 +2021-03-26 08:55:25,521 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:55:26,455 epoch 82 - iter 2/25 - loss 2.47193301 - samples/sec: 137.23 - lr: 0.000122 +2021-03-26 08:55:27,478 epoch 82 - iter 4/25 - loss 2.63737261 - samples/sec: 125.33 - lr: 0.000122 +2021-03-26 08:55:28,439 epoch 82 - iter 6/25 - loss 2.50588151 - samples/sec: 133.44 - lr: 0.000122 +2021-03-26 08:55:29,414 epoch 82 - iter 8/25 - loss 2.43969005 - samples/sec: 131.43 - lr: 0.000122 +2021-03-26 08:55:30,411 epoch 82 - iter 10/25 - loss 2.53163679 - samples/sec: 128.63 - lr: 0.000122 +2021-03-26 08:55:31,489 epoch 82 - iter 12/25 - loss 2.43317682 - samples/sec: 118.98 - lr: 0.000122 +2021-03-26 08:55:32,472 epoch 82 - iter 14/25 - loss 2.46037332 - samples/sec: 130.34 - lr: 0.000122 +2021-03-26 08:55:33,534 epoch 82 - iter 16/25 - loss 2.42180952 - samples/sec: 120.75 - lr: 0.000122 +2021-03-26 08:55:34,533 epoch 82 - iter 18/25 - loss 2.50896418 - samples/sec: 128.27 - lr: 0.000122 +2021-03-26 08:55:35,466 epoch 82 - iter 20/25 - loss 2.50509016 - samples/sec: 137.43 - lr: 0.000122 +2021-03-26 08:55:36,436 epoch 82 - iter 22/25 - loss 2.48789684 - samples/sec: 132.13 - lr: 0.000122 +2021-03-26 08:55:37,362 epoch 82 - iter 24/25 - loss 2.45878165 - samples/sec: 138.38 - lr: 0.000122 +2021-03-26 08:55:37,842 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:55:37,843 EPOCH 82 done: loss 2.4828 - lr 0.0001221 +2021-03-26 08:55:38,598 DEV : loss 6.72271728515625 - score 0.9094 +2021-03-26 08:55:38,619 BAD EPOCHS (no improvement): 4 +2021-03-26 08:55:38,620 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:55:38,621 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:55:38,621 learning rate too small - quitting training! +2021-03-26 08:55:38,621 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:55:47,863 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:55:47,864 Testing using best model ... +2021-03-26 08:55:47,864 loading file /home/tmp/megahedm/models/multipos/multipos_UDMADAR_4Diale-LEV_EGY_GLF_MGR__fasttext_flairbwfw__64__0.5_202103260834/best-model.pt +2021-03-26 08:55:55,013 0.901 +2021-03-26 08:55:55,014 +Results: +- F-score (micro): 0.8974 +- F-score (macro): 0.5188 +- Accuracy (incl. no class): 0.901 + +By class: + precision recall f1-score support + + INTJ 0.8182 0.9000 0.8571 10 + NOUN 0.9009 0.9402 0.9201 435 + NUM 0.9524 0.8333 0.8889 24 + ADJ 0.8762 0.7603 0.8142 121 + ADP 0.9903 0.9623 0.9761 106 + CCONJ 0.9600 0.9730 0.9664 74 + PROPN 0.9333 0.9333 0.9333 15 + ADV 0.9135 0.8051 0.8559 118 + VERB 0.8852 0.9231 0.9038 117 + PRON 0.9620 0.9465 0.9542 187 + SCONJ 0.8571 0.9474 0.9000 19 + PART 0.9350 0.9791 0.9565 191 + DET 0.9348 0.9149 0.9247 47 + PUNCT 1.0000 1.0000 1.0000 35 + AUX 0.9286 0.9811 0.9541 53 + MENTION 0.9231 1.0000 0.9600 12 + V 0.8571 0.8780 0.8675 82 + FUT_PART+V+PREP+PRON 1.0000 0.0000 0.0000 1 + PROG_PART+V+PRON+PREP+PRON 0.0000 1.0000 0.0000 0 + ADJ+NSUFF 0.6111 0.8462 0.7097 26 + NOUN+NSUFF 0.8182 0.8438 0.8308 64 + PREP+PRON 0.9565 0.9565 0.9565 23 + PUNC 0.9941 1.0000 0.9971 169 + EOS 1.0000 1.0000 1.0000 70 + NOUN+PRON 0.6986 0.8500 0.7669 60 + V+PRON 0.7258 0.8036 0.7627 56 + PART+PRON 1.0000 0.9474 0.9730 19 + PROG_PART+V 0.8333 0.9302 0.8791 43 + DET+NOUN 0.9625 1.0000 0.9809 77 + NOUN+NSUFF+PRON 0.9091 0.7143 0.8000 14 + PROG_PART+V+PRON 0.7083 0.9444 0.8095 18 + PREP+NOUN+NSUFF 0.6667 0.4000 0.5000 5 + NOUN+NSUFF+NSUFF 1.0000 0.0000 0.0000 3 + CONJ 0.9722 1.0000 0.9859 35 + V+PRON+PRON 0.6364 0.5833 0.6087 12 + FOREIGN 0.6667 0.6667 0.6667 3 + PREP+NOUN 0.6316 0.7500 0.6857 16 + DET+NOUN+NSUFF 0.9000 0.9310 0.9153 29 + DET+ADJ+NSUFF 1.0000 0.5714 0.7273 7 + CONJ+PRON 1.0000 0.8750 0.9333 8 + NOUN+CASE 0.0000 0.0000 0.0000 2 + DET+ADJ 1.0000 0.6667 0.8000 6 + PREP 1.0000 0.9718 0.9857 71 + CONJ+FUT_PART+V 0.0000 0.0000 0.0000 1 + CONJ+V 0.6667 0.7500 0.7059 8 + FUT_PART 1.0000 1.0000 1.0000 2 + ADJ+PRON 1.0000 0.0000 0.0000 8 + CONJ+PREP+NOUN+PRON 1.0000 0.0000 0.0000 1 + CONJ+NOUN+PRON 0.3750 1.0000 0.5455 3 + PART+ADJ 1.0000 0.0000 0.0000 1 + PART+NOUN 0.5000 1.0000 0.6667 1 + CONJ+PREP+NOUN 1.0000 0.0000 0.0000 1 + CONJ+NOUN 0.7000 0.7778 0.7368 9 + URL 1.0000 1.0000 1.0000 3 + CONJ+FUT_PART 1.0000 0.0000 0.0000 1 + FUT_PART+V 0.8571 0.6000 0.7059 10 + PREP+NOUN+NSUFF+NSUFF 1.0000 0.0000 0.0000 1 + HASH 1.0000 0.9412 0.9697 17 + ADJ+PREP+PRON 1.0000 0.0000 0.0000 3 + PREP+NOUN+PRON 0.0000 0.0000 0.0000 1 + EMOT 1.0000 0.8889 0.9412 18 + CONJ+PREP 1.0000 0.7500 0.8571 4 + PREP+DET+NOUN+NSUFF 1.0000 0.7500 0.8571 4 + PRON+DET+NOUN+NSUFF 0.0000 1.0000 0.0000 0 + V+PREP+PRON 1.0000 0.0000 0.0000 5 + V+PRON+PREP+PRON 0.0000 1.0000 0.0000 0 + CONJ+NOUN+NSUFF 0.5000 0.5000 0.5000 2 + V+NEG_PART 1.0000 0.0000 0.0000 2 + PREP+DET+NOUN 0.9091 1.0000 0.9524 10 + PREP+V 1.0000 0.0000 0.0000 2 + CONJ+PART 1.0000 0.7778 0.8750 9 + CONJ+V+PRON 1.0000 1.0000 1.0000 5 + PROG_PART+V+PREP+PRON 1.0000 0.5000 0.6667 2 + PREP+NOUN+NSUFF+PRON 1.0000 1.0000 1.0000 1 + ADJ+CASE 1.0000 0.0000 0.0000 1 + PART+NOUN+PRON 1.0000 1.0000 1.0000 1 + PART+V 1.0000 0.0000 0.0000 3 + PART+V+PRON 0.0000 1.0000 0.0000 0 + FUT_PART+V+PRON 0.0000 1.0000 0.0000 0 + FUT_PART+V+PRON+PRON 1.0000 0.0000 0.0000 1 + CONJ+PREP+PRON 1.0000 0.0000 0.0000 1 + CONJ+V+PRON+PREP+PRON 1.0000 0.0000 0.0000 1 + CONJ+V+PREP+PRON 0.0000 1.0000 0.0000 0 + CONJ+DET+NOUN+NSUFF 1.0000 0.0000 0.0000 1 + CONJ+DET+NOUN 0.6667 1.0000 0.8000 2 + CONJ+PREP+DET+NOUN 1.0000 1.0000 1.0000 1 + PREP+PART 1.0000 0.0000 0.0000 2 + PART+V+PRON+NEG_PART 0.3333 0.3333 0.3333 3 + PART+V+NEG_PART 0.3333 0.5000 0.4000 2 + PART+PREP+NEG_PART 1.0000 1.0000 1.0000 3 + PART+PROG_PART+V+NEG_PART 1.0000 0.3333 0.5000 3 + PREP+DET+NOUN+NSUFF+PREP+PRON 1.0000 0.0000 0.0000 1 + PREP+PRON+DET+NOUN 0.0000 1.0000 0.0000 0 + PART+NSUFF 1.0000 0.0000 0.0000 1 + CONJ+PROG_PART+V+PRON 1.0000 1.0000 1.0000 1 + PART+PREP+PRON 1.0000 0.0000 0.0000 1 + CONJ+PART+PREP 1.0000 0.0000 0.0000 1 + NUM+NSUFF 0.6667 0.6667 0.6667 3 + CONJ+PART+V+PRON+NEG_PART 1.0000 1.0000 1.0000 1 + PART+NOUN+NEG_PART 1.0000 1.0000 1.0000 1 + CONJ+ADJ+NSUFF 1.0000 0.0000 0.0000 1 + PREP+ADJ 1.0000 0.0000 0.0000 1 + ADJ+NSUFF+PRON 1.0000 0.0000 0.0000 2 + CONJ+PROG_PART+V 1.0000 0.0000 0.0000 1 +CONJ+PART+PROG_PART+V+PREP+PRON+NEG_PART 1.0000 0.0000 0.0000 1 + CONJ+PART+PREP+PRON+NEG_PART 0.0000 1.0000 0.0000 0 + PREP+PART+PRON 1.0000 0.0000 0.0000 1 + CONJ+ADV+NSUFF 1.0000 0.0000 0.0000 1 + CONJ+ADV 0.0000 1.0000 0.0000 0 + PART+NOUN+PRON+NEG_PART 0.0000 1.0000 0.0000 0 + CONJ+ADJ 1.0000 1.0000 1.0000 1 + + micro avg 0.8977 0.8971 0.8974 2662 + macro avg 0.8002 0.6126 0.5188 2662 + weighted avg 0.9091 0.8971 0.8912 2662 + +2021-03-26 08:55:55,015 ---------------------------------------------------------------------------------------------------- +2021-03-26 08:55:55,015 ----------------------------------------------------------------------------------------------------